Facing imbalanced data - Recommendations for the use of performance metrics

Proceedings - 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, ACII 2013

Volumn , Issue , 2013, Pages 245-251

  Jeni, László A ^a   Cohn, Jeffrey F ^a,b   De La Torre, Fernando ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

Action unit detection; Imbalanced data; Performance metrics; Skew normalization

Indexed keywords

ACTION UNIT; AREA UNDER THE ROC CURVE; IMBALANCED DATA; MACHINE LEARNING COMMUNITIES; PERFORMANCE METRICS; RECEIVER OPERATING CHARACTERISTIC CURVES; SKEW NORMALIZATION; SKEWED DISTRIBUTION;

ASSOCIATION REACTIONS; COMPUTER KEYBOARDS; FACE RECOGNITION; LEARNING ALGORITHMS;

CLASSIFICATION (OF INFORMATION);

EID: 84893287695     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACII.2013.47     Document Type: Conference Paper

References (29)

1. S. Abe: Support vector machines for pattern classification. Springer, (2010)
2. R. Akbani, S. Kwek, N. Japkowicz. Applying support vector machines to imbalanced datasets. Machine Learning: ECML 2004. Springer Berlin Heidelberg, 39-50. (2004)
3. A. B. Ashraf, S. Lucey, J. F. Cohn, T. Chen, Z. Ambadar, K. M. Prkachin, P. E. Solomon: The painful face Pain expression recognition using active appearance models, Image and Vision Computing 27(12), 1788-1796. (2009)
4. N. V. Chawla, N. Japkowicz, A. Kotcz. Editorial: special issue on learning from imbalanced data sets. SIGKDD Explor. Newsl. 6, 1 (June 2004), 1-6. (2004)
5. S. W. Chew, P. J. Lucey, S. Lucey, J. Saragih, J. F. Cohn S. Sridharan. Person-independent facial expression detection using constrained local models. In Proceedings of FG 2011 Facial Expression Recognition and Analysis Challenge, Santa Barbara, CA, (2011)
6. J. Cohen, A coefficient of agreement for nominal scales. Educational and psychological measurement 20(1), 37-46. (1960)
7. J. F. Cohn and F. De la Torre. Automated face analysis for affective computing. In Handbook of Affective Computing, R. A. Calvo, S. K. D'Mello, J. Gratch, and A. Kappas, Eds., ed New York, NY: Oxford, In press.
8. T. Eitrich, B. Lang. Efficient optimization of support vector machine learning parameters for unbalanced datasets. Journal of computational and applied mathematics 196(2). 425-436. (2006)
9. P. Ekman, W. Friesen: Facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)
10. P. Ekman, W. Friesen, J. Hager: Facial action coding system: Research nexus. Network Research Information, Salt Lake City, UT (2002)
11. T. Fawcett. An introduction to ROC analysis. Pattern Recognition Letters, 2006. Elsevier Science Inc., 861-874 (2006)
12. C. Ferri, J. Hernndez-Orallo, R. Modroiu. An experimental comparison of performance measures for classification. Pattern Recognition Letters 30(1). 27-38. (2009)
13. M. Frank, J. Movellan, M. Bartlett, G. Littleworth. RU-FACS- 1 database, Machine Perception Laboratory, U.C. San Diego
14. V. Garcia, R. A. Mollineda, J. S. Sanchez. Index of balanced accuracy: A performance measure for skewed class distributions. Pattern Recognition and Image Analysis. Springer Berlin Heidelberg, 441-448. (2009)
15. V. Garcia, R. A. Mollineda, J. S. Sanchez. Theoretical analysis of a performance measure for imbalanced data. Pattern Recognition (ICPR), 2010 20th International Conference on. IEEE, (2010)
16. H. He, E. A. Garcia. Learning from imbalanced data. Knowledge and Data Engineering, IEEE Transactions on 21(9), 1263-1284. (2009)
17. L. A. Jeni, A. Lorincz, T. Nagy, Zs. Palotai, J. Sebok, Z. Szabo, D. Takacs, 3D shape estimation in video sequences provides high precision evaluation of facial expressions, Image and Vision Computing, 30(10), 785-795 21 February (2012) 3
18. T. Kanade, J. F. Cohn, Y. Tian: Comprehensive database for facial expression analysis. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG'00), Grenoble, France, 46-53. (2000)
19. K. Krippendorff. Estimating the reliability, systematic error and random error of interval data. Educational and Psychological Measurement, 30, 61-70. (1970)
20. K. Krippendorff. Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage (2004)
21. P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion specified expression, In: 3rd IEEE Workshop on CVPR for Human Communicative Behavior Analysis (2010) 1
22. P. Lucey, J. F. Cohn, K. M. Prkachin, P. E. Solomon, I. Matthews. Painful data: The UNBC-McMaster shoulder pain expression archive database. IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011), 57-64, 21-25 March (2011) 1
23. I. Matthews and S. Baker. Active appearance models revisited. Int. J. Comp. Vision, 60(2): 135-164, (2004)
24. R. Ranawana, V. Palade. Optimized Precision-A new measure for classifier performance evaluation. Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. IEEE, (2006)
25. M. A. Sayette, K. G. Creswell, J. D. Dimoff, C. E. Fairbairn, J. F. Cohn, B. W. Heckman, T. R. Kirchner, J. M. Levine, R. L. Moreland: Alcohol and group formation: A multimodal investigation of the effects of alcohol on emotion and social bonding. Psychological Science 23(8). 869-878. (2012)
26. Y. Tang, Y.-Q. Zhang, N.V. Chawla, S. Krasser. SVMs modeling for highly imbalanced classification. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on 39(1). 281-288. (2009)
27. J. Van Hulse, T. M. Khoshgoftaar, A. Napolitano. Experimental perspectives on learning from imbalanced data. Proceedings of the 24th international conference on Machine learning. ACM, (2007)
28. J. Xiao, S. Baker, I. Matthews, T. Kanade: Real-time combined 2D+3D active appearance models. Proceedings of the 2004 IEEE computer society conference on Computer vision and pattern recognition, Washington, D.C., USA, 535-542, (2004) 2
29. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1), 39-58. (2009) 1