Machine learning classification of design team members' body language patterns for real time emotional state detection

Design Studies

Volumn 39, Issue , 2015, Pages 100-127

  Behoora, Ishan ^a   Tucker, Conrad S ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

computational models; design activity; information processing; team work; user behavior

Indexed keywords

BEHAVIORAL RESEARCH; DATA PROCESSING; MACHINE LEARNING; WEARABLE TECHNOLOGY;

AUTOMATED DETECTION; COMPUTATIONAL MODEL; DESIGN ACTIVITY; DESIGN TEAM INTERACTIONS; ENGINEERING DESIGN PROCESS; MACHINE LEARNING CLASSIFICATION; TEAM WORK; USER BEHAVIORS;

DESIGN;

EID: 84931037222     PISSN: 0142694X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.destud.2015.04.003     Document Type: Article

References (92)

1. ABC Nightline - IDEO Shopping Cart. 2009. https://www.youtube.com/watch?v=M66ZU2PCIcM&feature=youtube-gdata-player.
2. S. Ackroyd The oxford handbook of work and organization 2006 Oxford Handbooks Online
3. D.S. Alexiadis, P. Kelly, P. Daras, Noel E. O'Connor, T. Boubekeur, and M.B. Moussa Evaluating a dancer's performance using kinect-based skeleton tracking Proceedings of the 19th ACM international conference on multimedia 2011 ACM 659 662
4. C. Ambroise, and G.J. McLachlan Selection bias in gene extraction on the basis of microarray gene-expression data Proceedings of the National Academy of Sciences 99 10 2002 6562 6566
5. Y. Amit, and D. Geman Shape quantization and recognition with randomized trees Neural Computation 9 7 1997 1545 1588
6. H. Aviezer, Y. Trope, and A. Todorov Body cues, not facial expressions, discriminate between intense positive and negative emotions Science 338 6111 2012 1225 1229
7. R.S. Baker, S.K. D'Mello, M.M.T. Rodrigo, and A.C. Graesser Better to be frustrated than bored: the incidence, persistence, and impact of learners' cognitive-affective states during interactions with three different computer-based learning environments International Journal of Human-Computer Studies 68 4 2010 223 241
8. D. Baker, S.J. Ryan, S.M. Gowda, M. Wixon, J. Kalka, and A.Z. Wagner Towards sensor-free affect detection in cognitive tutor algebra Stamper Proceedings of the 5th international conference on educational data mining, Chania, Greece, June 19, 21 2012 International Educational Data Mining Society 978-1-74210-276-4 126
9. C. Barker, N. Pistrang, and R. Elliott Research methods in clinical psychology 2002 Wiley London
10. R.L. Birdwhistell Kinesics and context: Essays on body motion communication 2010 University of Pennsylvania Press
11. K. Bodine, and F. Gemperle Effects of functionality on perceived comfort of wearables 2012 16th international symposium on wearable computers 2003 IEEE Computer Society 57-57
12. M.T. Brannick, and C. Prince An overview of team performance measurement Team Performance Assessment and Measurement. Theory, Methods, and Applications 3 1997
13. L. Breiman Random forests Machine Learning 45 1 2001 5 32
14. L.L. Bucciarelli Designing engineers 1994 MIT Press
15. N. Burba, M. Bolas, D.M. Krum, and E.A. Suma Unobtrusive measurement of subtle nonverbal behaviors with the Microsoft Kinect Virtual reality short papers and posters (VRW), 2012 IEEE 2012 IEEE 1 4
16. M.A. Busseri, and J.M. Palmer Improving teamwork: the effect of self-assessment on construction design teams Design Studies 21 3 2000 223 238
17. S. Celebi, A.S. Aydin, T.T. Temiz, and T. Arici Gesture recognition using Skeleton data with weighted dynamic time warping VISAPP 1 2013 620 625
18. H. Chung, and H.-D. Yang Conditional random field-based gesture recognition with depth information Optical Engineering 52 1 2013 017201-017201
19. D.A. Coker, and J. Burghoon The nature of conversational involvement and nonverbal encoding patterns Human Communication Research 13 4 1987 463 494
20. T. Cover, and P. Hart Nearest neighbor pattern classification Information Theory, IEEE Transactions on 13 1 1967 21 27
21. S. Craig, A. Graesser, J. Sullins, and B. Gholson Affect and learning: an exploratory look into the role of affect in learning with autotutor Journal of Educational Media 29 3 2004 241 250
22. N. Cross, and A.C. Cross Observations of teamwork and social processes in design Design Studies 16 2 1995 143 170
23. S. D'Mello, and A. Graesser Automatic detection of learner's affect from gross body language Applied Artificial Intelligence 23 2 2009 123 150
24. S. D'Mello, and A. Graesser The half-life of cognitive-affective states during complex learning ahead-of-p (ahead-of-p) Cognition & Emotion 2011 1 10 10.1080/02699931.2011.613668
25. Depth Sensors Comparison - iPiSoft Wiki." 2015. Accessed February 14 http://wiki.ipisoft.com/Depth-Sensors-Comparison.
26. P. Domingos, and M. Pazzani On the optimality of the simple Bayesian classifier under zero-one loss Machine Learning 29 2-3 1997 103 130
27. C.L. Dym, A.M. Agogino, O. Eris, D.D. Frey, and L.J. Leifer Engineering design thinking, teaching, and learning Journal of Engineering Education 94 1 2005 103 120
28. J.D. Eastwood, A. Frischen, M.J. Fenske, and D. Smilek The unengaged mind defining boredom in terms of attention Perspectives on Psychological Science 7 5 2012 482 495
29. A.C. Edmondson, and I.M. Nembhard Product development and learning in project teams: the challenges are the benefits∗ Journal of Product Innovation Management 26 2 2009 123 138
30. K. Fisher Leading self-directed work teams, a guide to developing new team leadersip skills 1993 McGraw-HillFisherLeading Self-Directed Work Teams: A Guide to Developing New Team Leadership skills1993 New York
31. A. Furnham, and E. Petrova Body language in business: Decoding the signals 2010 Palgrave Macmillan
32. X. Gao, and D. Maurer Influence of intensity on children's sensitivity to happy, sad, and fearful facial expressions Journal of Experimental Child Psychology 102 4 2009 503 521
33. N. Gillian, R.B. Knapp, and S. O'Modhrain Recognition of multivariate temporal musical gestures using N-dimensional dynamic time warping Proc of the 11th int'l conference on new interfaces for musical expression 2011
34. G. Goldschmidt, and D. Tatsa How good are good ideas? Correlates of design creativity Design Studies 26 6 2005 593 611
35. A.C. Graesser, B. McDaniel, P. Chipman, A. Witherspoon, S. D'Mello, and B. Gholson Detection of emotions during learning with autotutor Proceedings of the 28th annual meetings of the cognitive science society 2006 Citeseer 285 290
36. G. Guo, H. Wang, D. Bell, Y. Bi, and K. Greer KNN model-based approach in classification On the move to meaningful internet systems 2003: CoopIS, DOA, and ODBASE 2003 Springer 986 996
37. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten The WEKA data mining software: an update ACM SIGKDD Explorations Newsletter 11 1 2009 10 18
38. T.K. Ho Random Decision Forests Document analysis and recognition, 1995., proceedings of the third international conference on 1 1995 IEEE 278 282
39. J. Hong Considering privacy issues in the context of google glass Commun. ACM 56 11 2013 10 11
40. D. Jensen, J. Wood, and K. Wood Hands-on activities, interactive multimedia and improved team dynamics for enhancing mechanical engineering curricula International Journal of Engineering Education 19 6 2003 874 884
41. A. Kapoor, W. Burleson, and R.W. Picard Automatic Prediction of Frustration International Journal of Human-Computer Studies 65 8 2007 724 736
42. A. Kapoor, and R.W. Picard A real-time head nod and shake detector Proceedings of the 2001 workshop on perceptive user interfaces 2001 ACM 1 5
43. A. Kapoor, and R.W. Picard Multimodal Affect Recognition in Learning Environments Proceedings of the 13th annual ACM international conference on multimedia 2005 ACM 677 682
44. M. Kassner, W. Patera, and A. Bulling Pupil: An open source platform for pervasive eye tracking and mobile gaze-based interaction Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication 2014 ACM 1151 1160
45. J.R. Kelly, and S.G. Barsade Mood and emotions in small groups and work teams Organizational Behavior and Human Decision Processes 86 1 2001 99 130
46. K. Khoshelham, and S.O. Elberink Accuracy and resolution of kinect depth data for indoor mapping applications Sensors 12 2 2012 1437 1454
47. R. Klimoski, and S. Mohammed Team mental model: construct or metaphor? Journal of Management 20 2 1994 403 437
48. R. Kohavi A study of cross-validation and bootstrap for accuracy estimation and model selection Ijcai 14 1995 1137 1145
49. I. Kononenko Comparison of inductive and naive Bayesian learning approaches to automatic knowledge acquisition Current Trends in Knowledge Adquisition 1990 190 197
50. S.B. Kotsiantis Supervised machine learning: A review of classification techniques 2007 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.95.9683
51. S. Lang, M. Block, and R. Rojas Sign language recognition using kinect Artificial intelligence and soft computing 2012 Springer 394 402
52. C.A. Le Dantec, and E.Yi.-L. Do The mechanisms of value transfer in design meetings Design Studies 30 2 2009 119 137
53. P. Li, X. Hu, and X. Wu Mining concept-drifting data streams with multiple semi-random decision trees Advanced data mining and applications 2008 Springer 733 740
54. T.-S. Lim, W.-Y. Loh, and Yu.-S. Shih A comparison of prediction accuracy, Complexity, and training time of thirty-three old and new classification algorithms Machine Learning 40 3 2000 203 228
55. Y.-C. Liu, A. Chakrabarti, and T. Bligh Towards an 'ideal' approach for concept generation Design Studies 24 4 2003 341 355
56. G. Manohar, and C. Tucker A privacy preserving data mining methodology for dynamically predicting emerging human threats ASME 2013 international design engineering technical conferences and computers and information in engineering conference 2013 American Society of Mechanical Engineers V02AT02A069-V02AT02A069
57. G.C. Marchand, and A.P. Gutierrez The role of emotion in the learning process: comparisons between online and face-to-face learning settings The Internet and Higher Education 15 3 2012 150 160
58. M. Marneweck, A. Loftus, and G. Hammond Psychophysical measures of sensitivity to facial expression of emotion Frontiers in Psychology 4 2013
59. L. Miranda, T. Vieira, D. Martinez, T. Lewiner, A.W. Vieira, and M.F.M. Campos Real-time gesture recognition from depth data through key poses learning and decision forests Graphics, patterns and images (SIBGRAPI), 2012 25th SIBGRAPI conference on 2012 IEEE 268 275
60. T.M. Mitchell Machine learning. 1997 1997 McGraw Hill 45 Burr Ridge, IL
61. M.K. Mount, M.R. Barrick, and G.L. Stewart Five-factor model of personality and performance in jobs involving interpersonal interactions Human Performance 11 2-3 1998 145 165
62. D. Munoz, and C. Tucker Assessing students' emotional states: an approach to identify lectures that provide an optimal learning experience Proceedings of the 2014 ASME IDETC/CIE, DETC2014-34782 August 17-20 2014 Buffalo, NY
63. S.K. Murthy Automatic construction of decision trees from data: a multi-disciplinary survey Data Mining and Knowledge Discovery 2 4 1998 345 389
64. G.M. Olson, J.S. Olson, M.R. Carter, and M. Storrosten Small group design meetings: an analysis of collaboration Human-Computer Interaction 7 4 1992 347 374
65. J. Panksepp Affective neuroscience: The foundations of human and animal emotions 1998 Oxford University Press
66. G.M. Parker Cross-functional teams: Working with allies, enemies, and other strangers 2003 John Wiley & Sons
67. B. Pease, and A. Pease The definitive book of body language 2008 Bantam
68. R.R. Provine Yawning as a stereotyped action pattern and releasing stimulus Ethology 72 2 1986 109 122
69. Projects | Mobile Kinect | BIG." 2015. Accessed February 16. http://big.cs.bris.ac.uk/projects/mobile-kinect.
70. J.R. Quinlan C4. 5: Programs for machine learning 2014 Elsevier
71. M. Raptis, D. Kirovski, and H. Hoppe Real-time classification of dance gestures from skeleton animation Proceedings of the 2011 ACM SIGGRAPH/Eurographics symposium on computer animation 2011 ACM 147 156
72. P. Refaeilzadeh, L. Tang, and H. Liu Cross-Validation Encyclopedia of database systems 2009 Springer 532 538
73. R.R. Reilly, G.S. Lynn, and Z.H. Aronson The role of personality in new product development team performance Journal of Engineering and Technology Management 19 1 2002 39 58
74. J.J. Shah, S.V. Kulkarni, and N. Vargas-Hernandez Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments Journal of Mechanical Design 122 4 2000 377 384
75. L. Shen, M. Wang, and R. Shen Affective e-learning: using 'emotional' data to improve learning in pervasive learning environment Educational Technology & Society 12 2 2009 176 189
76. J. Shotton, T. Sharp, A. Kipman, A. Fitzgibbon, M. Finocchio, A. Blake, and M. Cook Real-time human pose recognition in parts from single depth images Communications of the ACM 56 1 2013 116 124
77. A. Spagnolli, E. Guardigli, V. Orso, A. Varotto, and L. Gamberini Measuring user acceptance of wearable symbiotic devices: validation study across application scenarios Symbiotic interaction 2014 Springer 87 98
78. G. Stasser, and B. Dietz-Uhler Collective choice, judgment, and problem solving M.A. Hogg, R.S. Tindale, Blackwell handbook of social psychology: Group processes 2001 Blackwell Publishers Ltd 31 55 http://onlinelibrary.wiley.com/doi/10.1002/9780470998458.ch2/summary
79. J. Stempfle, and P. Badke-Schaub Thinking in design teams - an analysis of team communication Design Studies 23 5 2002 473 496 10.1016/S0142-694X(02)00004-2
80. R. Tibshirani, T. Hastie, B. Narasimhan, and G. Chu Diagnosis of multiple cancer types by shrunken centroids of gene expression Proceedings of the National Academy of Sciences2 99 10 2002 6567 6572
81. C.S. Tucker, and H.M. Kim Trend mining for predictive product design Journal of Mechanical Design 133 11 2011 111008
82. A. Verikas, A. Gelzinis, and M. Bacauskiene Mining data with random forests: a survey and results of new tests Pattern Recognition 44 2 2011 330 349 10.1016/j.patcog.2010.08.011
83. H.G. Wallbott Bodily expression of emotion European Journal of Social Psychology 28 6 1998 879 896 10.1002/(SICI)1099-0992(1998110)28:6<879::AID-EJSP901>3.0.CO;2-W
84. J.-Y. Wang, and H.-M. Lee Recognition of human actions using motion capture data and support vector machine WRI world congress on software engineering, 2009. WCSE'09 1 2009 234 238 10.1109/WCSE.2009.354
85. D. Wettschereck, D.W. Aha, and T. Mohri A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms Artificial Intelligence Review 11 1-5 1997 273 314 10.1023/A:1006593614256
86. A.S. Won, and J.N. Bailenson Automatic detection of nonverbal behavior predicts learning in dyadic interactions Affective Computing, IEEE Transactions on 5 2 2014 112 125 10.1109/TAFFC.2014.2329304
87. L. Xia, C.-C. Chen, and J.K. Aggarwal View invariant human action recognition using histograms of 3D joints 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) 2012 20 27 10.1109/CVPRW.2012.6239233
88. Xtion PRO - Overview." 2015. Accessed February 14. http://www.asus.com/Multimedia/Xtion-PRO/.
89. M.C. Yang Consensus and single leader decision-making in teams using structured design methods Design Studies 31 4 2010 345 362 10.1016/j.destud.2010.03.002
90. Z. Ye, Y. Li, A. Fathi, Y. Han, A. Rozga, and G.D. Abowd Detecting eye contact using wearable eye-tracking glasses Proceedings of the 2012 ACM conference on ubiquitous computing 2012 ACM 699 704
91. Z. Zafrulla, H. Brashear, T. Starner, H. Hamilton, and P. Presti American sign language recognition with the kinect Proceedings of the 13th international conference on multimodal interfaces 2011 ACM 279 286
92. H. Zhang, and J. Su Naive Bayesian classifiers for ranking J.-F. Boulicaut, F. Esposito, F. Giannotti, D. Pedreschi, Machine learning: ECML 2004 Lecture notes in computer science 3201 2004 Springer Berlin Heidelberg 501 512 http://link.springer.com/chapter/10.1007/978-3-540-30115-8-46