Robust online gesture recognition with crowdsourced annotations

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 3187-3220

  Nguyen Dinh, Long Van ^a   Calatroni, Alberto ^a   Tröster, Gerhard ^a  

^a ETH ZURICH   (Switzerland)

Author keywords

Accelerometer sensors; Crowdsourced annotation; Gesture spotting; Longest common subsequence; Template matching methods

Indexed keywords

ACCELEROMETERS; SUPPORT VECTOR MACHINES; TEMPLATE MATCHING;

ACCELEROMETER SENSOR; CROWDSOURCED ANNOTATION; GESTURE SPOTTING; LONGEST COMMON SUBSEQUENCES; TEMPLATE MATCHING METHOD;

GESTURE RECOGNITION;

EID: 84919723308     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (57)

1. J. Aggarwal and M. Ryoo. Human activity analysis: A review. ACM Computing Surveys, 43(3):16:1-16:43, April 2011.
2. J. Alon, V. Athitsos, Q. Yuan, and S. Sclaroff. A unified framework for gesture recognition and spatiotemporal gesture segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(9):1685-1699, Sept 2009.
3. R. Amini and P. Gallinari. Semi-supervised learning with an imperfect supervisor. Knowledge and Information Systems, 8:385-413, November 2005.
4. D. Angluin and P. Laird. Learning from noisy examples. Machine Learning, 2:343-370, April 1988.
5. O. Banos, A. Calatroni, M. Damas, H. Pomares, I. Rojas, H. Sagha, J. del R. Millán, G. Tróster, R. Chavarriaga, and D. Roggen. Kinect=imu? learning mimo signal mappings to automatically translate activity recognition systems across sensor modalities. In Proceedings of the 2012 16th International Symposium on Wearable Computers (ISWC), pages 92-99, 2012.
6. L. Bao and S. S. Intille. Activity recognition from user-annotated acceleration data. In Proceedings of the 2nd International Conference on Pervasive Computing, 2004.
7. B. Bauer and K. Karl-Friedrich. Towards an automatic sign language recognition system using subunits. In International Gesture Workshop on Gesture and Sign Languages in Human-Computer Interaction, pages 64-75. 2002.
8. M. Berchtold, M. Budde, D. Gordon, H. Schmidtke, and M. Beigl. Actiserv: Activity recognition service for mobile phones. In Proceedings of the 2010 14th International Symposium on Wearable Computers (ISWC), pages 1-8, Oct 2010.
9. R. Bowden, D. Windridge, T. Kadir, A. Zisserman, and M. Brady. A linguistic feature vector for the visual interpretation of sign language. In European Conference on Computer Vision, ECCV '04. 2004.
10. C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011.
11. L. Chen, J. Hoey, C. D. Nugent, D. J. Cook, and Z. Yu. Sensor-based activity recognition. In IEEE Transactions on Systems, Man and Cybernetics, 2012.
12. H. Cooper, E.-J. Ong, N. Pugeault, and R. Bowden. Sign language recognition using subunits. Journal of Machine Learning Research, 13(1):2205-2231, July 2012.
13. T. H. Cormen, C. Stein, R. L. Rivest, and C. E. Leiserson. Introduction to Algorithms. 2nd edition, 2001. ISBN 0070131511.
14. A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the EM algorithm. Applied Statistics, 28(1):20-28, 1979.
15. J. Deng and H. Tsui. An HMM-based approach for gesture segmentation and recognition. In Proceedings of the International Conference on Pattern Recognition, ICPR '00, 2000.
16. A. Doan, R. Ramakrishnan, and A. Y. Halevy. Crowdsourcing systems on the world-wide web. Communications of the ACM, 54(4):86-96, April 2011.
17. M. Elmezain, A. Al-Hamadi, and B. Michaelis. Improving hand gesture recognition using 3D combined features. In Proceedings of the 2nd International Conference on Machine Vision, ICMV '09, pages 128-132, Dec 2009.
18. G. Fang, X. Gao, W. Gao, and Y. Chen. A novel approach to automatically extracting basic units from chinese sign language. In Proceedings of the 17th International Conference on Pattern Recognition, volume 4, pages 454-457, Aug 2004.
19. J. Froehlich, M. Y. Chen, S. Consolvo, B. Harrison, and J. A. Landay. Myexperience: A system for in situ tracing and capturing of user feedback on mobile phones. In Proceedings of the 5th International Conference on Mobile Systems, Applications and Services, MobiSys '07, 2007.
20. D. Frolova, H. Stern, and S. Berman. Most probable longest common subsequence for recognition of gesture character input. IEEE Transactions on Cybernetics, 43(3):871-880, June 2013.
21. T.-C. Fu. A review on time series data mining. Engineering Applications of Artificial Intelligence, 24(1):164-181, February 2011.
22. N. Gayar, F. Schwenker, and G. Palm. A study of the robustness of KNN classifiers trained using soft labels. In Artificial Neural Networks in Pattern Recognition, volume 4087. 2006.
23. I. Guyon, J. Makhoul, R. Schwartz, and V. Vapnik. What size test set gives good error rate estimates? IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (1):52-64, Jan 1998.
24. J. Hao and T. Shibata. Digit-writing hand gesture recognition by hand-held camera motion analysis. In Proceedings of the 3rd International Conference on Signal Processing and Communication Systems, ICSPCS '09, pages 1-5, Sept 2009.
25. B. Hartmann and N. Link. Gesture recognition with inertial sensors and optimized DTW prototypes. In Proceedings of the 2010 IEEE International Conference on Systems Man and Cybernetics (SMC), 2010.
26. Z. He, L. Jin, L. Zhen, and J. Huang. Gesture recognition based on 3D accelerometer for cell phones interaction. In IEEE Asia Pacific Conference on Circuits and Systems (APCCAS), pages 217-220, Nov 2008.
27. J. Howe. The Rise of Crowdsourcing. (accessed July 20, 2010), jun 2006. URL http://www.wired.com/wired/archive/14.06/crowds.html.
28. P. G. Ipeirotis, F. Provost, and J. Wang. Quality management on Amazon Mechanical Turk. In Proceedings of the ACM SIGKDD Workshop on Human Computation, HCOMP '10, pages 64-67, 2010.
29. H. Junker, O. Amft, P. Lukowicz, and G. Tröster. Gesture spotting with body-worn inertial sensors to detect user activities. Pattern Recognition, 41(6), 2008.
30. C. Keskin, A. Cemgil, and L. Akarun. DTW based clustering to improve hand gesture recognition. In Proceedings of the 2nd International Conference on Human Behavior Unterstanding, HBU'11, pages 72-81. 2011.
31. A. Kittur, E. H. Chi, and B. Suh. Crowdsourcing user studies with mechanical turk. In Proceedings of the Twenty-sixth SIGCHI Conference on Human Factors in Computing Systems, CHI '08, pages 453-456, 2008.
32. M. H. Ko, G. West, S. Venkatesh, and M. Kumar. Online context recognition in multisensor systems using dynamic time warping. In Proceedings of the Intelligent Sensors, Sensor Networks and Information Processing Conference, 2005.
33. W. S. Lasecki, Y. C. Song, H. Kautz, and J. P. Bigham. Real-time crowd labeling for deployable activity recognition. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, CSCW '13, pages 1203-1212, 2013.
34. N. D. Lawrence and B. Schölkopf. Estimating a kernel sher discriminant in the presence of label noise. In Proceedings of the Eighteenth International Conference on Machine Learning, ICML '01, pages 306-313, 2001.
35. H.-K. Lee and J. H. Kim. An hmm-based threshold model approach for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(10):961-973, October 1999.
36. L.-V. Nguyen-Dinh, D. Roggen, A. Calatroni, and G. Tröster. Improving online gesture recognition with template matching methods in accelerometer data. In Proceedings of the 12th International Conference on Intelligent Systems Design and Applications (ISDA), 2012.
37. L.-V. Nguyen-Dinh, U. Blanke, and G. Tröster. Towards scalable activity recognition: Adapting zero-effort crowdsourced acoustic models. In Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, MUM '13, 2013a.
38. L.-V. Nguyen-Dinh, M. Rossi, U. Blanke, and G. Tröster. Combining crowd-generated media and personal data: Semi-supervised learning for context recognition. In Proceedings of the 1st ACM International Workshop on Personal Data Meets Distributed Multimedia, PDM '13, 2013b.
39. L.-V. Nguyen-Dinh, C. Waldburger, D. Roggen, and G. Tröster. Tagging human activities in video by crowdsourcing. In Proceedings of the ACM International Conference on Multimedia Retrieval, ICMR '13, 2013c.
40. L.-V. Nguyen-Dinh, A. Calatroni, and G. Tröster. Towards a unified system for multimodal activity spotting: Challenges and a proposal. In Proceedings of the ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, UbiComp '14 Adjunct, 2014.
41. N. Ravi, N. D, P. Mysore, and M. L. Littman. Activity recognition from accelerometer data. In Proceedings of the Seventeenth Conference on Innovative Applications of Artificial Intelligence(IAAI), pages 1541-1546. AAAI Press, 2005.
42. V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from crowds. Journal of Machine Learning Research, 11, 2010.
43. D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. Forster, G. Troster, and et al. Collecting complex activity data sets in highly rich networked sensor environments. In Proceedings of the 7th International Conference on Networked Sensing Systems. IEEE Press, 2010.
44. M. Rossi, O. Amft, and G. Tröster. Recognizing daily life context using web-collected audio data. In Proceedings of the 16th IEEE International Symposium on Wearable Computers (ISWC), June 2012.
45. T. Schlömer, B. Poppinga, N. Henze, and S. Boll. Gesture recognition with a Wii controller. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, 2008.
46. V. S. Sheng, F. Provost, and P. G. Ipeirotis. Get another label? improving data quality and data mining using multiple, noisy labelers. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '08, 2008.
47. H. Stern, M. Shmueli, and S. Berman. Most discriminating segment - longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification. Pattern Recognition Letters, 34(15):1980-1989, 2013.
48. T. Stiefmeier, D. Roggen, G. Ogris, P. Lukowicz, and G. Tröster. Wearable activity tracking in car manufacturing. IEEE Pervasive Computing Magazine, 7(2), 2008.
49. M. Stikic, D. Larlus, S. Ebert, and B. Schiele. Weakly supervised recognition of daily life activities with wearable sensors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(12):2521-2537, December 2011.
50. K. Van Laerhoven, D. Kilian, and B. Schiele. Using rhythm awareness in long-term activity recognition. In Proceedings of the IEEE International Symposium on Wearable Computers (ISWC), October 2008.
51. C. Vogler and D. N. Metaxas. Toward scalability in ASL recognition: Breaking down signs into phonemes. In Gesture-Based Communication in Human-Computer Interaction, Lecture Notes in Computer Science, pages 211-224, 1999.
52. J. A.Ward, P. Lukowicz, and H. W. Gellersen. Performance metrics for activity recognition. ACM Transactions on Intelligent Systems and Technology, 2(1), January 2011.
53. A. Wilson and A. Bobick. Parametric hidden markov models for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, 1999.
54. J. Wu, G. Pan, D. Zhang, G. Qi, and S. Li. Gesture recognition with a 3-D accelerometer. In Proceedings of the 6th International Conference on Ubiquitous Intelligence and Computing, UIC '09, pages 25-38, 2009.
55. H.-S. Yoon, J. Soh, Y. J. Bae, and H. S. Yang. Hand gesture recognition using combined features of location, angle and velocity. Pattern Recognition, 34, 2001.
56. M.-C. Yuen, I. King, and K.-S. Leung. A survey of crowdsourcing systems. In Social-Com/PASSAT, pages 766-773, 2011.
57. P. Zappi, C. Lombriser, T. Stiefmeier, E. Farella, D. Roggen, L. Benini, and G. Tröster. Activity recognition from on-body sensors: Accuracy-power trade-o by dynamic sensor selection. In Proceedings of the 5th European Conference on Wireless Sensor Networks, EWSN'08, pages 17-33, 2008.