In-air gestures around unmodified mobile devices

UIST 2014 - Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology

Volumn , Issue , 2014, Pages 319-330

  Song, Jie ^a   Sörös, Gábor ^a   Pece, Fabrizio ^a   Fanello, Sean Ryan ^b   Izadi, Shahram ^b   Keskin, Cem ^b   Hilliges, Otmar ^a  

^a ETH ZURICH   (Switzerland)

^b MICROSOFT RESEARCH   (United Kingdom)

Author keywords

Gesture recognition; HCI; Mobile computing; Mobile gestures; Mobile interaction; Random forests

Indexed keywords

DECISION TREES; GESTURE RECOGNITION; HUMAN COMPUTER INTERACTION; MACHINE LEARNING; MOBILE COMPUTING; RANDOM FORESTS; USER INTERFACES;

MEMORY FOOTPRINT; MOBILE GESTURES; MOBILE INTERACTION; MODEL PARAMETERS; RECOGNITION ACCURACY; TASK MANAGEMENT; USER EVALUATIONS; VARYING LIGHTING;

LEARNING ALGORITHMS;

EID: 84911995436     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2642918.2647373     Document Type: Conference Paper

References (40)

1. Ashbrook, D., Baudisch, P., and White, S. Nenya : Subtle and Eyes-Free Mobile Input with a Magnetically-tracked Finger Ring. In Proc. ACM SIGCHI (2011), 2043-2046.
2. Breiman, L. Random Forests. Mach. Learn. 45, 1 (Oct. 2001), 5-32.
3. Butler, A., Izadi, S., and Hodges, S. SideSight: multi-touch interaction around small devices. In Proc. ACM UIST, UIST '08 (2008), 201-204.
4. Chen, K.-Y., Lyons, K., White, S., and Patel, S. uTrack: 3D input using two magnetic sensors. In Proc. ACM UIST (2013).
5. Chen, X. A., Grossman, T., Wigdor, D., and Fitzmaurice, G. Duet: Exploring Joint Interactions on a Smart Phone and a Smart Watch. In Proc. ACM CHI'14 (2014).
6. Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., and Twombly, X. Vision-based hand pose estimation: A review. Comput. Vis. Image Underst. (2007).
7. Goc, M. L., Taylor, S., Izadi, S., and Keskin, C. A Low-cost Transparent EF Sensor for 3D Interaction on Mobile Devices. In Proc. ACM CHI (2014).
8. Guiard, Y. Asymmetric Division of Labor in Human Skilled Bimanual Action. J. Mot. Behav. 19, 4 (1987).
9. Gustafson, S., Bierwirth, D., and Baudisch, P. Imaginary interfaces. In Proc. ACM UIST (2010), 3-12.
10. Harrison, C., Benko, H., and Wilson, A. D. OmniTouch: Wearable Multitouch Interaction Everywhere. In Proc. ACM UIST, ACM (2011), 441-450.
11. Harrison, C., Tan, D., and Morris, D. Skinput: Appropriating the Body As an Input Surface. In Proc. ACM CHI (2010), 453-462.
12. Hilliges, O., Kim, D., Izadi, S., and Weiss, M. HoloDesk: Direct 3D Interactions with a Situated See-Through Display. In Proc. ACM CHI (2012).
13. Hinckley, K., and Song, H. Sensor synaesthesia: touch in motion, and motion in touch. In Proc. ACM CHI (2011).
14. Hinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H., and Buxton, B. Pen + touch = new tools. In Proc. ACM UIST (2010).
15. Hwang, S., Ahn, M., and Wohn, K.-y. MagGetz: customizable passive tangible controllers on and around mobile devices. In Proc. ACM UIST (2013).
16. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., and Fitzgibbon, A. KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera. In Proc. ACM UIST (2011), 559-568.
17. Jones, B., Sodhi, R., Forsyth, D., Bailey, B., and Maciocci, G. Around device interaction for multiscale navigation. In Proc. of MobileHCI (2012).
18. Kakumanu, P., Makrogiannis, S., and Bourbakis, N. A survey of skin-color modeling and detection methods. Pattern Recognit. 40, 3 (2007), 1106-1122.
19. Keskin, C., Kirac, F., Kara, Y. E., Akarun, L., and Kiraç, F. Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests. In Proc. ECCV (Berlin, Heidelberg, 2012).
20. Ketabdar, H., Roshandel, M., and Yüksel, K. A. Towards using embedded magnetic field sensor for around mobile device 3D interaction. In Proc. MobileHCI (2010).
21. Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., and Olivier, P. Digits: Freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In Proc. ACM UIST (2012).
22. Kim, D., Izadi, S., Dostal, J., Rhemann, C., Keskin, C., Zach, C., Shotton, J., Large, T., Bathiche, S., Nießner, M., Butler, A., Fanello, S., and Pradeep, V. RetroDepth: 3D Silhouette Sensing for High-Precision Input On and Above Physical Surfaces. In Proc. ACM CHI (2014).
23. Kim, J., He, J., Lyons, K., and Starner, T. The Gesture Watch: A Wireless Contact-free Gesture based Wrist Interface. In IEEE ISWC (2007), 1-8.
24. Kratz, S., and Rohs, M. HoverFlow: expanding the design space of around-device interaction. In Proc. MobileHCI, ACM (2009), 4.
25. Lee, S. C., Li, B., and Starner, T. AirTouch: Synchronizing In-air Hand Gesture and On-body Tactile Feedback. In IEEE ISWC (2011), 3-10.
26. Mistry, P., and Maes, P. SixthSense: a wearable gestural interface. In ACM SIGGRAPH ASIA Sketches (2009).
27. Molyneaux, D., Izadi, S., Kim, D., Hilliges, O., Hodges, S., Cao, X., Butler, A., and Gellersen, H. Interactive environment-aware handheld projectors for pervasive computing spaces. In Pervasive Comput. (2012).
28. Niikura, T., Hirobe, Y., Cassinelli, A., Watanabe, Y., Komuro, T., and Ishikawa, M. In-air typing interface for mobile devices with vibration feedback. In ACM SIGGRAPH Emerg. Technol. (2010).
29. Ogata, M., Sugiura, Y., Makino, Y., Inami, M., and Imai, M. SenSkin: adapting skin as a soft interface. In Proc. ACM UIST (New York, New York, USA, 2013).
30. Oikonomidis, I., Kyriazis, N., and Argyros, A. A. Tracking the Articulated Motion of Two Strongly Interacting Hands. In IEEE CVPR (Providence, 2012).
31. Rekimoto, J. GestureWrist and GesturePad: unobtrusive wearable interaction devices. In IEEE ISWC (2001).
32. Saponas, S., Tan, D., Morris, D., Balakrishnan, R., Turner, J., and Landay, J. Enabling always-available input with muscle-computer interfaces. In Proc. ACM UIST (2009).
33. Shotton, J., Fitzgibbon, A. W., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. Real-time human pose recognition in parts from single depth images. Commun. ACM 56, 1 (2013), 116-124.
34. Song, H., Guimbretiere, F., Grossman, T., and Fitzmaurice, G. MouseLight: Bimanual Interactions on Digital Paper Using a Pen and a Spatially-aware Mobile Projector. In Proc. ACM CHI (2010).
35. Sridhar, S., Oulasvirta, A., and Theobalt, C. Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data. In IEEE ICCV (2013).
36. Starner, T., Auxier, J., Ashbrook, D., and Gandy, M. The gesture pendant: a self-illuminating, wearable, infrared computer vision system for home automation control and medical monitoring. In IEEE ISWC (2000), 87-94.
37. Starner, T., Weaver, J., and Pentland, A. A wearable computer based American sign language recognizer. In IEEE ISWC, ISWC '97 (1997).
38. Tang, D., Yu, T., and Kim, T. Real-time Articulated Hand Pose Estimation using Semi-supervised Transductive Regression Forests. In IEEE ICCV (2013).
39. Taylor, S., Keskin, C., Hilliges, O., Izadi, S., and Helmes, J. Type-Hover-Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard. In Proc. ACM CHI (2014).
40. Wigdor, D., Forlines, C., Baudisch, P., Barnwell, J., and Shen, C. LucidTouch: A See-Through Mobile Device. In Proc. ACM UIST, ACM Press (2007).