The perceptive workbench: Computer-vision-based gesture tracking, object tracking, and 3D reconstruction for augmented desks

Machine Vision and Applications

Volumn 14, Issue 1, 2003, Pages 59-71

  Starner, Thad ^a   Leibe, Bastian ^b   Minnen, David ^a   Westyn, Tracy ^a   Hurst, Amy ^a   Weeks, Justin ^a  

^a Georgia Institute of Technology   (United States)

^b ETH ZURICH   (Switzerland)

Author keywords

3 D object reconstruction; Computer vision; Gesture; Tracking; Virtual reality

Indexed keywords

IMAGE RECONSTRUCTION; OBJECT RECOGNITION; OPTIMIZATION; THREE DIMENSIONAL; TRACKING (POSITION); USER INTERFACES; VIRTUAL REALITY;

3-D OBJECT RECONSTRUCTION; AUGMENTED REALITY GAMING; GESTURE TRACKING; OBJECT TRACKING;

COMPUTER VISION;

EID: 4544285478     PISSN: 09328092     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00138-002-0096-8     Document Type: Article

References (45)

1. Bimber O (1999) Gesture controlled object interaction: A virtual table case study. In: Proceedings of the 7th International Conference in Central Europe on Computer Graphics, Visualization, and Interactive Digital Media (WSCG'99), vol 1, Plzen, Czech Republic. Univ. of West Bohemia Press
2. Bobick A, Intille S, Davis J, Baird F, Pinhanez C, Campbell L, Ivanov Y, Wilson A (1999) The kidsroom: A perceptually-based interactive and immersive story environment. PRESENCE: Teleop Virtual Environ 8(4):367-391
3. Bolt R, Herranz E (1992) Two-handed gesture in multi-modal natural dialogue. In: Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'92), pp 7-14. ACM Press, New York
4. Cignoni P, Rocchini C, Scopigno R (1998) Metro: Measuring error on simplified surfaces. Comput Graph Forum 17(2):167-174
5. Daum D, Dudek G (1998) On 3-D surface reconstruction using shape from shadows. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'98), pp 461-468. IEEE CS Press, Los Alamitos, Calif.
6. Davis JW, Bobick AF (1998) Sideshow: A silhouette-based interactive dual-screen environment. Technical Report TR-457, MIT Media Lab Tech Report, Cambridge
7. Computer Aided Design and Graphics Laboratory (CAD-LAB). TWIN Solid Modeling Package Reference Manual. School of Mechanical Engineering, Purdue University, http://cadlab.www.ecn.purdue.edu/cadlab/twin
8. Dorfmueller-Ulhaas K, Schmalstieg D (2001) Finger tracking for interaction in augmented environments. In: Proceedings of the 2nd ACM/IEEE International Symposium on Augmented Reality (ISAR 2001). IEEE CS Press, Los Alamitos, Calif.
9. Fitzmaurice GW, Ishii H, Buxton W (1995) Bricks: Laying the foundations for graspable user interfaces. In: Proceedings of CHI'95, pp 442-449. ACM Press, New York
10. Friedmann M, Starner T, Pentland A (1991) Synchronization in Virtual Realities. In: Presence, 1 (1), MIT Press
11. Fukumoto M, Mase K, SuenagaY (1992) Real-time detection of pointing actions for a glove-free interface. In: Proceedings of the IAPR Workshop on Machine Vision Applications, Tokyo
12. Ishii H, Ullmer B (1997) Tangible bits: Towards seamless interfaces between people, bits, and atoms. In: Proceedings of CHI'97, pp 234-241. ACM Press, New York
13. Jebara T, Eyster C, Weaver J, Starner T, Pentland A (1997) Stochasticks: Augmenting the billiards experience with probabilistic vision and wearable computers. In: Proceedings of the 1 st International Symposium on Wearable Computers, Cambridge, Mass. IEEE CS Press, Los Alamitos, Calif.
14. Jennings C (1999) Robust ringer tracking with multiple cameras. In: Proceedings of the International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp 152-160. IEEE CS Press, Los Alamitos, Calif.
15. Krueger M (1991) Artificial Reality II. Addison-Wesley, New York
16. Krueger W, Bohn CA, Froehlich B, Schueth H, Strauss W, Wesche G (1995) The responsive workbench: A virtual work environment. IEEE Comput 28(7):42-48
17. Krum DM, Ometoso O, Ribarsky W, Starner T, Hodges L (2002) Speech and gesture multimodal control of a whole earth 3D virtual environment. In: VisSym'02, Joint Eurographics - IEEE TCVG Symposium on Visualization, May 27-29, Barcelona, Spain, IEEE CS Press, Los Alamitos, Calif.
18. Laurentini A (1997) How many 2D silhouettes does it take to reconstruct a 3d object? Comput Vision Image Underst (CVIU) 67(1):81-87
19. Leibe B, Minnen D, Weeks J. Starner T (2001) Integration of wireless gesture tracking, object tracking, and 3D reconstruction in the perceptive workbench. In: Proceedings of the 2nd international workshop on computer vision systems (ICVS 2001), Lecture notes in computer science, vol 2095. Springer, Berlin Heidelberg New York, pp 73-92
20. Leibe B, Starner T, Ribarsky W, Wartell Z, Krum D, Singletary B, Hodges L (2000) Toward spontaneous interaction with the perceptive workbench. IEEE Comput Graph Appl 20(6):54-65
21. Matusik W, Buehler C, Gortler S, Raskar R, McMillan L (2000) Image based visual hulls, pp 369-374 ACM SIGGRAPH 2000
22. Quek FKH (1995) Eyes in the interface. Image Vision Comput 13(6):511-525
23. Rehg JM, Kanada T (1994) Visual tracking of high DOF articulated structures: An application to human hand tracking. In: Proceedings of the European Conference on Computer Vision (ECCV'94), pp 35-46. Lecture Notes in Computer Science, Vol.800, Springer, Berlin
24. Rekimoto J, Matsushita N (1997) Perceptual surfaces: Towards a human and object sensitive interactive display. In: Workshop on Perceptual User Interfaces (PUI'97)
25. Rosin PL, West GAW (1995) Non-parametric segmentation of curves into various representations. IEEE Trans Pattern Anal Mach Intell 17(12):1140-1153
26. Sato Y, Kobayashi Y, and Koike H (2000) Fast tracking of hands and fingertips in infrared images for augmented desk interface. In: Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition, pp 462-467. IEEE CS Press, Los Alamitos, Calif.
27. Seay AF, Krum D, Ribarsky W, Hodges L (1999) Multimodal interaction techniques for the virtual workbench. In: Proceedings of CHI'99, extended abstracts, pp 282-283. ACM Press, New York
28. Segen J, Kumar S (1999) Shadow gestures: 3D hand pose estimation using a single camera. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'99), vol 1, pp 479-485. IEEE CS Press, Los Alamitos, Calif.
29. Sharma R, Molineros J (1997) Computer vision based augmented reality for guiding manual assembly. PRESENCE: Teleop Virtual Environ 6(3):292-317
30. Srivastava SK, Ahuja N (1990) An algorithm for generating octrees from object silhouettes in perspective views. Comput Vision Graph Image Process: Image Underst (CVGIP:IU) 49(1):68-84
31. Starner T, Leibe B, Singletary B, Pair J (2000) Mind-warping: Towards creating a compelling collaborative augmented reality gaming interface through wearable computers and multi-modal input and output. In: IEEE International Conference on Intelligent User Interfaces (IUI'2000). ACM Press, New York
32. Starner T, Weaver J, Pentland A (1998) Real-time American sign language recognition using desk and wearable computer based video. IEEE Trans Pattern Anal Mach Intell 20(12):1371-1375
33. Sturman D (1992) Whole-hand input. PhD thesis, MIT Media Lab, Cambridge
34. Sullivan S, Ponce J (1998) Automatic model construction, pose estimation, and object recognition from photographs using triangular splines. IEEE Trans Pattern Anal Mach Intell 20(10):1091-1097
35. Ullmer B, Ishii H (1997) The metadesk: Models and prototypes for tangible user interfaces. In: ACM Symposium on User Interface Software and Technology (UIST'97), pp 223-232. ACM Press, New York
36. Underkoffler J, Ishii H (1998) Illuminating light: An optical design tool with a luminous-tangible interface. In: Proceedings of CHI'98, pp 542-549. ACM Press, New York
37. Utsumi A, Ohya J (1999) Multiple-hand-gesture tracking using multiple cameras. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR'99), vol 1, pp 473-478. IEEE CS Press, Los Alamitos, Calif.
38. van de Pol R, Ribarsky W, Hodges L, Post F (1999) Interaction in semi-immersive large display environments. In: Virtual Environments '99 Proceedings of the Eurographics Workshop in Vienna, 31 May-1 June, pp 157-168. Springer, Wien
39. Vogler C, Metaxas D (1998) Asl recognition based on coupling between hmms and 3D motion analysis. In: Proceedings of the International Conference on Computer Vision (ICCV'98), pp 363-369. IEEE CS Press, Los Alamitos, Calif.
40. Ware C, Balakrishnan R (1994) Reaching for objects in vr displays: Lag and frame rate. ACM Trans Comput Human Interact 1(4):331-356
41. Wartell Z, Ribarsky W, Hodges LF (1999) Third-person navigation of whole-planet terrain in a head-tracked stereoscopic environment. In: IEEE Virtual Reality '99 Conference, pp 141-149. IEEE CS Press, Los Alamitos, Calif.
42. Watson B, Walker N, Ribarsky W, Spaulding V (1998) The effects of variation of system responsiveness on user performance in virtual environments. Human Factors 40(3):403-414
43. Welch G, Bishop G (1997) SCAAT: Incremental tracking with incomplete information, pp333-344. ACM SIGGRAPH '97
44. Wellner P (1993) Interacting with paper on the digital desk. Comm ACM 36(7):86-89
45. Wren C, Azarbayejani A, Darrell T, Pentland A (1997) Pfinder: Real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780-785