Smart meeting systems: A survey of state-of-the-art and open issues

ACM Computing Surveys

Volumn 42, Issue 2, 2010, Pages

  Yu, Zhiwen ^a   Nakamura, Yuichi ^b  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

^b KYOTO UNIVERSITY   (Japan)

Author keywords

Evaluation; Meeting capture; Meeting recognition; Semantic processing; Smart meeting system

Indexed keywords

AUDIO-VISUAL CONTENT; DESIGN ISSUES; EVALUATION METHOD; MEETING CAPTURE; SEMANTIC PROCESSING; SYSTEM ARCHITECTURES;

INDUSTRIAL RESEARCH; SEMANTICS; SURVEYS; VISUAL COMMUNICATION;

AUDIO SYSTEMS;

EID: 77949368907     PISSN: 03600300     EISSN: 15577341     Source Type: Journal    
DOI: 10.1145/1667062.1667065     Document Type: Review

References (96)

1. AIZAWA, K., ISHIJIMA, K., AND SHIINA, M. 2001. Summarizing wearable video. In Proceedings of the IEEE ICIP. IEEE, Los Alamitos, CA, 398-401.
2. AKITA, Y., TRONCOSO, C., AND KAWAHARA, T. 2006. Automatic transcription of meetings using topic-oriented language model adaptation. In Proceedings of the Western Pacific Acoustics Conference (WESPAC).
3. AMI. 2007. AMI project homepage, http://www.amiproject.org/.
4. BARON, D., SHRIBERG, E., AND STOLCKE, A. 2002. Automatic punctuation and disfluency detection in multiparty meetings using prosodic and lexical cues. In Proceedings of the International Conference on Spoken Language Processing. 949-952.
5. BELL, G. AND GEMMELL, J. 2007. A digital life. Scientific American.
6. BETT, M., GROSS, R., YU, H., ZHU, X., PAN, Y., YANG, J., AND WAIBEL, A. 2000. Multimodal meeting tracker. In Proceedings of the International Conference on Content-Based Multimedia Information Access (RIAO). 32-45.
7. BOUNIF, H., DRUTSKYY, O., JOUANOT, F., AND SPACCAPIETRA,S. 2004. Amultimodal database framework for multimedia meeting annotations. In Proceedings of the International Conference on Multi-Media Modeling (MMM). 17-25.
8. BUDNIKOV, D., CHIKALOV, I., KOZINTSEV, I., AND LIENHART, R. 2004. Distributed array of synchronized sensors and actuators. In Proceedings of the EUSIPCO.
9. BUSSO, C., HERNANZ, S., CHU, C. W., KWON, S., LEE, S., GEORGIOU, P. G., COHEN, I., AND NARAYANAN,S. 2005. Smart room: Participant and speaker localization and identification. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vol.2. IEEE, Los Alamitos, CA, 1117-1120.
10. CHEN, X. AND YANG, J. 2002. Towards monitoring human activities using an omnidirectional camera. In Proceedings of the 4th IEEE International Conference on Multimodal Interfaces. IEEE, Los Alamitos, CA, 423-428.
11. CHIU, P., KAPUSKAR, A., REITMEIER, S., AND WILCOX, L. 2000. Room with a rear view: Meeting capture in a multimedia conference room. IEEE Multimedia 7, 4, 48-54.
12. COLBATH, S. AND KUBALA, F. 1998. Rough'n'ready: A meeting recorder and browser. In Proceedings of the Perceptual User Interface Conference. 220-223.
13. CUTLER, R., RUI, Y., GUPTA, A., CADIZ, J. J., TASHEV, I., HE, L., COLBURN, A., ZHANG, Z., LIU, Z., AND SILVERBERG, S. 2002. Distributed meetings: A meeting capture and broadcasting system. In Proceedings of the 10th ACM Conference on Multimedia. ACM, New York, 503-512.
14. DATTA, R., JOSHI, D., LI, J., AND WANG, J. Z. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv. 40, 2, Article 5.
15. DEFAGO, X., SCHIPER, A., AND URBON, P. 2004. Total order broadcast and multicast algorithms: Taxonomy and Survey. ACM Comput. Surv. 36, 4, 372-421.
16. DEY, A. K., SALBER, D., ABOWD, G. D., AND FUTAKAWA, M. 1999. The conference assistant: Combining context-awareness with wearable computing. In Proceedings of the 3rd International Symposium on Wearable Computers (ISWC). 21-28.
17. DIELMANN, A. AND RENALS, S. 2004. Dynamic Bayesian networks for meeting structuring. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). IEEE, Los Alamitos, CA, 629-632.
18. DIMICCO, J. M., HOLLENBACH, K. J., AND BENDER, W. 2006. Using visualizations to review a group's interaction dynamics. Extended abstracts. In Proceedings of the CHI. 706-711.
19. FOOTE, J. AND KIMBER, D. 2000. FlyCam: Practical panoramic video and automatic camera control. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). IEEE, Los Alamitos, CA, 1419-1422.
20. GATICA-PEREZ, D., MCCOWAN, I., ZHANG, D., AND BENGIO, S. 2005. Detecting group interest-level in meetings. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vol.1. IEEE, Los Alamitos, CA, 489-492.
21. GATICA-PEREZ, D., MCCOWAN, I., BARNARD, M., BENGIO, S., AND BOURLARD, H. 2003. On automatic annotation of meeting databases. In Proceedings of the International Conference on Image Processing (ICIP). Vol.3. 629-632.
22. GEYER, W., RICHTER, H., AND ABOWD, G. D. 2005. Towards a smarter meeting record-Capture and access of meetings revisited. Multimedia Tools Appl. 27, 3, 393-410.
23. GEYER, W., RICHTER, H., AND ABOWD, G.D. 2003. Making multimedia meeting records more meaningful. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). Vol.2. IEEE, Los Alamitos, CA, 669-672.
24. GROSS, R., YANG, J., AND WAIBEL, A. 2000. Face recognition in a meeting room. In Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, Los Alamitos, CA, 294-299.
25. HAN, S., KIM, N., CHOI, K., AND KIM, J. W. 2007. Design of multi-party meeting system for interactive collaboration. In Proceedings of COMSWARE. 1-8.
26. HILLARD, D., OSTENDORF, M., AND SHRIBERG, E. 2003. Detection of agreement vs. disagreement in meetings: Training with unlabeled data. In Proceedings of the Human Language Technology and North American Chapter of the Association for Computational Linguistics Conference (HLT-NAACL). 34-36.
27. HORSTER, E. AND LIENHART, R. 2006. Approximating optimal visual sensor placement. In Proceedings of the IEEE ICME. IEEE, Los Alamitos, CA.
28. JAIMES, A. AND MIYAZAKI, J. 2005. Building a smart meeting room: from infrastructure to the video gap (research and open issues). In Proceedings of the 21st International Conference on Data Engineering Workshops (ICDEW). 1173-1182.
29. JAIMES, A., OMURA, K., NAGAMINE, T., AND HIRATA, K. 2004. Memory cues for meeting video retrieval. In Proceedings of the First ACM Workshop on Continuous Archival and Retrieval of Personal Experiences (CARPE). ACM, New York, 74-85.
30. JAIN, R. 2003. Experiential computing. Comm. ACM. 46, 7, 48-54.
31. JAIN, R., KIM, P., AND LI, Z. 2003. Experiential meeting systems. In Proceedings of the ACM Workshop on Experiential TelePresence. ACM, New York, 1-12.
32. JANIN, A., ANG, J., BHAGAT, S., DHILLON, R., EDWARDS, J., MACIAS-GUARASA, J., MORGAN, N., PESKIN, B., SHRIBERG, E., STOLCKE, A., WOOTERS, C., AND WREDE, B. 2004. The ICSI meeting project: Resources and research. In Proceedings of the NIST ICASSP Meeting Recognition Workshop.
33. KAPLAN, J. 2005. Next-generation conference rooms. In Proceedings of the UBICOMP Workshop on Ubiquitous Computing in Next Generation Conference Rooms.
34. KENNEDY, L. AND ELLIS, D. 2004. Laughter detection in meetings. In Proceedings of the NIST ICASSP Meeting Recognition Workshop. 118-121.
35. KENNEDY, L. AND ELLIS, D. 2003. Pitch-based emphasis detection for characterization of meeting recordings. In Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). IEEE, Los Alamitos, CA, 243-248.
36. KERN, N., SCHIELE, B., JUNKER, H., LUKOWICZ, P., AND TROSTER, G. 2003. Wearable sensing to annotate meeting recordings. Person. Ubiquit. Comput. 7, 5, 263-274.
37. KULYK, O., WANG, C., AND TERKEN, J. 2005. Real-time feedback based on nonverbal behaviour to enhance social dynamicsinsmall group meetings. InProceedings of the Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms (MLMI). 150-161.
38. LEE, D., EROL, B., GRAHAM, J., HULL, J. J., AND MURATA, N. 2002. Portable meeting recorder. In Proceedings of the 10th ACM Conference on Multimedia. ACM, New York, 493-502.
39. LIU, Z., ZHANG, Z., HE, L.W., AND CHOU, P. 2007. Energy-based sound source localization and gain normalization for ad hoc microphone arrays. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). IEEE. Los Alamitos, CA.
40. LIWICKI, M., SCHLAPBACH, A., BUNKE, H., BENGIO, S., MARIETHOZ, J., AND RICHIARDI, J. 2006. Writer identification for smart meeting room systems. In Proceedings of the 7th IAPR Workshop on Document Analysis Systems. 186-195.
41. LUO, C., LI, J., AND LI, S. 2004. DigiMetro-An application-level multicast system for multiparty video conferencing. In Proceedings of Globecom. 982-987.
42. MBOSS. 2007. Mboss homepage. http://www.mboss.force9.co.uk/.
43. MCCOWAN, I., GATICA-PEREZ, D., BENGIO, S., LATHOUD, G., BARNARD, M., AND ZHANG, D. 2005. Automatic analysis of multimodal group actions in meetings. IEEE Trans. Pattern Anal. Mach. Intell. 27, 3, 305-317.
44. MIKIC, I., HUANG, K., AND TRIVEDI, M. 2000. Activity monitoring and summarization for an intelligent meeting room. In Proceedings of the IEEE Workshop on Human Motion. IEEE, Los Alamitos, CA, 107-112.
45. MIMIO. 2007. Mimio homepage. http://www.mimio.com/.
46. MONCRIEFF, S., VENKATESH, S., AND WEST, G. 2007. Privacy and the access of information in a smart house environment. In Proceedings of the 15th Multimedia Conference. ACM, New York, 671-680.
47. MOORE, D. AND MCCOWAN, I. 2003. Microphone array speech recognition: Experiments on overlapping speech in meetings. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vol.5, 497-500.
48. MURRAY, G., RENALS, S., AND CARLETTA, J. 2005. Extractive summarization of meeting recordings. In Proceedings of the Ninth European Conference on Speech. 593-596.
49. NAIT-CHARIF, H. AND MCKENNA, S. J. 2003. Head tracking and action recognition in a smart meeting room. In Proceedings of the IEEE International Workshop on Performance Evaluation of Tracking and Surveillance. IEEE, Los Alamitos, CA, 24-31.
50. NIJHOLT, A., ZWIERS, J., AND PECIVA, J. 2005. The distributed virtual meeting room exercise. In Proceedings of the ICMI Workshop on Multimodal Multiparty Meeting Processing. 93-99.
51. OZEKI, M., MAEDA, S., OBATA, K., AND NAKAMURA, Y. 2008. Virtual assistant: An artificial agent for enhancing content acquisition. In Proceedings of the 1st ACM International Workshop on Semantic Ambient Media Experience. ACM, New York.
52. PIANESI, F., ZANCANARO, M., NOT, E., LEONARDI, C., FALCON, V., AND LEPRI, B. 2008. Multimodal support to group dynamics. Person. Ubiquit. Comput. 12, 2.
53. PIMENTEL, M. C., CATTELAN, R. G., AND BALDOCHI, L. A. 2007. Prototyping applications to document human experiences. IEEE Pervasive Comput. 6, 2, 93-100.
54. RAM, S., RAMAKRISHNAN, K. R., ATREY, P. K., SINGH, V. K., AND KANKANHALLI, M. S. 2006. A design methodology for selection and placement of sensors in multimedia surveillance systems. InProceedings of the 4th ACM International Workshop on Video Surveillance and Sensor Networks (VSSN). ACM, New York, 121-130.
55. RAYKAR, V. C., KOZINTSEV, I. V., AND LIENHART, R. 2005. Position calibration of microphones and loudspeakers in distributed computing platforms. IEEE Trans. Speech Audio Process. 13, 1, 70-83.
56. REIDSMA, D., RIENKS, R., AND JOVANOVIC, N. 2004. Meeting modelling in the context of multimodal research. InProceedings of the 1st International Workshop on Machine Learning for Multimodal Interaction. 22-35.
57. REITER, S. AND RIGOLL, G. 2005. Meeting event recognition using a parallel recurrent neural net approach. In Proceedings of the 2nd Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms.
58. RENALS, S. AND ELLIS, D. 2003. Audio information access from meeting rooms. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vol.4. IEEE, Los Alamitos, CA, 744-747.
59. RICHTER, H., ABOWD, G. D., GEYER, W., FUCHS, L., DAIJAVAD, S., AND POLTROCK, S. 2001. Integrating meeting capture within a collaborative team environment. In Proceedings of the International Conference on Ubiquitous Computing (UbiComp'01). Lecture Notes in Computer Science, Vol.2201, Springer, Berlin, 123-138.
60. RIJSBERGEN, C. J. 1979. Information Retrieval 2nd Ed., Butterworths.
61. RUI, Y. 2002. System and process for locating a speaker using 360 degree sound source localization, U.S. Patent.
62. RUI, Y., GUPTA, A., AND CADIZ, J. J. 2001. Viewing meetings captured by an omnidirectional camera. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 450-457.
63. RUI, Y., GUPTA, A., GRUDIIN, J., AND HE, L. 2004. Automating lecture capture and broadcast: technology and videography. Multimedia Syst. 10, 1, 3-15.
64. SCHULTZ, T., WAIBEL, A., BETT, M., METZE, F., PAN, Y., RIES, K., SCHAAF, T., SOLTAU, H., WESTPHAL, M., YU, H., AND ZECHNER, K. 2001. The ISL meeting room system. In Proceedings of the Workshop on Hands-Free Speech Communication (HSC).
65. SINGH, V. K., ATREY, P. K., AND KANKANHALLI, M. S. 2007. Coopetitive multi-camera surveillance using model predictive control. Machine Vision Appl. DOI: 10.1007/s00138-007-0082-2.
66. SRINIVASAN, V., STANKOVIC, J., AND WHITEHOUSE, K. 2008. Protecting your daily in-home activity information from a wireless snooping attack. In Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp). 202-211.
67. STAJANO, F. 2002. Security for Ubiquitous Computing. Wiley, New York.
68. STANFORD, V., GAROFOLO, J., GALIBERT, O., MICHEL, M., AND LAPRUN, C. 2003. The NIST smart space and meeting room projects: Signals, acquisition, annotation and metrics. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Vol.4. IEEE, Los Alamitos, CA, 6-10.
69. STIEFELHAGEN, R. 2002. Tracking focus of attention in meetings. In Proceedings of the 4th IEEE International Conference on Multimodal Interfaces (ICMI), IEEE, Los Alamitos, CA, 273-280.
70. STIEFELHAGEN, R., YANG, J., AND WAIBEL, A. 2002. Modeling focus of attention for meeting indexing based on multiple cues. IEEE Trans. Neural Netw. 13, 4, 928-938.
71. STIEFELHAGEN, R. AND ZHU, J. 2002. Head orientation and gaze direction in meetings. In Proceedings of the Conference on Extended Abstracts on Human Factors in Computing Systems (Extended Abstracts) (CHI'02). ACM, New York, 858-859.
72. STIEFELHAGEN, R., YANG, J., AND WAIBEL, A. 2001. Estimating focus of attention based on gaze and sound. In Proceedings of the Workshop on Perceptive User Interfaces (PUI). 1-9.
73. STIEFELHAGEN, R., YANG, J., AND WAIBEL, A. 1999. Modeling focus of attention for meeting indexing. In Proceedings of the 7th ACM International Conference on Multimedia (Part 1). ACM, New York, 3-10.
74. TOMOBE, H. AND NAGAO, K. 2006. Discussion ontology: Knowledge discovery from human activities in meetings. In New Frontiers in Artificial Intelligence, Lecture Notes in Computer Science. Vol.4384, Springer, Berlin, 33-41.
75. TRIVEDI, M., MIKIC, I., AND BHONSLE, S. 2000. Active camera networks and semantic event databases for intelligent environments. In Proceedings of the IEEE Workshop on Human Modeling, Analysis and Synthesis (in conjunction with CVPR). IEEE, Los Alamitos, CA.
76. TRUONG, K. N. AND ABOWD, G. D. 2004. INCA: A software infrastructure to facilitate the construction and evolution of ubiquitous capture and access applications. In Pervasive Computing. Lecture Notes in Computer Science. Vol.3001, Springer, Berlin, 140-157.
77. TUCKER, S. AND WHITTAKER, S. 2004. Accessing multimodal meeting data: Systems, problems and possibilities. In Proceedings of the Workshop on Multimodal Interaction and Related Machine Learning Algorithms. 1-11.
78. TURK, M. AND PENTLAND, A. 1991. Eigenfaces for recognition. J. Cognitive Neurosci. 3, 1, 71-86.
79. ULUDAG, S., LUI, K. S., NAHRSTEDT, K. AND BREWSTER, G. 2007. Analysis of topology aggregation techniques for QoS routing. ACM Comput. Surv. 39, 3, Article 7.
80. VALIN, J. M., MICHAUD, F., ROUAT, J., AND LETOURNEAU, D. 2003. Robust sound source localization using a microphone array on a mobile robot. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, Los Alamitos, CA, 1228-1233.
81. WAIBEL, A., BETT, M., METZE, F., RIES, K., SCHAAF, T., SCHULTZ, T., SOLTAU, H., YU, H., AND ZECHNER, K. 2001. Advances in automatic meeting record creation and access. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). IEEE, Los Alamitos, CA, 597-600.
82. WAIBEL, A., BETT, M., AND FINKE, M. 1998. Meeting browser: Tracking and summarizing meetings. In Proceedings of the Broadcast News Transcription and Understanding Workshop. 281-286.
83. WANG, Z. AND CROWCROFT, J. 1996. Quality-of-service routing for supporting multimedia applications. IEEE J. Select. Areas Comm. 14, 7, 1228-1234.
84. WELLNER, P., FLYNN, M., AND GUILLEMOT, M. 2004. Browsing recorded meetings with Ferret. In Proceedings of the First International Workshop on Machine Learning for Multimodal Interaction (MLMI). 12-21.
85. WREDE, B. AND SHRIBERG, E. 2003a. Spotting 'hot spots' in meetings: Human judgments and prosodic cues. In Proceedings of the European Conference on Speech Communication and Technology. 2805-2808.
86. WREDE, B. AND SHRIBERG, E. 2003b. The relationship between dialogue acts and hot spots in meetings. In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, Los Alamitos, CA, 180-185.
87. YANG, J., ZHU, X., GROSS, R., KOMINEK, J., PAN, Y., AND WAIBEL, A. 1999. Multimodal people IDfor a multimedia meeting browser. In Proceedings of ACM Multimedia. ACM, New York, 159-168.
88. YU, H., FINKE, M., AND WAIBEL, A. 1999. Progress in automatic meeting transcription. In Proceedings of the 6th European Conference on Speech Communication and Technology (Eurospeech). Vol.2, 695-698.
89. YU, Z., ZHOU, X., ZHANG, D., CHIN, C. Y., WANG, X., AND MEN, J. 2006. Supporting context-aware media recommendations for smart phones. IEEE Pervasive Comput. 5, 3, 68-75.
90. YU, Z., OZEKI, M., FUJII, Y., AND NAKAMURA, Y. 2007. Towards smart meeting: Enabling technologies and a real-world application. In Proceedings of the 9th International Conference on Multimodal Interfaces (ICMI). 86-93.
91. YU, Z., AOYAMA, H., OZEKI, M., AND NAKAMURA, Y. 2008a. Collaborative capturing and detection of human interactions in meetings. In Adjunct Proceedings of the 6th International Conference on Pervasive Computing (Pervasive). 65-69.
92. YU, Z., AOYAMA, H., OZEKI, M., AND NAKAMURA, Y. 2008b. Social interaction detection and browsing in meetings. In Adjunct Proceedings of the 10th International Conference on Ubiquitous Computing (UbiComp). 40-41.
93. YU, Z., ZHOU, X., AND NAKAMURA, Y. 2008c. Meeting warming-up: Detecting common interests and conflicts before a meeting. In Proceedings of the ACM Computer Supported Cooperative Work Conference Supplement (CSCW).
94. ZHANG, D., GATICA-PEREZ, D., BENGIO, S., MCCOWAN, I., AND LATHOUD, G. 2004. Modeling individual and group actions in meetings: A two-layer HMM framework. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Event Mining in Video (CVPR-EVENT). Vol.7. IEEE, Los Alamitos, CA, 117-124.
95. ZHAO, J. AND CHEUNG, S.-C.S. 2007. Multi-camera surveillance with visual tagging and generic camera placement. In Proceedings of the First ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC). 259-266.
96. ZOBL, M., WALLHOFF, F., AND RIGOLL, G. 2003. Action recognition in meeting scenarios using global motion features. In Proceedings of the IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS-CCVS). 32-36.