Online diarization of streaming audio-visual data for smart environments

IEEE Journal on Selected Topics in Signal Processing

Volumn 4, Issue 5, 2010, Pages 845-856

  Schmalenstroeer, Joerg ^a   Haeb Umbach, Reinhold ^a  

^a UNIVERSITY OF PADERBORN   (Germany)

Author keywords

ambient communication; diarization; middleware

Indexed keywords

AMBIENT COMMUNICATION; AUDIO-VISUAL DATA; CONTEXT INFORMATION; CONTEXTUAL INFORMATION; DIARIZATION; FACE IDENTIFICATION; I/O DEVICE; LATENCY ASPECTS; MICROPHONE ARRAYS; MULTI-MODAL; ONLINE PROCESSING; REMOTE PARTNERS; SERVICE-ORIENTED MIDDLEWARE; SMART ENVIRONMENT; SPEAKER DIARIZATION; SPEAKER LOCALIZATION; TEMPORAL SEGMENTATIONS; VIDEO DATA;

AUDIO STREAMING; CAMERAS; MIDDLEWARE; VIDEO RECORDING;

COMMUNICATION;

EID: 77956739906     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2010.2050519     Document Type: Article

References (58)

1. E. Aarts and S. Marzano, The New Everyday: Views on Ambient Intelligence. Rotterdam, The Netherlands: 010 Uitgeverij, 2004.
2. "Information Society Technologies Advisory Group Reports," 2003 [Online]. Available: http://cordis.europa.eu/fp7/ict/istag/reports- en.html
3. S. Tranter and D. Reynolds, "An overview of automatic speaker diarization systems," IEEE Trans. Audio, Speech, Lang. Process., vol.14, no.5, pp. 1557-1565, Sep. 2006.
4. S. Meignier, D. Moraru, C. Fredouille, J. Bonastre, and L. Besacier, "Step-by-step and integrated approaches in broadcast news speaker diarization," Comput. Speech Lang., vol.20, no.2-3, pp. 303-330, Jul. 2006.
5. C. Wooters and M. Huijbregts, "The ICSI RT07s speaker diarization system," in Proc. Multimodal Technol. Percept. Humans: Int. Eval. Workshops CLEAR 2007 and RT 2007, Baltimore, MD, May 8-11, 2007, pp. 509-519, 2008 Revised Selected Papers.
6. X. Zhu, C. Barras, L. Lamel, and J. Gauvain, "Multi-stage speaker diarization for conference and lecture meetings," in Lecture Notes in Computer Science: Machine Learning for Multimodal Interaction, no.4625, 2008, pp. 533-542.
7. S. Borkowski and G. Privat, "Spatial interaction in ambient communication," in Proc. Int. Conf. Enactive Interfaces (ENACTIVE'07), Grenoble, France, Nov. 2007.
8. H. K. Ekenel, M. Fischer, Q. Jin, and R. Stiefelhagen, "Multi-modal person identification in a smart environment," in Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR'07), Minneapolis, MN, Jun. 2007, pp. 1-8.
9. A. Salah, R. Morros, J. Luque, C. Segura, J. Hernando, O. Ambekar, B. Schouten, and E. Pauwels, "Multimodal identification and localization of users in a smart environment," J. Multimodal User Interfaces, vol.2, no.2, pp. 75-91, Sep. 2008.
10. A. Noulas and B. J. A. Krose, "On-line multi-modal speaker diarization," in Proc. Int. Conf. Multimodal Interfaces (ICMI'07), New York, Apr. 2007, pp. 350-357.
11. J. Schmalenstroeer and R. Haeb-Umbach, "Online speaker change detection by combining bic with microphone array beamforming," in Proc. Conf. Int. Speech Commun. Assoc. (Interspeech'06), Pittsburgh, PA, Sep. 2006.
12. J. Pardo, X. Anguera, and C. Wooters, "Speaker diarization for multiple-distant-microphone meetings using several sources of information," IEEE Trans. Comput., vol.56, no.9, pp. 1212-1224, Sep. 2007.
13. S. Borkowski, T. Flury, A. Gerodolle, and G. Privat, "Ambient communication and context-aware presence management," Commun. Comput. Inf. Sci.: Constructing Ambient Intell., vol.11, pp. 391-396, 2008.
14. A. Härmä, "Ambient telephony: Scenarios and research challenges," in Proc. Conf. Int. Speech Commun. Assoc. (Interspeech'07), Antwerp, Belgium, Aug. 2007.
15. "Computers in the human interaction loop," CHIL Consortium Jan. 2004 [Online]. Available: http://chil.server.de/
16. "Augmented multi-party interaction," AMI Consortium Jan. 2004 [Online]. Available: http://www.amiproject.org/
17. A. Temko, R. Malkin, C. Ziegler, D. Macho, C. Nadeu, and M. Omologo, "Acoustic event detection and classification in smart-room environments: Evaluation of CHIL project systems," J. Tecnologia del Habla, vol.4, pp. 1-6, Nov. 2006.
18. X. Anguera, C.Wooters, and J. Hernando, "Acoustic beamforming for speaker diarization of meetings," IEEE Trans. Audio, Speech, Lang. Process., vol.15, no.7, pp. 2011-2022, Sep. 2007.
19. "Detection and identification of rare audiovisual cues," DIRAC Consortium Jan. 2006 [Online]. Available: http://www.diracproject.org/
20. T. Kühnapfel, T. Tan, S. Venkatesh, and E. Lehmann, "Calibration of audio-video sensors for multi-modal event indexing," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP'07), Honolulu, HI, Apr. 2007, pp. 741-744.
21. D. Lo, R. A. Goubran, R. M. Dansereau, G. Thompson, and D. Schulz, "Robust joint audio-video localization in video conferencing using reliability information," IEEE Trans. Instrum. Meas., vol.53, no.4, pp. 1132-1139, Aug. 2004.
22. E. Warsitz and R. Haeb-Umbach, "Acoustic filter-and-sum beamforming by adaptive principal component analysis," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP'05), Philadelphia, PA, Mar. 2005, pp. 797-800.
23. J. Schmalenstroeer and R. Haeb-Umbach, "Joint speaker segmentation, localization and identification for streaming audio," in Proc. Conf. Int. Speech Commun. Assoc. (Interspeech' 07), Antwerp, Belgium, Aug. 2007.
24. J. Schmalenstroeer, M. Kelling, V. Leutnant, and R. Haeb-Umbach, "Fusing audio and video information for online speaker diarization," in Proc. Conf. Int. Speech Commun. Assoc. (Interspeech'09), Brighton, U.K., Sep. 2009.
25. M. J. Carey, "SOA what?," IEEE Trans. Comput. , vol.41, no.3, pp. 92-94, Mar. 2008.
26. "Ambient intelligence for the networked home environment," 2006 [Online]. Available: http://www.hitechprojects. com/euprojects/amigo
27. Speech Processing, Transmission and Quality Aspects (Stq); Distributed Speech Recognition; Advanced Frontend Feature Extaction Algorithm; Compression Algorithms, ETSI. (2007) Es 202 212 v1.1.5 [Online]. Available: http://www.etsi.org
28. B. Wildermoth and K. Paliwal, "Use of voicing and pitch information for speaker recognition," in Proc. IEEE Conf. Speech Sci. Technol. (SST'00), Canberra, Australia, Dec. 2000, pp. 324-328.
29. P. Delacourt and C. J. Wellekens, "Distbic: A speaker-based segmentation for audio data indexing," Speech Commun., vol.32, no.1-2, pp. 111-126, Sep. 2000.
30. J. Ramirez and J. Segura, "Efficient voice activity detection algorithms using long-term speech information," Speech Commun., vol.42, no.3-4, pp. 271-287, Apr. 2004.
31. C. Kim, S. Seong, J. Lee, and L. Kim, "Winscale: An image-scaling algorithm using an area pixel model," IEEE Trans. Circuits Syst. Video Technol., vol.13, no.6, pp. 549-553, Jun. 2003.
32. B. Froeba and C. Kueblbeck, "Face tracking by means of continous detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognition Workshop (CVPRW'04), Washington, DC, Mar. 2004, pp. 65-71.
33. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Conf. Comput. Vis. Pattern Recognition (CVPR'01), Kauai, HI, Dec. 2001, pp. 511-518.
34. P. Belhumeur, J. Hespanha, and D. Kriegman, "Eigenfaces vs. fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol.19, no.7, pp. 711-720, Jul. 1997.
35. "The Yale Face Database," Yale University. New Haven, CT, Apr. 2009 [Online]. Available: http://cvc.yale.edu/projects/yalefaces/yalefaces. html
36. C. Knapp and G. Carter, "The generalized correlation method for estimation of time delay," IEEE Trans. Acoust., Speech, Signal Process., vol.ASSP-24, no.4, pp. 320-327, Aug. 1976.
37. M. Omologo, P. Svaizer, A. Brutti, and L. Cristoforetti, "Speaker localization in CHIL lectures: Evaluation criteria and results," Lecture Notes in Computer Science, no.3869, pp. 476-487, 2006.
38. J. B. Allen and D. A. Berkley, "Image method for efficiently simulating small-room acoustics," J. Acoust. Soc. Amer., vol.65, no.4, pp. 943-950, Apr. 1979.
39. M. Kotti, V. Moschou, and C. Kotropoulos, "Speaker segmentation and clustering," Signal Process., vol.88, no.5, pp. 1091-1124, May 2007.
40. S. S. Chen and P. S. Gopalakrishnan, "Speaker, environment and channel change detection and clustering via the bayesian information criterion," in Proc. DARPA Broadcast News Transcription and Understanding Workshop, Lansdowne, VA, Feb. 1998.
41. D. Reynolds, T. Quatieri, and R. Dunn, "Speaker verification using adapted Gaussian mixture models," Digital Signal Process., vol.10, no.1-3, pp. 19-41, Jan. 2000.
42. D. Sacchetti, Y. Bromberg, N. Georgantas, V. Issarny, J. Parra, and R. Poortinga, "The amigo interoperable middleware for the networked home environment," in Proc. Middleware, Grenoble, France, Dec. 2005.
43. N. Georgantas, S. B. Mokhtar, Y. Bromberg, V. Issarny, J. Kalaoja, J. Kantarovich, A. Gerodolle, and R. Mevissen, "The amigo service architecture for the open networked home environment," in Proc. Working IEEE/IFIP Conf. Software Architecture (WICSA'05), Pittsburgh, PA, Nov. 2005.
44. T. Berners-Lee, J. Hendler, and O. Lassila, "The semantic web," Sci. Amer. Mag., pp. 1-4, May 2001.
45. D. L. McGuinness and F. van Harmelen, "Web ontologie language," World Wide Web Consortium Feb. 2004 [Online]. Available: http:// www.w3.org/TR/owl-features/
46. D. Beckett et al., "Resource description framework, "WorldWideWeb Consortium, Jan. 2008 [Online]. Available: http://www.w3.org/RDF/
47. WWW. (2002)Web Services.WorldWideWeb Consortium. [Online]. Available: http://www.w3.org/2002/ws/
48. S. Mokhtar, A. Kaul, N. Georgantas, and V. Issarny, "Efficient semantic service discovery in pervasive computing environments," Lecture Notes in Computer Science, no.4290, pp. 240-259, 2007.
49. R. Chinnici et al., "Web services description language," World Wide Web Consortium. Jun. 2007 [Online]. Available: http://www.w3. org/TR/wsdl20/
50. DAML. (2006) Web Ontology Language for Web Services. DARPA Agent Markup Language. [Online]. Available: http://www.daml.org/ services/owl-s/
51. F. Ramparany, R. Poortinga, M. Stikic, J. Schmalenstroeer, and T. Prante, "An open context information management infrastructure," in Proc. IET Int. Conf. Intell. Environments (IE'07), Ulm, Germany, Sep. 2007.
52. E. Prud'hommeaux and A. Seaborne, "SPARQL protocol and RDF query language," World Wide Web Consortium, Jan. 2008 [Online]. Available: http://www.w3.org/TR/rdf-sparql-query/
53. Open Service Gateway Initiative, Jan. 2008 [Online]. Available: http:// www.osgi.org
54. F. Ramparany et al., "Amigo software repository," Amigo Consortium, 2009 [Online]. Available: http://amigo.gforge.inria.fr
55. J. Schmalenstroeer, V. Leutnant, and R. Haeb-Umbach, "Amigo context management service with applications in ambient communication scenarios," Commun. Comput. Inf. Sci.: Construct. Ambient Intell., vol.11, pp. 397-402, 2008.
56. Audio Codec, 2008 [Online]. Available: http://www.speex.org
57. H. Schulzrinne et al., "RTP: A Transport Protocol for Real-Time Applications. Internet Engineering Task Force," Jul. 2003 [Online]. Available: http://tools.ietf.org/html/rfc3550
58. "Theora Codec," 2008 [Online]. Available: http://www.Theora.org