Target detection and tracking with heterogeneous sensors

IEEE Journal on Selected Topics in Signal Processing

Volumn 2, Issue 4, 2008, Pages 503-513

  Zhou, Huiyu ^a   Taj, Murtaza ^a   Cavallaro, Andrea ^a  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Heterogeneous sensors; Kalman filter; Low bit rate communication; Multimodal detection and tracking; Particle filter

Indexed keywords

AIR FILTERS; BOOLEAN FUNCTIONS; CONTROL THEORY; IMAGE CODING; KALMAN FILTERS; METADATA; MICROPHONES; NETWORK PROTOCOLS; NONLINEAR FILTERING; SENSOR NETWORKS; SENSORS; SPEECH RECOGNITION; TARGET TRACKING; WAVE FILTERS;

HETEROGENEOUS SENSORS; KALMAN FILTER; LOW BIT RATE COMMUNICATION; MULTIMODAL DETECTION AND TRACKING; PARTICLE FILTER;

SIGNAL FILTERING AND PREDICTION;

EID: 54049156157     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2008.2001429     Document Type: Article

References (57)

1. J. Vermaak, M. Gangnet, A. Blake, and P. Pérez, "Sequential Monte Carlo fusion of sound and vision for speaker tracking," in Proc. IEEE Int. Conf. Comput. Vision, Vancouver, BC, Canada, Jul. 2001, pp. 741-746.
2. R. Chellappa, G. Qian, and Q. Zheng, "Vehicle detection and tracking using acoustic and video sensors," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Montreal, QC, Canada, May 2004, pp. 793-796.
3. V. Cevher, A. C. Sankaranarayanan, J. H. McClellan, and R. Chellappa, "Target tracking using ajoint acoustic video system," IEEE Trans. Multimedia, vol. 9, no. 4, pp. 715-727, Jun. 2007.
4. R. Cutler and L. S. Davis, "Look who's talking: Speaker detection using video and audio correlation," in Proc. IEEE Int. Conf. Multimedia Expo (III), New York, Jul.-Aug. 2000, pp. 1589-1592.
5. D. Gatica-Perez, G. Lathoud, I. McCowan, J.-M. Odobez, and D. Moore, "Audio-visual speaker tracking with importance particle filters," in Proc. IEEE Int. Conf. Image Process., Barcelona, Spain, Sep. 2003, pp. 25-28.
6. B. Kapralos, M. Jenkin, and E. Milios, "Audio-visual localization of multiple speakers in a video teleconferencing setting," Int. J. Imag. Syst. Technol., vol. 13, no. 1, pp. 95-105, Jun. 2003.
7. H. Asoh et al., "An application of a particle filter to Bayesian multiple sound source tracking with audio and video information fusion," in Proc. 7th Int. Conf. Inf. Fusion, Stockholm, Sweden, Jun. 2004, pp. 805-812.
8. D. Gatica-Perez, G. Lathoud, J.-M. Odobez, and I. McCowan, "Audiovisual probabilistic tracking of multiple speakers in meetings," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 2, pp. 601-616, Feb. 2007.
9. M. J. Beal, H. Attias, and N. Jojic, "Audio-video sensor fusion with probabilistic graphical models," in Proc. Eur. Conf. Computer Vision Copenhagen, Denmark, Jun. 2002, pp. 736-752.
10. P. Perez, J. Vermaak, and A. Blake, "Data fusion for visual tracking with particles," Proc. IEEE, vol. 92, no. 3, pp. 495-513, Mar. 2004.
11. T. Gehrig, K. Nicel, H. K. Ekenel, U. Klee, and J. McDonough, "Kalman filters for audio-video source localization," in Proc. IEEE Workshop Applicat. Signal Process. Audio Acoust. (WASPAA , New York, Oct. 2005, pp. 118-121.
12. N. Strobel, S. Spors, and R. Rabenstein, "Joint audio-video object localization and tracking," IEEE Signal Process. Mag., vol. 18, no. 1, pp. 22-31, Jan. 200 1.
13. Y. Chen and Y. Rui, "Speaker tracking using particle filter sensor fusion," Proc. IEEE, vol. 92, no. 3, pp. 485-494, Mar. 2004.
14. H. Zhou, M. Taj, and A. Cavallaro, "Audiovisual tracking using STAC sensors," in Proc. ACM/IEEE Int. Conf. Distributed Smart Cameras, Vienna, Austria, Sep. 25-28, 2007, pp. 170-177.
15. Z. Ding, H. Leung, and L. Hong, "Decoupling joint probabilistic data association algorithm for multiple target tracking," IEE Trans. Radar Sonar Navigation, vol. 146, no. 5, pp. 251-254, 1999.
16. J. Vermaak, S. J. Godsill, and P. P. Pérez, "Monte Carlo filtering for multi-target tracking and data association," IEEE Trans. Aerospace Electron. Syst. (AES), vol. 41, no. 1, pp. 309-332, Jan. 2005.
17. C. Stauffer, "Automated audio-visual analysis," Comput. Sci. Artif. Intell. Lab., MIT, Tech. Rep. MIT-CSAIL-TR-2005-057, Sep. 20,2005.
18. D. Smith and S. Singh, "Approaches to multisensor data fusion in target tracking: A survey," IEEE Trans. Knowledge Data Eng., vol. 18, no. 12, pp. 1696-1710, Dec. 2006.
19. A. Yilmaz, O. Javed, and M. Shah, "Object tracking: A survey," ACM Comput. Surveys (CSUR), vol. 38, no. 4, pp. 1-45, Dec. 2006.
20. E. Maggio and A. Cavallaro, "Hybrid particle filter and mean shift tracker with adaptive transition model," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Philadelphia, PA, Mar. 19-23, 2005, pp. 221-224.
21. O. Lanz, "Approximate Bayesian multibody tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 9, pp. 1436-1449, Sep. 2006.
22. D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-based object tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 5, pp. 564-577, May 2003.
23. K. Shafique and M. Shah, "A noniterative greedy algorithm for multiframe point correspondence," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 1, pp. 51-65, Jan. 2005.
24. K. Zia, T. Balch, and F. Dellaert, "MCMC-based particle filtering for tracking a variable number of interacting targets," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 11, pp. 1805-1819, Nov. 2005.
25. D. B. Ward and R. C. Williamson, "Particle filter beamforming for acoustic source localisation in a reverberant environment," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Orlando, FL, May 2002, pp. 1777-1780.
26. J. Vermaak and A. Blake, "Nonlinear filtering for speaker tracking in noisy and reverberant environments," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Salt Lake City, UT, May 2001, pp. 3021-3024.
27. E. A. Lehmann, D. B. Ward, and R. C. Williamson, "Experimental comparison of particle filtering algorithms for acoustic source localization in a reverberant room," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Hong Kong, China, Apr. 6-10, 2003, pp. 177-180.
28. D. B. Ward, E. A. Lehmann, and R. C. Williamson, "Particle filtering algorithms for acoustic source localization," IEEE Trans. Speech Audio Process., vol. 11, no. 6, pp. 826-836, Nov. 2003.
29. R. Brunelli, A. Brutti, P. Chippendale, O. Lanz, M. Omologo, P. Svaizer, and F. Tobia, "A generative approach to audio-visual person tracking," in Proc. CLEAR, Southampton, U.K., Apr. 2006, pp. 55-68.
30. D. N. Zotkin, R. Duraiswami, and L. S. Davis, "Joint audio-visual tracking using particle filters," EURASIP J. Appl. Signal Process. vol. 2002, pp. 1154-1164, 2001.
31. M. Bregonzio, M. Taj, and A. Cavallaro, "Multi-modal particle filtering tracking using appearance, motion and audio likelihoods," in Proc. IEEE Int. Conf. Image Process., San Antonio, TX, Sep. 2007, pp. V-33-V-36.
32. N. Checka, K. W. Wilson, M. R. Siracusa, and T. Darrell, "Multiple person and speaker activity tracking with a particle filter," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Cambridge, MA, May 2004, vol. 5, pp. V-881-V-884.
33. D. Gatica-Perez, G. Lathoud, I. McCowan, J. Odobez, and D. Moore, "Audio-visual speaker tracking with importance particle filters," in Proc. IEEE Int. Conf. Image Process., 2003, pp. 25-28.
34. Y. Rui and Y. Chen, "Better proposal distributions: Object tracking using unscented particle filter," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognition, Dec. 2001, pp. 786-793.
35. X. Zou and B. Bhanu, "Tracking humans using multi-modal fusion," in IEEE Int. Workshop Object Tracking Classification in and Beyond the Visible Spectrum (OTCBVS), San Diego, CA, Jun. 2005.
36. A. Abad et al., "UPC audio, video and multimodal person tracking systems in the CLEAR evaluation campaign," in Proc. CLEAR, Southampton, U.K., Apr. 2006, pp. 93-104.
37. A. Stergiou, A. Pnevmatikakis, and L. Polymenakos, "A decision fusion system across time and classifiers for audio-visual person identification," in Proc. CLEAR, Southampton, U.K., Apr. 2006, pp. 223-232.
38. D. Gatica-Perez, G. Lathoud, J. M. Odobez, and I. McCowan, "Audio-visual probabilistic tracking of multiple speakers in meetings," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 2, pp. 601-616, Feb. 2006.
39. A. Blake, M. Gangnet, P. Perez, and J. Vermaak, "Integrated tracking with vision and sound," in Proc. IEEE Int. Conf. Image Anal. Process., Sep. 2001, vol. 1, pp. 354-357.
40. B. D. O. Anderson and J. B. Moore, Optimal Filtering. Englewood Cliffs, NJ: Prentice-Hall, 1979.
41. M. Isard and A. Blake, "Condensation - Conditional density propagation for visual tracking," Int. J. Comput. Vision, vol. 29, no. 1, pp. 5-28, 1998.
42. M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, "A tutorial on particle filters for online nonlinear/non-gaussian bayesian tracking," IEEE Trans. Signal Process., vol. 50, no. 2, pp. 174-188, Feb. 2002.
43. K. Wilson, N. Checka, D. Demirdjian, and T. Darrell, "Audio-video array source separation for perceptual user interfaces," in Proc. 2001 Workshop Perceptive User Interfaces (PUI), Nov. 2001, pp. 1-7.
44. U. Bub, M. Hunke, and A. Waibel, "Knowing who to listen to in speech recognition: Visually guided beamforming," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., May 9-12, 1995, vol. 1, pp. 848-851.
45. B. Kapralos, M. Jenkin, and E. Milios, "Audio-visual localization of multiple speakers in a video teleconferencing setting," Int. J. Imag. Syst. Technol. pp. 95-105, 2003.
46. A. Cavallaro and T. Ebrahimi, "Interaction between high-level and low-level image analysis for semantic video object extraction," EURASIP J. Appl. Signal Process., vol. 6, pp. 786-797, Jun. 2004.
47. L. R. Rabiner and R. W. Schafer, Digital Processing of Speech Signal Englewood Cliffs, NJ: Prentice-Hall, 1978.
48. R. Y. Litovsky, H. S. Colburn, W. A. Yost, and S. J. Guzman, "The precedence effect," J. Acoust. Soc. Amer., vol. 106, pp. 1633-1654, 1999.
49. K. Wilson and T. Darrell, "Improving audio source localization by learning the precedence effect," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Philadelphia, PA, Mar. 18-23, 2005, pp. IV-1125-IV-1128.
50. W. A. Yost and G. Gourevitch, Directional Hearing, Chapter the Precedence Effect. New York: Springer-Verlag, 1987.
51. L. E. Kinsler, A. R. Frey, A. B. Coppens, and J. V. Sanders, Fundamentals of Acoustics, 4th ed. New York: Wiley, 2000, ch. 12, p. 341, 0-471-84789-5.
52. C. H. Knapp and Carter G. C., "The generalised correlation method for estimation of time delay," IEEE Trans. Acoust., Speech Signal Process., vol. ASSP-24, no. 4, pp. 320-327, Aug. 1976.
53. S. M. Bozic, Digital and Kalman Filtering. London, U.K.: Edward Arnold, 1979.
54. A. Bhattacharyya, On a Measure of Divergence Between Two Statistical Populations Defined by Probability Distributions. Calcutta, India: Bulletin of the Calcutta Mathematical Society, 1943.
55. T. Kailath, "The divergence and bhattacharyya distance measures in signal selection," IEEE Trans. Commun. Technol., vol. CT-15, no., pp. 52-60, Feb. 1967.
56. K. Nummiaro, E. Koller-Meier, and L. Van Gool, "A color-based particle filter," in Proc. Workshop Generative-Model-Based Vision, Jun. 2002, pp. 53-60.
57. M. Taj, E. Maggio, and A. Cavallaro, "Multi-feature graph-based object tracking," in Proc. CLEAR, Springer LNCS 4122, Southampton, U.K., Apr. 2006, pp. 190-199.