Algorithms for cooperative multisensor surveillance

Proceedings of the IEEE

Volumn 89, Issue 10, 2001, Pages 1456-1477

  Collins, Robert T ^a,d   Lipton, Alan J ^b   Fujiyoshi, Hironobu ^c   Kanade, Takeo ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Diamondback Vision Inc   (United States)

^c CHUBU UNIVERSITY   (Japan)

^d CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Active vision; Cooperative systems; Geolocation; Multisensor systems; Site security monitoring; User interfaces; Video surveillance. 2001 ieee

Indexed keywords

EID: 0000097352     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/5.959341     Document Type: Article

References (90)

1. D. Beymer, P. McLauchlan, B. Coifman, and J. Malik, "A real-time computer vision system for measuring traffic parameters," in Proc. 1997 Conf. Computer Vision and Pattern Recognition , San Juan, Puerto Rico, June 1997, pp. 495-501.
2. [21 T. Darrell, G. G. Gordon, M. Harville, and J. Woodfill, "Integrated person tracking using stereo, color and pattern detection," Int. J. Comput. Vis., vol. 37, pp. 175-185, June 2000.
3. J. M. Ferryman, S. J. Maybank, and A. D. Worrall, "Visual surveillance for moving vehicles," Int. J. Comput. Vis., vol. 37, no. 2, pp. 187-197, June 2000.
4. [41 G. L. Foresti, "Real-time system for video surveillance of unattended outdoor environments," IEEE Trans. Circuits Syst. Video Technol., vol. 8, p. 697, Oct. 1998.
5. M. Haag and H. H. Nagel, "Incremental recognition of traffic situations from video image sequences." Image Vis. Comput., vol. 18, pp. 137-153, Jan. 2000.
6. I. Haritaoglu, D. Harwood, and L. S. Davis, "W4: Real-time surveillance of people and their activities." IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 809-830, Aug. 2000.
7. J. Heikkila and O. Silven, "A real-time system for monitoring of cyclists and pedestrians," in Proc. 2nd IEEE Int. Workshop Visual Surveillance, Fort Collins, CO, June 1999.
8. T. Huang and S. Russell, "Object identification: 4 Bayesian analysis with application to traffic surveillance," Artif. Intell., vol. 103, no. 1-2, pp. 77-93, Aug. 1998.
9. D. Koller, K. Daniilidis, and H. Nagel, "Model-based object tracking in monocular image sequences of road traffic scenes," Int. J. Comput. Vis., vol. 10, pp. 257-281, June 1993.
10. N. M. Oliver, B. Rosario, and A. P. Pentland, "A Bayesian computer vision system for modeling human interactions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 831-843, Aug. 2000.
11. P. Remagnino, T. Tan, and K. Baker, "Multiagent visual surveillance of dynamic scenes,'" Image Vis. Comput., vol. 16, pp. 529-532, June 1998.
12. R. Rosales and S. Sclaroff, "3D trajectory recovery for tracking multiple objects and trajectory-guided recognition of actions," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Fort Collins, CO, June 1999, pp. 117-123.
13. "Section I, video surveillance and monitoring," in Proc. DARPA Image Understanding Workshop, vol. 1, Monterey, CA, Nov. 1998, pp. 1-400.
14. M. Bogaert, N. Chleq, P. Cornez, C. S. Regazzoni, A. Teschioni, and M. Thonnat, "The passwords project," in Int. Conf. Image Processing, 1996, pp. 675-678.
15. M. Rota and M. Thonnat, "Video sequence interpretation for visual surveillance," in Proc. 3rd IEEE Int. Workshop on Visual Surveillance, Dublin, Ireland, July 2000.
16. "The Views project and wide-area surveillance," VIEWS , PM-02-ECCV92-01, 1992.
17. H. Buxton and S. G. Gong, "Visual surveillance in a dynamic and uncertain world," Artif. Intell., vol. 78, pp. 431-459, Oct. 1995.
18. T. Matsuyama, "Cooperative distributed vision," in Proc. DARPA Image Understanding Workshop, vol. 1, Nov. 1998, pp. 365-384.
19. "CDVWS," in Proc. 1t/2nd/3nt Int. Workshop Cooperative Distributed Vision, Kyoto, Japan, 1997/1998/1999.
20. "VS'2000," in Proc. 3rd IEEE Int. Workshop Visual Surveillance, Dublin, Ireland, July 2000.
21. "VS'98." in Proc. 1st IEEE Int. Workshop Visual Surveillance, Bombay, India, Jan. 1998.
22. "VS'99," in Proc. 2nd IEEE Int. Workshop Visual Surveillance, Fort Collins, CO, June 1999.
23. "PETS'2000," in Proc. First IEEE Int. Workshop Performance Evaluation of Tracking and Surveillance, J. Ferryman, Ed., Grenoble, France, Mar. 2000.
24. R. T. Collins, A. J. Lipton, and T. Kanade, "Introduction to the special section on video surveillance," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 745-746, Aug. 2000.
25. S. Maybank and T. Tan, "Introduction - surveillance," Int. J. Comput. Vis., vol. 37, pp. 173-173, June 2000.
26. T. E. Boult, R. Micheals, X. Gao, P. Lewis, C. Power, W. Yin, and A. Erkan, "Frame-rate omnidirectional surveillance and tracking of camouflaged and occluded targets,"in Proc 2nd IEEE Int. Workshop Visual Surveillance, Fort Collins, CO, June 1999.
27. B. E. Flinchbaugh, "Robust video motion detection and event recognition,'' in Proc. 1997 DARPA Image Understanding Workshop, New Orleans, LA, May 1997, pp. 51-54.
28. L. Wixson, J. Eledath, M. Hansen, R. Mandelbaum, and D. Mishra, "Image alignment for precise camera fixation and aim," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Santa Barbara, CA, June 1998, pp. 594-600.
29. K. Cauble, "The OpenScene ModStealth." in Proc. Simulation Interoperability Workshop , 1997, Paper 97S-SIW-008.
30. D. B. Cavitt, C. M. Overstreat, and K. J. Maly, "Modeling and distributed simulation techniques for synthetic training environments," in Modeling and Simulation of Advanced Computer Systems: Applications and Systems, K. Bagchi, Ed. Philadelphia, PA: Gordon & Breach, 1996, pp. 237-260.
31. R. Collins, Y. Tsin, J. R. Miller, and A. Lipton, "Using a DEM to determine geospatial object trajectories," in Proc. 1998 DARPA Image Understanding Workshop, Monterey, CA, Nov. 1998, pp. 115-122.
32. R. T. Collins and Y. Tsin, "Calibration of an outdoor active camera system," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Fort Collins, CO, June 1999, pp. 528-534.
33. M. Haag, T. Frank, H. Kollnig, and H. H. Nagel, "Influence of an explicitly modeled 3D scene on the tracking of partially occluded vehicles," Comput. Vis. Image Understand., vol. 65, pp. 206-225, Feb. 1997.
34. K. Toyama, J. Krumm, B. Brumitt, and B. Meyers, "Wallflower: Principles and practice of background maintenance," in Proc. Int. Conf. Computer Vision, Corfu, Greece, 1999, pp. 255-261.
35. C. Anderson, P. Burt, and G. van der Wal, "Change detection and tracking using pyramid transformation techniques," in Proc. SPIE Intelligent Robots and Computer Vision, vol. 579, 1985, pp. 72-78.
36. P. L. Rosin and T. Ellis, "Image difference threshold strategies and shadow detection," in Proc. British Machine Vision Conf., 1995, pp. 347-356.
37. [371 C. Stauffer and W. E. L. Grimson, "Learning patterns of activity using real-time tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 747-757, Aug. 2000.
38. C. R. Wren, A. Azarbayejani, T. J. Darrell, and A. P. Pentland, "Pfinder: Real-time tracking of the human body," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, pp. 780-785, July 1997.
39. J. Barron, D. Fleet, and S. Beauchemin, "Performance of optical flow techniques," Int. J. Comput. Vis., vol. 12, no. 1, pp. 42-77, 1994.
40. C. Cedras and M. Shah, "Motion-based recognition: A survey," Image Vis. Comput., vol. 13, pp. 129-155, Mar. 1995.
41. G. Halevy and D. Weinshall, "Motion of disturbances: Detection and tracking of multi-body nonrigid motion," Mach. Vis. Applicat., vol. 11, no. 3, pp. 122-137, 1999.
42. D. G. Lowe, "Robust model-based motion tracking through the integration of search and estimation." Int. J. Comput. Vis., vol. 8, pp. 113-122, Aug. 1992.
43. S. Wachter and H. H. Nagel, "Tracking persons in monocular image sequences," Comput. Vis. Image Understand., vol. 74, pp. 174-192, June 1999.
44. M. Isard and A. Blake, "C-conditional density propagation for visual tracking," Int. J. Comput. Vis., vol. 29, pp. 5-28, Aug. 1998.
45. G. D. Hager and P. N. Belhumeur, "Efficient region tracking with parametric models of geometry and illumination," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, pp. 1025-1039, Oct. 1998.
46. Q. Zheng and R. Chellappa, "Automatic feature point extraction and tracking in image sequences for arbitrary camera motion," Int. J. Comput. Vis., vol. 15, pp. 31-76, June 1995.
47. Y. Bar-Shalom and T. Fortmann, Tracking and Data Association. Boston. MA: Academic, 1988.
48. I. J. Cox, "A review of statistical data association techniques for motion correspondence," Int. J. Comput. Vis., vol. 10, pp. 53-66, Feb. 1993.
49. K. Bradshaw, I. Reid, and D. Murray, "The active recovery of 3D motion trajectories and their use in prediction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19. pp. 219-234, Mar. 1997.
50. A. Rosenfeld and A. C. Kak, Digital Picture Processing. Boston, MA: Academic, 1982.
51. A. Lipton, "Local application of optic flow to analyze rigid versus nonrigid motion," in IEEE Int. Workshop Frame-Rate Vision, Corfu, Greece, Sept. 1999.
52. [521 _, "Virtual postman - an illustrative example of virtual video," Int. J. Robot. Automat., vol. 15, no. 1, pp. 9-16, 2000.
53. B. Bhanu, Y. Q. Lin, G. Jones III, and J. Peng, "Adaptive target recognition," Mach. Vis. Applicat., vol. 11, no. 6, pp. 289-299, 2000.
54. B. G. Mobasseri, "Automatic target scoring system using machine vision," Machine Vis. Applied., vol. S. no. 1 pp. 20-30, 1995.
55. A. Lipton, H. Fujiyoshi, and R. S. Paul. "Moving target detection and classification from real-time video." in Proc. 1998 Workshop Applications of Computer Vision, Princeton, NJ, Oct. 1998.
56. D. P. Casasent and L. M. Neiberg, "Classifier and shift-invariant automatic target recognition neural networks," Neural Netw., vol. 8, no. 7-8, pp. 1117-1129, 1995.
57. G. Pasquariello, G. Satalino, V. Laforgia, and F. Spilotros, "Automatic target recognition for naval traffic control using neural networks," Image Vis. Comput., vol. 16, pp. 67-73, Feb. 1998.
58. K. Fukunaga, Statistical Pattern Recognition: Academic, 1989.
59. R. Collins, A. Lipton, T. Kanade, H. Fujiyoshi, D. Duggins, Y. Tsin, D. Tolliver, N. Enomoto, and O. Hasegawa, "A system for video surveillance and monitoring: VSAM final report," Robotics Inst., CMU-RI-TR-00-12, 2000.
60. A. Baumberg and D. Hogg, "Generating spatiotemporal models from examples," Image Vis. Comput., vol. 14, pp. 525-532, Aug. 1996.
61. C. Bregler, "Learning and recognizing human dynamics in video sequences," in Proc. 1997 Conf. Computer Vision and Pattern Recognition , San Juan, Puerto Rico, June 1997, pp. 568-574.
62. [621 R. Cutler and L. S. Davis, "Robust real-time periodic motion detection, analysis and applications." IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 781-796, Aug. 2000.
63. Y. Guo, G. Xu, and S. Tsuji, "Tracking human body motion based on a stick figure model," J. Vis. Commun. Image Represent., vol. 5, pp. 1-9, 1994.
64. D. Hogg, "Model-based vision: A program to see a walking person," Image Vis. Comput., vol. 1, no. 1, pp. 5-20, 1983.
65. Y. Ricquebourg and P. Bouthemy, "Real-time tracking of moving persons by exploring spatio-temporal image slices," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 797-808, Aug. 2000.
66. K. Rohr, "Toward model-based recognition of human movements in image sequences," Computer Vis., Graphics Image Process., vol. 59, pp. 94-115, Jan. 1994.
67. S. Ju, M. Black, and Y. Yacoob, "Cardboard people: A parameterized model of articulated image motion," in Proc. Int. Conf. Face and Gesture Analysis, Killington, VT, Oct. 1996.
68. J. K. Aggarwal and Q. Cai, "Human motion analysis: A review," Comput. Vis. Image Understand., vol. 73, pp. 428-440, Mar. 1999.
69. D. M. Gavrila, "The visual analysis of human movement: A survey," Comput. Vis. Image Understand., vol. 73, pp. 82-98, Jan. 1999.
70. H. Fujiyoshi and A. Lipton, "Real-time human motion analysis by image skeletonization," in Proc. 1998 Workshop Applications of Computer Vision, Princeton, NJ, Oct. 1998.
71. G. Strang and K. Borre, Linear Algebra, Geodesy and GPS. Cambridge, MA: Wellesley-Cambridge, 1997.
72. T. Kanade, R. Collins, A. Lipton, P. Burt, and L. Wixson, "Advances in cooperative multi-sensor video surveillance," in Proc. 1998 DARPA Image Understanding Workshop, vol. 1, Monterey, CA, Nov. 1998, pp. 3-24.
73. Manual of Photogrammetry, 4th ed., American Society of Photogrammetry, Falls Church, 1980.
74. L. Lee, R. Romano, and G. Stein, "Monitoring activities from multiple video streams: Establishing a common coordinate frame," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 758-767, Aug. 2000.
75. T. N. Tan, G. D. Sullivan, and K. D. Baker, "Recognizing objects on the ground-plane," Image Vis. Comput., vol. 12. pp. 164-172, Apr. 1994.
76. J. E. Bresenham, "An algorithm for computer control of a digital plotter," IBM Syst. J., vol. 4, no. 1, pp. 25-30, 1965.
77. J. M. Brady, "Special issue on sensor data fusion," Int. J. Robot. Res., vol. 7, no. 6, pp. 1-161, 1989.
78. P. K. Varshney, "Special issue on data fusion," Proc. IEEE, vol. 85, pp. 3-5, Jan. 1997.
79. J. Orwell, P. Remagnino, and G. A. Jones, "Multi-camera color tracking," in Proc. 2nd IEEE Int. Workshop Visual Surveillance, Fort Collins, CO, June 1999.
80. G. Healey, "Segmenting images using normalized color," IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 64-73, Jan. 1992.
81. J. L. Crowley and F. Ramparany, "Mathematical tools for representing uncertainty in perception," in Proceedings of Spatial Reasoning and Multi-Sensor Fusion Workshop. San Mateo, CA, 1987, pp. 293-302.
82. P. K. Davis, "Distributed interactive simulation in the evolution of DoD warfare modeling and simulation," Proc. IEEE, vol. 83, pp. 1138-1155, Aug. 1995.
83. J. E. Smith, "Recent developments in ModSAF terrain representation," in Proc. 5th Conf. Computer Generated Forces and Behavioral Representation, Orlando, FL, May 1995, pp. 375-381.
84. A. J. Courtemanche and A. Ceranowicz, "ModSAF development status," in Proc. 5th Conf. Computer Generated Forces and Behavioral Representation, Orlando, FL, May 1995, pp. 3-13.
85. M. D. Petty, "Computer generated forces in distributed interactive simulation," Distributed Interactive Simulation Systems for Simulation and Training In the Aerospace Environment, pp. 251-280, Apr. 1995.
86. T. Bautell, CGI Programming in C and Perl. Reading, MA: Addison-Wesley , 1996.
87. M. Brand and V. Kettnaker, "Discovery and segmentation of activities in video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 844-851, Aug. 2000.
88. R. J. Howarth and H. Buxton, "Conceptual descriptions from monitoring and watching image sequences," Image Vis. Comput., vol. 18, pp. 105-135, Jan. 2000.
89. Y. A. Ivanov and A. F. Bobick, "Recognition of visual activities and interactions by stochastic parsing," IEEE Trans. Pattern Anal Mach. Intell., vol. 22, pp. 852-872, Aug. 2000.
90. T. Wada and T. Matsuyama, "Multiobject behavior recognition by event driven selective attention method," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, pp. 873-867, Aug. 2000.