Calibrating distributed camera networks

Proceedings of the IEEE

Volumn 96, Issue 10, 2008, Pages 1625-1639

  Devarajan, Dhanya ^a,b   Cheng, Zhaolin ^a,c   Radke, Richard J ^a  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b Immersive Media Company   (United States)

^c Captira Analytical   (United States)

Author keywords

Belief propagation; Bundle adjustment; Camera calibration; Distributed algorithms; Metric reconstruction; Sensor networks; Structure from motion

Indexed keywords

CALIBRATION; INFERENCE ENGINES; NONLINEAR PROGRAMMING; PARALLEL ALGORITHMS; SENSOR NETWORKS; SENSOR NODES;

BELIEF PROPAGATION; BUNDLE ADJUSTMENTS; CAMERA CALIBRATION; METRIC RECONSTRUCTION; STRUCTURE FROM MOTION;

CAMERAS;

EID: 55549123492     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2008.928759     Document Type: Article

References (92)

1. I. Akyildiz, W. Su, Y. Sankarasubramaniain, and E. Cayirci, "Wireless sensor networks: A survey," Comput. Netw., vol. 38, pp. 393-422, 2002.
2. L. Davis, E. Borovikov, R. Cutler, D. Harwood, and T. Horprasert, "Multi-perspective analysis of human action," in Proc. 3rd Int. Workshop Cooperative Distrib. Vision, Nov. 1999.
3. T. Kanade, P. Rander, and P. Narayanan, "Virtualized reality: Constructing virtual worlds from real scenes," IEEE Multimedia, vol. 4, pp. 34-47, Jan. 1997.
4. S. Moezzi, L. C. Tai, and P. Gerard, "Virtual view generation for 3-D digital video," IEEE Multimedia, vol. 4, pp. 18-26, Jan.-Mar. 1997.
5. A. Savvides, C. C. Man, and M. B. Srivastava, "Dynamic fine-grained localization in ad-hoc wireless sensor networks," in Int. Conf. Mobile Comput. Netw. (MobiCom) 2001, Jul. 2001.
6. D. Estrin et al., Embedded, Everywhere: A Research Agenda for Networked Systems of Embedded Computers. Washington, DC: National Academy, 2001.
7. Z. Cheng, D. Devarajan, and R. Radke, "Determining vision graphs for distributed camera networks using feature digests," EURASIP J. Appl. Signal Process. (Special Issue on Visual Sensor Networks), 2007.
8. D. Devarajan and R. Radke, "Calibrating distributed camera networks using belief propagation," EURASIP J. Appl. Signal Process. (Special Issue on Visual Sensor Networks), 2007.
9. D. Devarajan, R. Radke, and H. Chung, "Distributed metric calibration of ad-hoc camera networks," ACM Trans. Sensor Netw., vol. 2, no. 3, pp. 380-403, Aug. 2006.
10. R. T. Collins and Y. Tsin, "Calibration of an outdoor active camera system," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., Jun. 1999.
11. R. Hartley, "Self-calibration from multiple views with a rotating camera," in Proc. ECCV '94, 1994, vol. 1, pp. 471-478.
12. G. P. Stein, "Accurate internal camera calibration using rotation, with analysis of sources of error," in Proc. ICCV, 1995, pp. 230-236.
13. Z. Haas et al., "Wireless ad hoc networks," in Encyclopedia of Telecommunications, J. Proakis, Ed. New York: Wiley, 2002.
14. M. Rahimi, R. Baer, O. Iroezi, J. Garcia, J. Warrior, D. Estrin, and M. Srivastava, "Cyclops: In situ image sensing and interpretation in wireless sensor networks," in Proc. ACM Conf. Embed. Netw. Sensor Syst., 2005.
15. A. Rowe, C. Rosenberg, and I. Nourbakhsh, "A low cost embedded color vision system," in Proc. IROS 2002, 2002.
16. W.-C. Feng, B. Code, E. Kaiser, W.-C. Feng, and M. Le Baillif, "Panoptes: Scalable low-power video sensor networking technologies, " ACM Trans. Multimedia Comput., Commun., Applicat., vol. 1, no. 2, pp. 151-167, May 2005.
17. S. Hengstler and H. Aghajan, "A smart camera mote architecture for distributed intelligent surveillance," in Proc. ACM SenSys Workshop Distrib. Smart Cameras, 2006.
18. M. Antone and S. Teller, "Scalable, extrinsic calibration of omni-directional image networks," Int. J. Comput. Vision, vol. 49, no. 2/3, pp. 143-174, Sep./Oct. 2002.
19. G. Sharp, S. Lee, and D. Wehe, "Multiview registration of 3-D scenes by minimizing error between coordinate frames," in Proc. Eur. Conf. Comput. Vision (BCCV), Copenhagen, Denmark, 2002, pp. 587-597.
20. P. Kulkarni, P. Shenoy, and D. Ganesan, "Approximate initialization of camera sensor networks," in Proc. 4th Eur. Conf. Wireless Sensor Netw., 2007.
21. D. Makris, T. Ellis, and J. Black, "Bridging the gaps between cameras," in Proc. IEEE Comput. Society Conf. Comput. Vision Pattern Recognit., 2004.
22. D. Marinakis and G. Dudek, "Topology inference for a vision-based sensor network," in Proc. Can. Conf. Comput. Robot Vision, May 2005.
23. X. Zou, B. Bhanu, B. Song, and A. Roy-Chowdhury, "Determining topology in a distributed camera network," in Proc. IEEE Int. Conf. Image Process., 2007.
24. O. Javed, Z. Rasheed, K. Shafique, and M. Shah, "Tracking across multiple cameras with disjoint views," in Proc. 9th Int. Conf. Comput. Vision, Nice, France, 2003.
25. C. Niu and E. Crimson, "Recovering non-overlapping network topology using far-field vehicle tracking data," in Proc. 18th Int. Conf. Pattern Recognit., 2006.
26. C. Jaynes, "Multi-view calibration from planar motion for video surveillance," in Proc. 2nd IEEE Workshop Visual Surveil. (VS'99), 1999, pp. 59-66.
27. A. Rahimi, B. Dunagan, and T. Darrell, "Simultaneous calibration and tracking with a network of non-overlapping sensors," in Proc. 2004 IEEE Comput. Society Conf. Comput. Vision Pattern Recognit., 2004, vol. 1, pp. 187-194.
28. J. Black, T. Ellis, and P. Rosin, "Multiview image surveillance and tracking," in Proc. Workshop Motion Video Comput., 2002, pp. 169-174.
29. L. Lee, R. Romano, and G. Stein, "Monitoring activities from multiple video streams: Establishing a common coordinate frame," IEEE Trans. Pattern Recognit. Machine Intell., vol. 22, pp. 758-767, Aug. 2000.
30. A. van den Hengel, A. Dick, H. Detmold, A. Cichowski, and R. Hill, "Finding camera overlap in large surveillance networks," in Proc. 8th Asian Conf. Comput. Vision, Nov. 2007.
31. M. Meingast, S. Oh, and S. Sastry, "Automatic camera network localization using object image tracks," in Proc. IEEE Int. Conf. Comput. Vision (ICCV) Workshop Visual Represent. Model. Large-Scale Environ., Oct. 2007.
32. E. Kang, I. Cohen, and G. Medioni, "A graph-based global registration for 2D mosaics," in Proc. 15th Int. Conf. Pattern Recognit. (ICPR), 2000, pp. 257-260.
33. R. Marzotto, A. Fusiello, and V. Murino, "High resolution video mosaicing with global alignment," in Proc. IEEE Conf. Comput. Vision Pattern Recognit. (CVPR), Jun. 2004, pp. 692-698.
34. H. Sawhney, S. Hsu, and R. Kumar, "Robust video mosaicing through topology inference and local to global alignment," in Proc. Eur. Conf. Comput. Vision (ECCV), 1998, pp. 103-119.
35. M. Brown and D. Lowe, "Recognising panoramas," in Proc. IEEE Int. Conf. Comput. Vision (ICCV), 2003.
36. M. Brown, R. Szeliski, and S. Winder, "Multi-image matching using multi-scale oriented patches," in Proc. IEEE Conf. Comput. Vision Pattern Recognit. (CVPR), San Diego, CA, 2005, pp. 510-517.
37. F. Schaffalitzky and A. Zisserman, "Multi-view matching for unordered image sets," in Proc. Eur. Conf. Comput. Vision (ECCV), 2002, pp. 414-431.
38. S. Avidan, Y. Moses, and Y. Moses, "Centralized and distributed multi-view correspondence," Int. J. Comput. Vision, vol. 71, no. 1, pp. 49-69, 2007.
39. P. Baker and Y. Aloimonos, "Complete calibration of a multi-camera network," in Proc. IEEE Workshop Omnidirect. Vision 2000, 2000.
40. X. Chen, J. Davis, and P. Slusallek, "Wide area camera calibration using virtual calibration objects," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., 2000.
41. A. Barton-Sweeney, D. Lymberopoulos, and A. Sawides, "Sensor localization and camera calibration in distributed camera sensor networks," in Proc. 3rd Int. Conf. Broadband Commun., Netw. Syst. (BROADNETS), San Jose, CA, 2006, pp. 1-10.
42. C. J. Taylor and B. Shirmohammadi, "Self localizing smart camera networks and their applications to 3D modeling," in Proc. Int. Workshop Distrib. Smart Cameras, Oct. 2006.
43. H.-G. Maas, "Image sequence based automatic multi-camera system calibration techniques," ISPRS J. Photogram. Remote Sensing, vol. 54, no. 5-6, pp. 352-359, Dec. 1999.
44. P. Baker and Y. Aloimonos, "Calibration of a multicamera network," in Proc. IEEE Workshop Omnidirect. Vision, 2003.
45. I. M. Rekletis and G. Dudek, "Automated calibration of a camera sensor network," in Proc. IEEE/RSI Int. Conf. Intell. Robots Syst., Aug. 2-6, 2005, pp. 401-406.
46. I. Rekleitis, D. Meger, and G. Dudek, "Simultaneous planning, localization, and mapping in a camera sensor network," Robot. Auton. Syst., vol. 54, no. 11, pp. 921-932, Nov. 2006.
47. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vision, vol. 60, no. 2, pp. 91-110, 2004.
48. K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 10, pp. 1615-1630, 2005.
49. K. Mikolajczyk and C. Schmid, "Scale and affine invariant interest point detectors," Int. J. Cornput. Vision, vol. 60, no. 1, pp. 63-86, Oct. 2004.
50. J. H. Freidman, J. L. Bentley, and R. A. Finkel, "An algorithm for finding best matches in logarithmic expected time, " ACM Trans. Math. Software, vol. 3, no. 3, pp. 209-226, Sep. 1977.
51. J. Jannotti and J. Mao, "Distributed calibration of smart cameras," in Proc. Workshop Distrib. Smart Cameras, Oct. 2006.
52. Z. Zhang, "Determining the epipolar geometry and its uncertainty: A review," Int. J. Comput. Vision, vol. 27, no. 2, pp. 161-195, 1998.
53. R. Hartley and A. Zisserman, Multiple View Geometry Comput. Vision. Cambridge, U.K.: Cambridge Univ. Press, 2000.
54. H. V. Poor, An Introduction to Signal Detection and Estimation. Berlin, Germany: Springer, 1998.
55. K. Langendoen and N. Reijers, "Distributed localization in wireless sensor networks: A quantitative comparison," Comput. Netw., vol. 43, no. 4, pp. 499-518, Nov. 2003.
56. R. Tsai, "A versatile camera calibration technique for high-accuracy 3-D machine vision metrology using off-the-shelf TV cameras and lenses," in Radiometry (Physics-Based Vision), L. Wolff, S. Shafer, and G. Healey, Eds. Sudbury, MA: Jones and Harriett, 1992.
57. B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, "Bundle adjustment - A modern synthesis," in Vision Algorithms: Theory and Practice, W. Triggs, A. Zisserman, and R. Szeliski, Eds. Berlin, Germany: Springer Verlag, 2000, pp. 298-375.
58. Y. Ma, S. Soatto, J. Košecká, and S. Sastry, An Invitation to 3-D Vision: From Images to Geometric Models. Berlin, Germany: Springer, 2004, ch. 11.
59. J. Leonard and H. Durrant-Whyte, "Simultaneous map building and localization for an autonomous mobile robot," in Proc. IEEE/RSJ Int. Workshop Intell. Robots Syst., 1991, pp. 1442-1447.
60. S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics: Intell. Robot. and Autonomous Agents. Cambridge, MA: MIT Press, 2005.
61. X. Liu, P. Kulkarni, P. Shenoy, and D. Ganesan, "Snapshot: A self-calibration protocol for camera sensor networks," in Proc. IEEE/CreateNet BASENETS 2006, 2006.
62. H. Lee and H. Aghajan, "Collaborative node localization in surveillance networks using opportunistic target observations," in Proc. 4th ACM Int. Workshop on Video Surveil. Sensor Netw., 2006, pp. 9-18.
63. S. Funiak, C. Guestrin, M. Paskin, and R. Sukthankar, "Distributed localization of networked cameras," in Proc. 5th Int. Conf. Inf. Process. Sensor Netw., Apr. 2006.
64. W. Mantzel, H. Choi, and R. Baraniuk, "Distributed camera network localization," in Proc. 38th Asilomar Conf. Signals, Syst. Comput., Nov. 2004.
65. D. Devarajan, "Distributed localization of camera networks," Ph.D. dissertation, Rensselaer Polytechnic Inst., Troy, NY, 2007.
66. M. Pollefeys, R. Koch, and L. J. Van Gool, "Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters," in Proc. IEEE Int. Conf. Comput. Vision (ICCV), 1998, pp. 90-95.
67. M. Pollefeys, F. Verbiest, and L. J. V. Gool, "Surviving dominant planes in uncalibrated structure and motion recovery," in Proc. 7th Eur. Conf. Comput. Vision - Part II (ECCV '02), London, U.K., 2002, pp. 837-851.
68. P. Sturm and B. Triggs, "A factorization based algorithm for multi-image projective structure and motion," in Proc. 4th Eur. Conf. Comput. Vision (ECCV '96), 1996, pp. 709-720.
69. B. Triggs, "Factorization methods for projective structure and motion," in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit. (CVPR '96), San Francisco, CA, 1996, pp. 845-851.
70. M. Andersson and D. Betsis, "Point reconstruction from noisy images," J. Math. Imag. Vision, vol. 5, pp. 77-90, Jan. 1995.
71. R. Hartley and P. Sturm, "Triangulation," Comput. Vision Image Understand., vol. 68, no. 2, pp. 146-157, 1997.
72. D. Estrin, R. Govindan, J. S. Heidemann, and S. Kumar, "Next century challenges: Scalable coordination in sensor networks," in Proc. ACM/IEEE Conf. Mobile Comput. Netw. (MobiCom 99), Seattle, WA, 1999, pp. 263-270.
73. N. Sastry, U. Shankar, and D. Wagner, "Secure verification of location claims," in Proc. WiSe '03 2nd ACM Workshop Wireless Security, 2003, pp. 1-10.
74. S. Capkun and J.-P. Hubaux, "Secure positioning of wireless devices with application to sensor networks," in Proc. IEEE INFOCOM, 2005.
75. Y. Wei, Z. Yu, and Y. Guan, "Location verification algorithms for wireless sensor networks," in Proc. 27th Int. Conf. Distrib. Comput. Syst., 2007, p. 70.
76. W. Du, L. Fang, and N. Peng, "LAD: Localization anomaly detection for wireless sensor networks," J. Parallel Distrib. Comput., vol. 66, no. 7, pp. 874-886, 2006.
77. K. P. Murphy, Y. Weiss, and M. I. Jordan, "Loopy belief propagation for approximate inference: An empirical study," in Proc. Uncertain. Artif. Intell. (UAI '99), 1999, pp. 467-475.
78. S. Venkatesh, M. Alanyali, and O. Savas, "Distributed detection in sensor networks with packet losses and finite capacity links," IEEE Trans. Signal Process., vol. 54, pp. 4118-4132, Nov. 2006.
79. C. Christopher and P. Avi, "Loopy belief propagation as a basis for communication in sensor networks," in Proc. 29th Annu. Conf. Uncertain. Artif. Intell. (UAI-03), San Francisco, CA, 2003, pp. 159-166.
80. J. Thorpe and R. McEliece, "Data fusion algorithms for collaborative robotic exploration," Interplanet. Netw. Progr. Rep., vol. 149, pp. 1-14, Jan. 2002.
81. A. Ihler, J. Fisher, R. Moses, and A. Willsky, "Nonparametric belief propagation for self-calibration in sensor networks," in Proc. 3rd Int. Symp. Inf. Process. Sensor Netw., Berkeley, CA, Apr. 2004.
82. J. Besag, "Spatial interaction and the statistical analysis of lattice systems," J. Roy. Statist. Soc., vol. B, no. 36, pp. 192-236, 1974.
83. J. Hammersley and P. E. Clifford, "Markov fields on finite graphs and lattices," 1971.
84. J. Pearl, Probablistic Reasoning in Intelligent Systems. San Mateo, CA: Morgan Kaufmann, 1988.
85. F. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inf. Theory, vol. 47, 2001.
86. R. Smith, M. Seif, and P. Cheeseman, "Estimating uncertain spatial relationships in robotics," in Autonomous Robot Vehicles. New York: Springer-Verlag, 1990, pp. 167-193.
87. J. V. Mascia, "Actuation-assisted external calibration of distributed camera sensor networks," master's thesis, Univ. of California, Los Angeles, 2007.
88. P. Bolliger, M. Köhler, and K. Römer, "Facet: Towards a smart camera network of mobile phones," in Proc. Autonomics 2007 (ACM First Int. Conf. Autonom. Comput. Commun. Syst.), Rome, Italy, Oct. 2007.
89. M. A. Paskin and C. E. Guestrin, "Robust probabilistic inference in distributed systems," in Proc. 20th Conf. Uncertain. Artif. Intell. (UAI '04), 2004, pp. 436-445.
90. M. A. Paskin, C. E. Guestrin, and J. McFadden, "A robust architecture for inference in sensor networks," in Proc. 4th Int. Symp. Inf. Process. Sensor Netw. (IPSN '05), 2005.
91. F. Dellaert, A. Kipp, and P. Krauthausen, "A multifrontal QR factorization approach to distributed inference applied to multirobot localization and mapping," in Proc. Nat. Conf. Artif. Intell. (AAAI 05), 2005, pp. 1261-1266.
92. W. H. Widen, "Smart cameras and the right to privacy," Proc. IEEE, vol. 96, no. 10, pp. 1688-1697, Oct. 2008.