Scene reconstruction and visualization from community photo collections

Proceedings of the IEEE

Volumn 98, Issue 8, 2010, Pages 1370-1390

  Snavely, Noah ^a   Simon, Ian ^b   Goesele, Michael ^c   Szeliski, Richard ^d   Seitz, Steven M ^b,e  

^a School of Operations Research and Information Engineering   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

^c DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

^d MICROSOFT RESEARCH   (United States)

^e GOOGLE INC   (United States)

Author keywords

3 D navigation and visualization; 3 D reconstruction; Internet photo collections; Multiview stereo; Scene summarization; Structure from motion

Indexed keywords

COMPUTER VISION; STEREO IMAGE PROCESSING; THREE DIMENSIONAL COMPUTER GRAPHICS; VISUALIZATION;

3-D NAVIGATION AND VISUALIZATIONS; 3D RECONSTRUCTION; INTERNET PHOTO COLLECTIONS; MULTI-VIEW STEREO; SCENE SUMMARIZATIONS; STRUCTURE FROM MOTION;

IMAGE RECONSTRUCTION;

EID: 77954863556     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2010.2049330     Document Type: Conference Paper

References (83)

1. D CitiesVSpatio-Temporal Reconstruction from Images. [Online]. Available: http://www-static.cc.gatech.edu/projects/4d-cities/dhtml/index.html
2. S. Ahern, M. Naaman, R. Nair, and J. H. Yang, BWorld explorer: Visualizing aggregate data from unstructured text in geo-referenced collections, [in Proc. Conf. Digital Libraries, 2007, pp. 1-10.
3. S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu, BAn optimal algorithm for approximate nearest neighbor searching fixed dimensions, [J. Assoc. Comput. Mach., vol. 45, no. 6, pp. 891-923, 1998.
4. S. Boivin and A. Gagalowicz, BImage-based rendering of diffuse, specular and glossy surfaces from a single image, [in Proc. SIGGRAPH, 2001, pp. 107-116.
5. M. Brown and D. Lowe, BUnsupervised 3D object recognition and reconstruction in unordered datasets, [in Proc. Int. Conf. 3D Imag. Model., 2003, pp. 1218-1225.
6. C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen, BUnstructured lumigraph rendering, [in Proc. SIGGRAPH, 2001, pp. 425-432.
7. S. Chen and L. Williams, BView interpolation for image synthesis, [in Proc. 20th Annu. Conf. Comput. Graph. Interactive Tech., 1993, pp. 279-288.
8. O. Chum, M. Perdoch, and J. Matas, BGeometric min-hashing: Finding a (thick) needle in a haystack, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 17-24.
9. O. Chum, J. Philbin, J. Sivic, M. Isard, and A. Zisserman, BTotal recall: Automatic query expansion with a generative feature model for object retrieval, [in Proc. Int. Conf. Comput. Vis., 2007, DOI: 10.1109/ICCV.2007.4408891.
10. D. Crandall, L. Backstrom, D. Huttenlocher, and J. Kleinberg, BMapping the world's photos, [in Proc. Int. World Wide Web Conf., 2009, pp. 761-770.
11. P. Debevec, BRendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography, [in Proc. SIGGRAPH, 1998, pp. 189-198.
12. P. Debevec, BMaking FThe Parthenon-, [in Proc. Int. Symp. Virtual Reality Archaeol. Cultural Heritage, 2005.
13. P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, BAcquiring the reflectance field of a human face, [in Proc. SIGGRAPH, 2000, pp. 145-156.
14. P. Debevec, C. Tchou, A. Gardner, T. Hawkins, C. Poullis, J. Stumpfel, A. Jones, N. Yun, P. Einarsson, T. Lundgren, M. Fajardo, and P. Martinez, BEstimating surface reflectance properties of a complex scene under captured natural illumination, " Univ. Southern California Inst. Creative Technol. (USC ICT), Marina del Rey, CA, Tech. Rep. ICT-TR-06.2004, 2004.
15. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, BIndexing by latent semantic analysis, [J. Amer. Soc. Inf. Sci., vol. 41, no. 6, pp. 391-407, 1990.
16. E. W. Dijkstra, BA note on two problems in connection with graphs, [Numerische Mathematik, vol. 1, no. 1, pp. 269-271, Dec. 1959.
17. B. Epshtein, E. Ofek, Y. Wexler, and P. Zhang, BHierarchical photo organization using geo-relevance, [in Proc. ACM Int. Symp. Adv. Geographic Inf. Syst., 2007. [Online]. Available: http://doi.acm.org/10.1145/1341012.1341036.
18. O. Faugeras and Q.-T. Luong, The Geometry of Multiple Images. Cambridge, MA: MIT Press, Mar. 2001.
19. Flickr. [Online]. Available: http://www. flickr.com.
20. W. Förstner, BA feature-based correspondence algorithm for image matching, [Int. Arch. Photogrammetry Remote Sens., vol. 26, no. 3, pp. 150-166, 1986.
21. Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski, BReconstructing building interiors from images, [in Proc. Int. Conf. Comput. Vis., 2009, pp. 80-87.
22. M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S. M. Seitz, BMulti-view stereo for community photo collections, [in Proc. Int. Conf. Comput. Vis., 2007. [Online]. Available: doi.ieeecomputersociety.org/10.1109/ICCV. 2007.4408933.
23. Google Maps. [Online]. Available: http://maps. google.com.
24. S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, BThe lumigraph, [in Proc. SIGGRAPH, 1996, pp. 43-54.
25. MGraphvizVGraph Visualization Software. [Online]. Available: http://www.graphviz.org/.
26. K. Grauman and T. Darrell, BThe pyramid match kernel: Discriminative classification with sets of image features, [in Proc. Int. Conf. Comput. Vis., 2005, pp. 1458-1465.
27. T. Haber, C. Fuchs, P. Bekaert, H.-P. Seidel, M. Goesele, and H. P. A. Lensch, BRelighting objects from image collections, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 627-634.
28. C. Hand, BA survey of 3D interaction techniques, [Comput. Graph. Forum, vol. 16, no. 5, pp. 269-281, 1997.
29. C. Harris and M. J. Stephens, BA combined corner and edge detector, [in Proc. Alvey Vis. Conf., 1988, pp. 147-152.
30. R. I. Hartley and A. Zisserman, Multiple View Geometry. Cambridge, U.K.: Cambridge Univ. Press, Sep. 2000.
31. J. Hays and A. A. Efros, BIM2GPS: Estimating geographic information from a single image, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, DOI: 10.1109/CVPR.2008. 4587784.
32. T. Hofmann, BProbabilistic latent semantic analysis, [in Proc. Uncertainty Artif. Intell., Stockholm, Sweden, 1999, pp. 289-296.
33. A. Irschara, C. Zach, and H. Bischof, BTowards Wiki-based dense city modeling, [in Proc. Workshop Virtual Represent. Model. Large-Scale Environ., 2007, DOI: 10.1109/ICCV.2007.4409216.
34. A. Jaffe, M. Naaman, T. Tassa, and M. Davis, BGenerating summaries for large collections of geo-referenced photographs, [in Proc. Int. Conf. World Wide Web, 2006, pp. 853-854.
35. L. Kennedy and M. Naaman, BGenerating diverse and representative image search results for landmarks, [in Proc. Int. World Wide Web Conf., 2008, pp. 297-306.
36. E. Kruppa, BZur Ermittlung eines Objektes aus zwei Perspektiven mit innerer Orientierung, [Sitz.-Ber. Akad. Wiss., Wien, Math. Naturw. Kl., Abt. Ila., vol. 122, pp. 1939-1948, 1913.
37. H. P. A. Lensch, BEfficient, image-based appearance acquisition of real-world objects, [Ph.D. dissertation, Dept. Comput. Sci., Saarland Univ., Saarbrücken, Germany, 2004.
38. H. P. A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, BImage-based reconstruction of spatial appearance and geometric detail, [ACM Trans. Graphics, vol. 22, no. 2, pp. 234-257, 2003.
39. M. Levoy and P. Hanrahan, BLight field rendering, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 1996, pp. 31-42.
40. Y. Li, D. Crandall, and D. P. Huttenlocher, BLandmark classification in large-scale image collections, [in Proc. Int. Conf. Comput. Vis., 2009, pp. 2-9.
41. A. Lippman, BMovie maps: An application of the optical videodisc to computer graphics, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 1980, pp. 32-43.
42. H. C. Longuet-Higgins, BA computer algorithm for reconstructing a scene from two projections, [Nature, vol. 293, pp. 133-135, 1981.
43. D. G. Lowe, BDistinctive image features from scale-invariant keypoints, [Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
44. B. D. Lucas and T. Kanade, BAn iterative image registration technique with an application in stereo vision, [in Proc. Int. Joint Conf. Artif. Intell., 1981, pp. 674-679.
45. S. Marschner, BInverse rendering for computer graphics, [Ph.D. dissertation, Dept. Comput. Sci., Cornell Univ., Ithaca, NY, 1998.
46. S. Marschner, S. Westin, E. Lafortune, K. Torrance, and D. Greenberg, BImage-based BRDF measurement including human skin, [in Proc. Eurographics Rendering Workshop, 2000, pp. 139-152.
47. L. McMillan and G. Bishop, BPlenoptic modeling: An image-based rendering system, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 1995, pp. 39-46.
48. K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and L. J. Van Gool, BA comparison of affine region detectors, [Int. J. Comput. Vis., vol. 65, no. 1-2, pp. 43-72, Nov. 2005.
49. R. Ng, R. Ramamoorthi, and P. Hanrahan, BTriple product wavelet integrals for all-frequency relighting, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 2004, pp. 477-487.
50. D. Nist́er and H. Steẃenius, BScalable recognition with a vocabulary tree, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2006, pp. 2118-2125.
51. D. Nist́er and H. Steẃenius, BScalable recognition with a vocabulary tree, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2006, pp. 2118-2125.
52. G. P. Otto and T. K. W. Chau, BFRegion-growing- algorithm for matching of terrain images, [Image Vis. Comput., vol. 7, no. 2, pp. 83-94, 1989.
53. Photobucket. [Online]. Available: http://photobucket.com/.
54. Microsoft's Photosynth. [Online]. Available: http://photosynth.net.
55. T. Quack, B. Leibe, and L. Van Gool, BWorld-scale mining of objects and events from community photo collections, [in Proc. Int. Conf. Content-Based Image Video Retrieval, 2008, pp. 47-56.
56. R. Ramamoorthi and P. Hanrahan, BA signal-processing framework for inverse rendering, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 2001, pp. 117-128.
57. Rephotography. [Online]. Available: http://en. wikipedia.org/wiki/ Rephotography.
58. F. Schaffalitzky and A. Zisserman, BMulti-view matching for unordered image sets, or FHow do I organize my holiday snaps?-, " in Proc. Eur. Conf. Comput. Vis., 2002, vol. I, pp. 414-431.
59. G. Schindler, F. Dellaert, and S. B. Kang, BInferring temporal order of images from 3D structure, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2007, DOI: 10.1109/CVPR. 2007.383088.
60. S. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, BA comparison and evaluation of multi-view stereo reconstruction algorithms, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2006, vol. 1, pp. 519-526.
61. I. Simon, N. Snavely, and S. M. Seitz, BScene summarization for online image collections, [in Proc. Int. Conf. Comput. Vis., 2007, DOI: 10.1109/ICCV.2007.4408863.
62. I. Simon and S. M. Seitz, BScene segmentation using the wisdom of crowds, [in Proc. Eur. Conf. Comput. Vis., 2008, pp. 541-553.
63. S. N. Sinha, D. Steedly, and R. Szeliski, BPiecewise planar stereo for image-based rendering, [in Proc. Int. Conf. Comput. Vis., 2009.
64. J. Sivic and A. Zisserman, BEfficient visual search of videos cast as text retrieval, [IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 4, pp. 591-606, Apr. 2009.
65. N. Snavely, S. M. Seitz, and R. Szeliski, BModeling the world from Internet photo collections, [Int. J. Comput. Vis., vol. 80, no. 2, pp. 189-210, Nov. 2008.
66. N. Snavely, S. M. Seitz, and R. Szeliski, BPhoto tourism: Exploring photo collections in 3D, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 2006, pp. 835-846.
67. N. Snavely, R. Garg, S. M. Seitz, and R. Szeliski, BFinding paths through the world's photos, [in Proc. Annu. Conf. Comput. Graph. Interactive Tech., 2008, article 15.
68. N. Snavely, S. M. Seitz, and R. Szeliski, BSkeletal graphs for efficient structure from motion, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2008, DOI: 10.1109/CVPR. 2008.4587678.
69. M. E. Spetsakis and J. Y. Aloimonos, BA multiframe approach to visual motion perception, [Int. J. Comput. Vis., vol. 6, no. 3, pp. 245-255, Aug. 1991.
70. R. Szeliski and S. B. Kang, BRecovering 3D shape and motion from image streams using nonlinear least squares, [J. Visual Commun. Image Represent., vol. 5, no. 1, pp. 10-28, Mar. 1994.
71. C. Tomasi and T. Kanade, BShape and motion from image streams under orthography: A factorization method, [Int. J. Comput. Vis., vol. 9, no. 2, pp. 137-154, Nov. 1992.
72. B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, BBundle adjustmentVA modern synthesis, [in Proc. Int. Workshop Vis. Algorithms, Sep. 1999, pp. 298-372.
73. L. Vincent, BTaking online maps down to street level, [IEEE Computer, vol. 40, no. 12, pp. 118-120, Dec. 2007.
74. R. Wang, R. Ng, D. Luebke, and G. Humphreys, BEfficient wavelet rotation for environment map rendering, [in Proc. Eurograhics Symp. Rendering, 2006, pp. 173-182.
75. Wikipedia. [Online]. Available: http://www. wikipedia.org.
76. Windows Bing Maps. [Online]. Available: http://maps.bing.com.
77. T. Yu, H. Wang, N. Ahuja, and W.-C. Chen, BSparse lumigraph relighting by illumination and reflectance estimation from multi-view images, [in Proc. Eurographics Symp. Rendering, 2006, pp. 41-50.
78. Y. Yu and J. Malik, BRecovering photometric properties of architectural scenes from photographs, [in Proc. 25th Annu. Conf. Comput. Graph. Interactive Tech., 1998, pp. 207-217.
79. Y.-T. Zheng, M. Zhao, Y. Song, H. Adam, U. Buddemeier, A. Bissacco, F. Brucher, T.-S. Chua, and H. Neven, BTour the world: Building a web-scale landmark recognition engine, [in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1085-1092.
80. T. Zickler, R. Ramamoorthi, S. Enrique, and P. Belhumeur, BReflectance sharing: Predicting appearance from a sparse set of images of a known shape, [Trans. Pattern Anal. Mach. Intell., vol. 28, no. 8, pp. 1287-1302, Aug. 2006.
81. S. Agarwal, N. Snavely, I. Simon, S. M. Seitz, and R. Szeliski, BBuilding Rome in a day, [in Proc. 12th IEEE Int. Conf. Comput. Vis., Kyoto, Japan, Sep. 2009, pp. 72-79. [Online]. Available: http://grail.cs.washington.edu/rome/.
82. S. Agarwal, Y. Furukawa, N. Snavely, B. Curless, S. M. Seitz, and R. Szeliski, BReconstructing Rome, [Computer, vol. 43, no. 6, pp. 40-47, Jun. 2010. [Online]. Available: http://grail.cs.washington. edu/rome/.
83. Y. Furukawa, B. Curless, S. M. Seitz, and R. Szeliski, BTowards Internet-scale multi-view stereo, [in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., San Francisco, CA, Jun. 2010.