Toward dynamic image mosaic generation with robustness to parallax

IEEE Transactions on Image Processing

Volumn 21, Issue 1, 2012, Pages 366-378

  Zhi, Qi ^a   Cooperstock, Jeremy R ^b  

^a School of Electronic Science and Engineering   (China)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Depth based image mosaicing (DBM); dynamic mosaic; object motion; parallax

Indexed keywords

CAMERA MOTIONS; DEPTH CUE; DYNAMIC IMAGES; FOREGROUND/BACKGROUND SEGMENTATION; IMAGE MOSAICING; INPUT IMAGE; INPUT SEQUENCE; MOSAICING; MOTION PERCEPTION; OBJECT MOTION; PARALLAX; SCENE CONSTRAINTS; SMOOTH TRANSITIONS; VIDEO MOSAIC;

IMAGE REGISTRATION;

GEOMETRICAL OPTICS;

ALGORITHM; ARTICLE; ARTIFACT; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

ALGORITHMS; ARTIFACTS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE;

EID: 84255188350     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2162743     Document Type: Article

References (60)

1. A. Agarwala, M. Agrawala, M. Cohen, D. Salesin, and R. Szeliski, "Photographing long scenes with multi-viewpoint panoramas," in Proc. 33th Annu. Conf. Comput. Graph. Interactive Techn., 2006, pp. 853-861. (Pubitemid 44481047)
2. A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, and M. Cohen, "Interactive digital photomontage," ACM Trans. Graph., vol. 23, no. 3, pp. 294-302, Aug. 2004.
3. R. A. Alex, Y. Pritch, D. Lischinski, and S. Peleg, "Dynamosaicing: Mosaicing of dynamic scenes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 10, pp. 1789-1801, Oct. 2007.
4. M. Bartoli, N. Dalal, B. Bose, and R. Horaud, "From video sequence to motion panoramas," in Proc.Workshop Motion Video Comput., MOTION, 2002, pp. 201-207.
5. H. Bay, A. Ess, T. Tuytelaars, and L. J. Van Gool, "Speeded-up robust features (SURF)," Comput. Vis. Image Process., vol. 110, no. 3, pp. 346-359, Jun. 2008.
6. J. V. Bouguet, Camera Calibration Toolbox for Matlab 2003 [Online]. Available: http://www.vision.caltech.edu/bouguetj/calib-doc/
7. Y. Boykov and V. Kolmogorov, "An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, pp. 1124-1137, Sep. 2004.
8. Y. Boykov, R. Zabih, and S. Gortler, "Multi-camera scene reconstruction via graph cuts," in Proc. 7th ECCV-Part III, 2002, pp. 82-96.
9. C. Bregler, A. Hertzmann, and H. Biermann, "Recovering non-rigid 3d shape from image streams," in Proc. Comput. Vis. Pattern Recog., 2000, pp. 690-696.
10. M. Brown and D. G. Lowe, "Recognising panoramas," in Proc. IEEE Int. Conf. Comput. Vis., 2003, pp. 1218-1225.
11. M. Brown and D. G. Lowe, "Automatic panoramic image stitching using invariant features," Int. J. Comput. Vis., vol. 74, no. 1, pp. 59-73, Aug. 2007. (Pubitemid 46673104)
12. P. J. Burt and E. H. Adelson, "A multiresolution spline with application to image mosaics," ACMTrans. Graph., vol. 2, no. 4, pp. 217-236, Oct. 1983.
13. S. C. Chen, "Quicktime VR: An image-based approach to virtual environment navigation," in Proc. 22nd Annu. Conf. Comput. Graph. Interactive Techn., SIGGRAPH, 1995, pp. 29-38.
14. J. Civera, A. J. Davison, J. A. Magallón, and J. M. Montiel, "Drift-free real-time sequential mosaicing," Int. J. Comput. Vis., vol. 81, no. 2, pp. 128-137, Feb. 2009.
15. R. Collins, "A space-sweep approach to true multi-image matching," in Proc. Comput. Vis. Pattern Recog., 1996, pp. 358-363.
16. C. Doutre and P. Nasiopoulos, "Fast vignetting correction and color matching for panoramic image stitching," in Proc. IEEE Int. Conf. Image Process., 2009, pp. 709-712.
17. A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proc. ACM SIGGRAPH, 2001, pp. 341-346. (Pubitemid 33047318)
18. B. Georgescu, I. Shimshoni, and P. Meer, "Mean shift based clustering in high dimensions: A texture classification example," in Proc. 9th IEEE ICCV, 2003, pp. 456-463.
19. R. Gupta and R. I. Hartley, "Linear pushbroom cameras," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 9, pp. 963-975, Sep. 1997. (Pubitemid 127762287)
20. C. Harris and M. J. Stephens, "A combined corner and edge detector," in Proc. 4th Alvey Vis. Conf., 1988, pp. 147-152.
21. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. New York: Cambridge Univ. Press, 2000.
22. M. Irani, P. Anandan, and S. Hsu, "Mosaic based representation of video sequence and their applications," in Proc. IEEE Int. Conf. Comput. Vis., 1995, pp. 605-611.
23. R. J. Jagacinski, W. W. Johnson, and R. A. Miller, "Quantifying the cognitive trajectories of extrapolated movements," J. Exp. Psychol.: Hum. Percept. Perform., vol. 9, no. 1, pp. 43-57, Feb. 1983.
24. J. Jia and C. K. Tang, "Image stitching using structure deformation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 4, pp. 617-631, Apr. 2008. (Pubitemid 351559716)
25. H. L. Jin, "A three-point minimal solution for panoramic stitching with lens distortion," in Proc. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
26. M. H. Ju and H. B. Kang, "A new simple method to stitch images with lens distortion," in Proc. Adv. Vis. Comput., 2010, pp. 273-282.
27. K. N. Kutulakos and S. M. Seitz, "A theory of shape by space carving," Int. J. Comput. Vis., vol. 38, no. 3, pp. 199-218, Jul./Aug. 2000.
28. V. Kwatra, A. Schodl, I. Essa, G. Turk, and A. Bobick, "Graphcut textures: Image and video synthesis using graph cuts," ACM Trans. Graph., vol. 22, no. 3, pp. 277-286, Jul. 2003.
29. D. S. Lee, "Effective Gaussian mixture learning for video background subtraction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 827-832, May 2005.
30. A. Levin, A. Zomet, S. Peleg, and Y.Weiss, "Seamless image stitching in the gradient domain," in Proc. Eur. Conf. Comput. Vis., 2004, pp. 377-389.
31. S. Lovegrove and A. J. Davison, "Real-time spherical mosaicing using whole image alignment," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 73-86.
32. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
33. A. Mills and G. Dudek, "Image stitching with dynamic elements," Image Vis. Comput., vol. 27, no. 10, pp. 1593-1602, Sep. 2009.
34. D. Newtson and G. Engquist, "The perceptual organization of ongoing behavior," J. Exp. Social Psychol., vol. 12, no. 5, pp. 436-450, Sep. 1976.
35. S. Peleg and J. Herman, "Panoramic mosaics by manifold projection," in Proc. Comput. Vis. Pattern Recog., 1997, pp. 338-343.
36. S. Peleg, B. Rousso, A. Rav-Acha, and A. Zomet, "Mosaicing on adaptive manifolds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 10, pp. 1144-1154, Oct. 2000. (Pubitemid 32066971)
37. K. Pulli, M. Tico, and Y. Xiong, "Mobile panoramic imaging system," in Proc. ECVW, 2010, pp. 108-115.
38. Z. Qi and J. R. Cooperstock, "Depth-based image mosaicing for both static and dynamic scenes," in Proc. 19th ICPR, 2008, pp. 1-4.
39. P. Rademacher and G. Bishop, "Multiple-center-of-projection images," in Proc. 25th Annu. Conf. Comput. Graph. Interactive Techn., SIGGRAPH, 1998, pp. 199-206.
40. C. Rao, A. Yilmaz, and M. Shah, "View-invariant representation and recognition of actions," Int. J. Comput. Vis., vol. 50, no. 2, pp. 203-226, Nov. 2002. (Pubitemid 35311927)
41. H. S. Sawhney and S. Ayer, "Compact representations of videos through dominant and multiple motion estimation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 8, pp. 814-830, Aug. 1996. (Pubitemid 126772823)
42. D. Scharstein and R. Szeliski, "High-accuracy stereo depth maps using structured light," in Proc. Comput. Vis. Pattern Recog., Jun. 2003, pp. 195-202.
43. S. M. Seitz and J. Kim, "The space of all stereo images," Int. J. Comput. Vis., vol. 48, no. 1, pp. 21-38, Jun. 2002. (Pubitemid 34491073)
44. H. Y. Shum, S. C. Chan, and S. B. Kang, Image-Based Rendering. Secaucus, NJ: Springer-Verlag, 2006.
45. H. Y. Shum and R. Szeliski, "Construction and refinement of panoramic mosaics with global and local alignment," in Proc. Int. Conf. Comput. Vis., 1998, pp. 953-960.
46. C. Stauffer and W. E. L. Grimson, "Adaptive background mixture models for real-time tracking," in Proc. Comput. Vis. Pattern Recog., 1999, pp. 23-25.
47. R. Szeliski and P. Golland, "Stereo matching with transparency and matting," Int. J. Comput. Vis., vol. 32, no. 1, pp. 45-61, Aug. 1999.
48. H. Tao, S. H. Sawhney, and R. Kumar, "A global matching framework for stereo computation," in Proc. 8th IEEE ICCV, 2001, pp. 532-547.
49. C. Tomasi and T. Kanade, "Shape and motion from image streams under orthography: A factorization method," Int. J. Comput. Vis., vol. 9, no. 2, pp. 137-154, Nov. 1992. (Pubitemid 23621943)
50. L. Torresani, A. Hertzmann, and C. Bregler, "Nonrigid structure-frommotion: Estimating shape and motion with hierarchical priors," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 5, pp. 878-892, May 2008. (Pubitemid 351486755)
51. M. Uyttendaele, A. Eden, and R. Szeliski, "Eliminating ghosting and exposure artifacts in image mosaics," in Proc. Comput. Vis. Pattern Recog., 2001, pp. 509-516.
52. L. Wang, J. Traub, S. Weidert, S. M. Heining, E. Euler, and N. Navab, "Parallax-free intro-operative X-ray image stitching," Med. Image Process., vol. 14, pp. 674-686, 2010.
53. W. Xu and J. Mulligan, "Performance evaluation of color correction approaches for automatic multi-view image and video stitching," in Proc. Comput. Vis. Pattern Recog., 2010, pp. 263-270.
54. Y. Liu, G. Chen, N. Max, C. Hofsetz, and P. McGuinness, "Undersampled light field rendering by a plane sweep," Comput. Graph. Forum, vol. 25, no. 2, pp. 225-236, Jun. 2006. (Pubitemid 43952836)
55. R. Yang, M. Pollefeys, H. Yang, and G. Welch, "A unified approach to real-time multi-resolution multi-baseline 2D view synthesis and 3D depth estimation using commodity graphics hardware," Int. J. Image Graph., vol. 4, no. 4, pp. 627-651, Oct. 2004.
56. W. Yang, J. Zheng, J. Cai, S. Rahardja, and C. W. Chen, "Natural and seamless image composition with color control," IEEE Trans. Image Process., vol. 18, no. 11, pp. 2584-2592, Nov. 2009.
57. J. M. Zacks and B. Tversky, "Event structure in perception and conception," Psychol. Bull., vol. 127, no. 1, pp. 3-21, Jan. 2001. (Pubitemid 33612309)
58. Y. Zhang and C. Kambhamettu, "Stereo matching with segmentationbased cooperation," in Proc. 7th ECCV-Part II, 2002, pp. 556-571.
59. Z. Y. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 11, pp. 1330-1334, Nov. 2000.
60. A. Zomet, D. Feldman, S. Peleg, and D. Weinshall, "Mosaicing new views: The crossed-slits projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 6, pp. 741-754, Jun. 2003.