Shape, illumination, and reflectance from shading

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 37, Issue 8, 2015, Pages 1670-1687

  Barron, Jonathan T ^a   Malik, Jitendra ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Color constancy; Computer vision; Intrinsic images; Machine learning; Shape estimation; Shape from shading

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; OPTIMIZATION; PAINT; REFLECTION;

COLOR CONSTANCY; COMPUTER VISION PROBLEMS; ILLUMINATION ESTIMATION; INTRINSIC IMAGES; OPTIMIZATION PROBLEMS; SHAPE ESTIMATION; SHAPE FROM SHADING; STATISTICAL INFERENCE;

COMPUTER VISION;

EID: 84947217753     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2014.2377712     Document Type: Article

References (64)

1. E. Adelson and A. Pentland, "The perception of shading and reflectance", in Perception As Bayesian Inference, New York, NY, USA: Cambridge Univ. Press, 1996.
2. D. Field, "Relations between the statistics of natural images and the response properties of cortical cells", J. Opt. Soc. Amer. A, vol. 4, pp. 2379-2394, 1987.
3. D. Ruderman and W. Bialek, "Statistics of natural images: Scaling in the woods", Phys. Rev. Lett., vol. 73, pp. 814-817, 1994.
4. R. Grosse, M. K. Johnson, E. H. Adelson, and W. T. Freeman, "Ground-truth dataset and baseline evaluations for intrinsic image algorithms", in Proc. Int. Conf. Comput. Vis., 2009, pp. 2335-2342.
5. J. T. Barron and J. Malik, "High-frequency shape and albedo from shading using natural image statistics", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 2521-2528.
6. J. T. Barron and J. Malik, "Shape, albedo, and illumination from a single image of an unknown object", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2012, pp. 334-341.
7. J. T. Barron and J. Malik, "Color constancy, intrinsic images, and shape estimation", in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 57-70.
8. A. Gilchrist, Seeing in Black and White. London, U. K.: Oxford Univ. Press, 2006.
9. E. H. Land and J. J. McCann, "Lightness and retinex theory", J. Opt. Soc. Amer., vol. 61, pp. 1-11, 1971.
10. B. K. P. Horn, "Determining lightness from an image", Comput. Graph. Image Process., vol. 3, pp. 277-299, 1974.
11. H. Barrow and J. Tenenbaum, "Recovering intrinsic scene characteristics from images", in Computer Vision Systems, New York, NY, USA: Academic, 1978.
12. M. Bell and W. T. Freeman, "Learning local evidence for shading and reflectance", in Proc. IEEE 8th Int. Conf. Comput Vis., 2001, pp. 670-677.
13. J. Shen, X. Yang, Y. Jia, and X. Li, "Intrinsic images using optimization", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 3481-3487.
14. M. F. Tappen, W. T. Freeman, and E. H. Adelson, "Recovering intrinsic images from a single image", IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 9, pp. 1459-1472, Sep. 2005.
15. P. Gehler, C. Rother, M. Kiefel, L. Zhang, and B. Schoelkopf, "Recovering intrinsic images with a global sparsity prior on reflectance", in Proc. Adv. Neural Inf. Process. Syst., 2011, pp. 765-773.
16. D. A. Forsyth, "A novel algorithm for color constancy", Int. J. Comput. Vis., vol. 5, pp. 5-36, 1990.
17. L. T. Maloney and B. A. Wandell, "Color constancy: A method for recovering surface spectral reflectance", J. Opt. Soc. Amer. A, vol. 3, pp. 29-33, 1986.
18. G. Klinker, S. Shafer, and T. Kanade, "A physical approach to color image understanding", Int. J. Comput. Vis., vol. 4, pp. 7-38, 1990.
19. G. Finlayson, S. Hordley, and P. Hubel, "Color by correlation: A simple, unifying framework for color constancy", IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 11, pp. 1209-1221, Nov. 2001.
20. D. H. Brainard and W. T. Freeman, "Bayesian color constancy", J. Opt. Soc. Amer. A, vol. 14, pp. 1393-1411, 1997.
21. B. K. P. Horn, "Obtaining shape from shading information", in The Psychology of Computer Vision, New York, NY, USA: McGraw-Hill, 1975.
22. T. Rindfleisch, "Photometric method for lunar topography", Photogrammetric Eng., vol. 40, pp. 101-111, 1966.
23. M. J. Brooks and B. K. P. Horn, Shape from Shading. Cambridge, MA, USA: MIT Press, 1989.
24. R. Zhang, P.-S. Tsai, J. E. Cryer, and M. Shah, "Shape-from-shading: A survey", IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 8, pp. 690-706, Aug. 2002.
25. P. Belhumeur, D. Kriegman, and A. Yuille, "The bas-relief ambiguity", Int. J. Comput. Vis., vol. 35, pp. 33-44, 1999.
26. J. Koenderink, A. van Doorn, C. Christou, and J. Lappin, "Shape constancy in pictorial relief", Perception, vol. 25, pp. 155-164, 1996.
27. K. Ikeuchi and B. Horn, "Numerical shape from shading and occluding boundaries", Artif. Intell., vol. 17, pp. 141-184, 1981.
28. H. Boyaci, K. Doerschner, J. L. Snyder, and L. T. Maloney, "Surface color perception in three-dimensional scenes", Vis. Neurosci., vol. 23, pp. 311-321, 2006.
29. R. Woodham, "Photometric method for determining surface orientation from multiple images", Optical Eng., vol. 19, pp. 139-144, 1980.
30. R. Basri, D. Jacobs, and I. Kemelmacher, "Photometric stereo with general, unknown lighting", Int. J. Comput. Vis., vol. 72, pp. 239-257, 2007.
31. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U. K.: Cambridge Univ. Press, 2003.
32. B. Triggs, P. F. McLauchlan, R. I. Hartley, and A. W. Fitzgibbon, "Bundle adjustment-a modern synthesis", in Proc. Int. Workshop Vis. Algorithms: Theory Practice, pp. 298-372, 1999.
33. Y. Yu, P. Debevec, J. Malik, and T. Hawkins, "Inverse global illumination: Recovering reflectance models of real scenes from photographs", in Proc. 26th Annu. Conf. Comput. Graph. Interactive Techn., 1999, pp. 215-224.
34. D. Hoiem, A. A. Efros, and M. Hebert, "Recovering surface layout from an image", Int. J. Comput. Vis., vol. 75, pp. 151-172, 2007.
35. A. Saxena, M. Sun, and A. Ng, "Make3d: Learning 3d scene structure from a single still image", IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 5, pp. 824-840, May 2008.
36. V. Blanz and T. Vetter, "A morphable model for the synthesis of 3D faces", in Proc. 26th Annu. Conf. Comput. Graph. Interactive Techn., 1999, pp. 187-194.
37. J. Huang and D. Mumford, "Statistics of natural images and models", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 1999, pp. 541-547.
38. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, "Image denoising using scale mixtures of Gaussians in the wavelet domain", IEEE Trans. Image Process., vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
39. S. Roth and M. J. Black, "Fields of experts: A framework for learning image priors", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2005, pp. 860-867.
40. R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. Freeman, "Removing camera shake from a single photograph", in Proc. ACM SIGGRAPH Papers, 2006, pp. 787-794.
41. J. Huang, A. B. Lee, and D. Mumford, "Statistics of range images", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2000, pp. 324-331.
42. F. Romeiro and T. Zickler, "Blind reflectometry", in Proc. 11th Eur. Conf. Comput. Vis., 2010, pp. 45-58.
43. R. O. Dror, A. S. Willsky, and E. H. Adelson, "Statistical characterization of real-world illumination", J. Vis., vol. 4, p. 11, 2004.
44. B. K. P. Horn, "Shape from shading: A method for obtaining the shape of a smooth opaque object from one view", Comput. Sci. Artif. Intell. Lab, Massachusetts Inst. Technol., Cambridge, MA, USA, Tech. Rep. AITR-232, 1970.
45. R. Ramamoorthi and P. Hanrahan, "An efficient representation for irradiance environment maps", in Proc. 28th Annu. Conf. Comput. Graph. Interactive Techn., 2001, pp. 497-500.
46. N. Alldrin, S. Mallick, and D. Kriegman, "Resolving the generalized bas-relief ambiguity by entropy minimization", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2007, pp. 1-7
47. G. D. Finlayson, M. S. Drew, and C. Lu, "Entropy minimization for shadow removal", Int. J. Comput. Vis., vol. 85, pp. 35-57, 2009.
48. I. Omer and M. Werman, "Color lines: Image specific color representation", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2004, pp. II-946-II-953.
49. J. C. Principe and D. Xu, "Learning from examples with quadratic mutual information", in Proc. IEEE Signal Process. Soc. Workshop Neural Netw. Signal Process., 1998, pp. 155-164.
50. J. Chen, S. Paris, and F. Durand, "Real-time edge-aware image processing with the bilateral grid", in Proc. ACM SIGGRAPH Papers, 2007, http://www.siggraph.org/publications/bibliography/index.php?detail=21157
51. S. Bell, P. Upchurch, N. Snavely, and K. Bala, "Opensurfaces: A richly annotated catalog of surface appearance", ACM Trans. Graphics, vol. 32, p. 111, 2013.
52. O. Woodford, P. Torr, I. Reid, and A. Fitzgibbon, "Global stereo reconstruction under second-order smoothness priors", IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 12, pp. 2115-2128, Dec. 2009.
53. D. Hilbert and C. S. Vossen, Geometry and the Imagination. New York, NY, USA: Chelsea, 1956.
54. H. P. Moreton and C. H. Séquin, "Functional optimization for fair surface design", in Proc. 19th Annu. Conf. Comput. Graph. Interactive Techn., 1992, pp. 167-176.
55. P. Besl and R. Jain, "Segmentation through variable-order surface fitting", IEEE Trans. Pattern Anal. Mach. Intell., vol. 10, no. 2, pp. 167-192, Mar. 1988.
56. D. A. Forsyth, "Variable-source shading analysis", Int. J. Comput. Vis., vol. 91, pp. 280-302, 2011.
57. J. Koenderink, "What does the occluding contour tell us about solid shape?" Perception, vol. 13, pp. 321-330, 1984.
58. P. Mamassian, D. Kersten, and D. C. Knill, "Categorical local-shape perception", Perception, vol. 25, pp. 95-107, 1996.
59. M. Brady and A. Yuille, "An extremum principle for shape from contour", IEEE Trans. Pattern Anal. Mach. Intell., vol. 6, no. 3, pp. 288-301, Mar. 1984.
60. J. Malik, "Interpreting line drawings of curved objects", Int. J. Comput. Vis., vol. 1, pp. 73-104, 1987.
61. D. Terzopoulos, "Image analysis using multigrid relaxation methods", IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 2, pp. 129-139, Mar. 1986.
62. M. K. Johnson and E. H. Adelson, "Shape estimation in natural illumination", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 2553-2560.
63. D. Forsyth and A. Zisserman, "Reflections on shading", IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 7, pp. 671-679, Jul. 1991.
64. J. T. Barron and J. Malik, "Intrinsic scene properties from a single RGB-D image", in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 17-24.