A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor

IEEE Transactions on Image Processing

Volumn 24, Issue 10, 2015, Pages 3048-3059

  Monno, Yusuke ^a   Kikuchi, Sunao ^b   Tanaka, Masayuki ^a   Okutomi, Masatoshi ^a  

^a Department of Mechanical and Control Engineering Tokyo Institute of Technology ^*  (Japan)

^b Olympus Research and Development Center   (Japan)

Author keywords

Demosaicking; guided filter; interpolation; multispectral filter array (MSFA); multispectral imaging

Indexed keywords

COLOR; COLOR IMAGE PROCESSING; COMPRESSED SENSING; IMAGING SYSTEMS; INTERPOLATION;

BAYER COLOR FILTER ARRAY; DEMOSAICKING; GUIDED FILTERS; MULTI-SPECTRAL CAMERAS; MULTI-SPECTRAL FILTER ARRAYS; MULTI-SPECTRAL IMAGING SYSTEMS; MULTISPECTRAL IMAGING; SINGLE-SENSOR IMAGING;

ALGORITHMS;

EID: 84933544246     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2436342     Document Type: Article

References (60)

1. A. Ribes, R. Pillay, F. Schmitt, and C. Lahanier, "Studying that smile," IEEE Signal Process. Mag., vol. 25, no. 4, pp. 14-26, Jul. 2008.
2. K. Ohsawa et al., "Six band HDTV camera system for spectrum-based color reproduction," J. Imag. Sci. Technol., vol. 48, no. 2, pp. 85-92, 2004.
3. J.-I. Park, M.-H. Lee, M. D. Grossberg, and S. K. Nayar, "Multispectral imaging using multiplexed illumination," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Oct. 2007, pp. 1-8.
4. X. Cao, H. Du, X. Tong, Q. Dai, and S. Lin, "A prism-mask system for multispectral video acquisition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 12, pp. 2423-2435, Dec. 2011.
5. X. Cao, X. Tong, Q. Dai, and S. Lin, "High resolution multispectral video capture with a hybrid camera system," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 297-304.
6. R. Shrestha, A. Mansouri, and J. Y. Hardeberg, "Multispectral imaging using a stereo camera: Concept, design and assessment," EURASIP J. Adv. Signal Process., vol. 57, p. 1-15, Sep. 2011.
7. J. Y. Hardeberg, F. Schmitt, and H. Brettel, "Multispectral color image capture using a liquid crystal tunable filter," Opt. Eng., vol. 41, no. 10, pp. 2532-2548, 2002.
8. H. Fukuda, T. Uchiyama, H. Haneishi, M. Yamaguchi, and N. Ohyama, "Development of a 16-band multispectral image archiving system," Proc. SPIE, vol. 5667, pp. 136-145, Jan. 2005.
9. S. Han, I. Sato, T. Okabe, and Y. Sato, "Fast spectral reflectance recovery using DLP projector," in Proc. Asian Conf. Comput. Vis. (ACCV), 2010, pp. 323-335.
10. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, "Video rate spectral imaging using a coded aperture snapshot spectral imager," Opt. Exp., vol. 17, no. 8, pp. 6368-6388, 2009.
11. R. Habel, M. Kudenov, and M. Wimmer, "Practical spectral photography," Comput. Graph. Forum, vol. 31, no. 2, pp. 449-458, 2012.
12. N. A. Hagen and M. W. Kudenov, "Review of snapshot spectral imaging technologies," Opt. Eng., vol. 52, no. 9, pp. 090901-1-090901-23, 2013.
13. B. E. Bayer, "Color imaging array," U.S. Patent 3 971 065, Jul. 20, 1976.
14. R. Lukac, Ed., Single-Sensor Imaging: Methods and Applications for Digital Cameras. Boca Raton, FL, USA: CRC Press, 2008.
15. X. Li, B. Gunturk, and L. Zhang, "Image demosaicing: A systematic survey," Proc. SPIE, vol. 6822, pp. 68221J-1-68221J-15, Jan. 2008.
16. B. K. Gunturk, J. Glotzbach, Y. Altunbasak, R. W. Schafer, and R. M. Mersereau, "Demosaicking: Color filter array interpolation," IEEE Signal Process. Mag., vol. 22, no. 1, pp. 44-54, Jan. 2005.
17. D. Menon and G. Calvagno, "Color image demosaicking: An overview," Signal Process., Image Commun., vol. 26, nos. 8-9, pp. 518-533, 2011.
18. L. Miao and H. Qi, "The design and evaluation of a generic method for generating mosaicked multispectral filter arrays," IEEE Trans. Image Process., vol. 15, no. 9, pp. 2780-2791, Sep. 2006.
19. L. Miao, H. Qi, R. Ramanath, and W. E. Snyder, "Binary tree-based generic demosaicking algorithm for multispectral filter arrays," IEEE Trans. Image Process., vol. 15, no. 11, pp. 3550-3558, Nov. 2006.
20. J. Brauers and T. Aach, "A color filter array based multispectral camera," in Proc. 12th Workshop Farbbildverarbeitung, 2006.
21. X. Wang, J.-B. Thomas, J. Y. Hardeberg, and P. Gouton, "Median filtering in multispectral filter array demosaicking," Proc. SPIE, vol. 8660, pp. 86000E-1-86000E-10, Feb. 2013.
22. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, "Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum," IEEE Trans. Image Process., vol. 19, no. 9, pp. 2241-2253, Sep. 2010.
23. H. K. Aggarwal and A. Majumdar, "Single-sensor multi-spectral image demosaicing algorithm using learned interpolation weights," in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), Jul. 2014, pp. 2011-2014.
24. G. A. Baone and H. Qi, "Demosaicking methods for multispectral cameras using mosaic focal plane array technology," Proc. SPIE, vol. 6062, pp. 60620A-1-60620A-13, Jan. 2006.
25. Y. Monno, M. Tanaka, and M. Okutomi, "Multispectral demosaicking using adaptive kernel upsampling," in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2011, pp. 3218-3221.
26. Y. Monno, M. Tanaka, and M. Okutomi, "Multispectral demosaicking using guided filter," Proc. SPIE, vol. 8299, pp. 82990O-1-82990O-7, Jan. 2012.
27. K. He, J. Sun, and X. Tang, "Guided image filtering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397-1409, Jun. 2013.
28. K. Hirakawa and T. W. Parks, "Adaptive homogeneity-directed demosaicing algorithm," IEEE Trans. Image Process., vol. 14, no. 3, pp. 360-369, Mar. 2005.
29. D. Paliy, V. Katkovnik, R. Bilcu, S. Alenius, and K. Egiazarian, "Spatially adaptive color filter array interpolation for noiseless and noisy data: Articles," Int. J. Imag. Syst. Technol., vol. 17, no. 3, pp. 105-122, 2007.
30. D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, "Residual interpolation for color image demosaicking," in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2013, pp. 2304-2308.
31. D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, "Minimized-Laplacian residual interpolation for color image demosaicking," Proc. SPIE, vol. 9023, pp. 90230L-1-90230L-8, Mar. 2014.
32. S. Yamanaka, "Solid state camera," U.S. Patent 4 054 906, Oct. 18, 1977.
33. M. Parmar and S. J. Reeves, "A perceptually based design methodology for color filter arrays [image reconstruction]," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), vol. 3. May 2004, pp. III-473-III-476.
34. R. Lukac and K. N. Plataniotis, "Color filter arrays: Design and performance analysis," IEEE Trans. Consum. Electron., vol. 51, no. 4, pp. 1260-1267, Nov. 2005.
35. R. Lukac and K. N. Plataniotis, "Universal demosaicking for imaging pipelines with an RGB color filter array," Pattern Recognit., vol. 38, no. 11, pp. 2208-2212, 2005.
36. A. Hore and D. Ziou, "An edge-sensing generic demosaicing algorithm with application to image resampling," IEEE Trans. Image Process., vol. 20, no. 11, pp. 3136-3150, Nov. 2011.
37. K. Hirakawa and P. J. Wolfe, "Spatio-spectral color filter array design for optimal image recovery," IEEE Trans. Image Process., vol. 17, no. 10, pp. 1876-1890, Oct. 2008.
38. L. Condat, "A new color filter array with optimal properties for noiseless and noisy color image acquisition," IEEE Trans. Image Process., vol. 20, no. 8, pp. 2200-2210, Aug. 2011.
39. J. Gu, P. J. Wolfe, and K. Hirakawa, "Filterbank-based universal demosaicking," in Proc. 7th IEEE Int. Conf. Image Process. (ICIP), Sep. 2010, pp. 1981-1984.
40. L. Condat, "A generic variational approach for demosaicking from an arbitrary color filter array," in Proc. IEEE Int. Conf. Image Process. (ICIP), Nov. 2009, pp. 1625-1628.
41. T. Kijima, H. Nakamura, J. T. Compton, J. F. Hamilton, and T. E. De Weese, "Image sensor with improved light sensitivity," U.S. Patent 7 916 362, Mar. 29, 2011.
42. M. Kumar, E. O. Morales, J. E. Adams, and W. Hao, "New digital camera sensor architecture for low light imaging," in Proc. 16th IEEE Int. Conf. Image Process. (ICIP), Nov. 2009, pp. 2681-2684.
43. J. Wang, C. Zhang, and P. Hao, "New color filter arrays of high light sensitivity and high demosaicking performance," in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2011, pp. 3153-3156.
44. Y. M. Lu, C. Fredembach, M. Vetterli, and S. Süsstrunk, "Designing color filter arrays for the joint capture of visible and near-infrared images," in Proc. IEEE Int. Conf. Image Process. (ICIP), Nov. 2009, pp. 3797-3800.
45. Z. Sadeghipoor, Y. M. Lu, and S. Süsstrunk, "Correlation-based joint acquisition and demosaicing of visible and near-infrared images," in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2011, pp. 3165-3168.
46. D. Kiku, Y. Monno, M. Tanaka, and M. Okutomi, "Simultaneous capturing of RGB and additional band images using hybrid color filter array," Proc. SPIE, vol. 9023, pp. 90230V-1-90230V-9, Mar. 2014.
47. J. Y. Hardeberg, "Filter selection for multispectral color image acquisition," J. Imag. Sci. Technol., vol. 48, no. 2, pp. 105-110, 2004.
48. D. Connah, A. Alsam, and J. Y. Hardeberg, "Multispectral imaging: How many sensors do we need?" J. Imag. Sci. Technol., vol. 50, no. 1, pp. 45-52, 2006.
49. D. Alleysson, S. Süsstrunk, and J. Marguier, "Influence of spectral sensitivity functions on color demosaicing," in Proc. 11th Color Imag. Conf., 2003, pp. 351-357.
50. M. Parmar and S. J. Reeves, "Selection of optimal spectral sensitivity functions for color filter arrays," IEEE Trans. Image Process., vol. 19, no. 12, pp. 3190-3203, Dec. 2010.
51. Z. Sadeghipoor, Y. M. Lu, and S. Süsstrunk, "Optimum spectral sensitivity functions for single sensor color imaging," Proc. SPIE, vol. 8299, pp. 829904-1-829904-14, Jan. 2012.
52. Y. Monno, T. Kitao, M. Tanaka, and M. Okutomi, "Optimal spectral sensitivity functions for a single-camera one-shot multispectral imaging system," in Proc. 19th IEEE Int. Conf. Image Process. (ICIP), Sep./Oct. 2012, pp. 2137-2140.
53. B. K. Gunturk, Y. Altunbasak, and R. M. Mersereau, "Color plane interpolation using alternating projections," IEEE Trans. Image Process., vol. 11, no. 9, pp. 997-1013, Sep. 2002.
54. H. Takeda, S. Farsiu, and P. Milanfar, "Kernel regression for image processing and reconstruction," IEEE Trans. Image Process., vol. 16, no. 2, pp. 349-366, Feb. 2007.
55. G. Sharma, W. Wu, and E. N. Dalal, "The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations," Color Res. Appl., vol. 30, no. 1, pp. 21-30, 2005.
56. CAVE | Projects: Multispectral Image Database. [Online]. Available: http://www.cs.columbia.edu/CAVE/databases/multispectral/, accessed May 27, 2015.
57. VariSpec LC Tunable Filters | PerkinElmer. [Online]. Available: http://www.perkinelmer.com/catalog/product/id/varispc, accessed May 27, 2015.
58. CIE-International Commission on Illumination. [Online]. Available: http://www.cie.co.at/index.php/LEFTMENUE/DOWNLOADS, accessed May 27, 2015.
59. Useful Color Data. [Online]. Available: http://www.cis.rit.edu/research/mcsl2/online/cie.php, accessed May 27, 2015.
60. Color Research. [Online]. Available: http://cs.joensuu.fi/~spectral/databases/download/munsell-spec-matt.htm, accessed May 27, 2015.