Multiple hypotheses image segmentation and classification with application to dietary assessment

IEEE Journal of Biomedical and Health Informatics

Volumn 19, Issue 1, 2015, Pages 377-388

  Zhu, Fengqing ^a   Bosch, Marc ^b   Khanna, Nitin ^c   Boushey, Carol J ^d   Delp, Edward J ^e  

^a HUAWEI TECHNOLOGIES   (United States)

^b QUALCOMM INC   (United States)

^c GRAPHIC ERA UNIVERSITY   (India)

^d UNIVERSITY OF HAWAII CANCER CENTER   (United States)

^e PURDUE UNIVERSITY   (United States)

Author keywords

Dietary assessment; image analysis; image features; image segmentation; object recognition

Indexed keywords

IMAGE ANALYSIS; IMAGE CLASSIFICATION; IMAGE ENHANCEMENT; OBJECT RECOGNITION; VECTOR SPACES;

DIETARY ASSESSMENTS; FEATURE CLASSIFICATION; IMAGE FEATURES; INDIVIDUAL FEATURES; MULTIPLE FEATURES; MULTIPLE HYPOTHESIS; MULTIPLE SEGMENTATION; SEGMENTED REGIONS;

IMAGE SEGMENTATION;

ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; CLASSIFICATION; COMPUTER ASSISTED DIAGNOSIS; FOOD; FOOD ANALYSIS; MEDICAL RECORD; MOBILE APPLICATION; PHOTOGRAPHY; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY;

ARTIFICIAL INTELLIGENCE; DIET RECORDS; FOOD; FOOD ANALYSIS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MOBILE APPLICATIONS; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 84920885244     PISSN: 21682194     EISSN: 21682208     Source Type: Journal    
DOI: 10.1109/JBHI.2014.2304925     Document Type: Article

References (54)

1. C. Boushey, D. Kerr, J. Wright, K. Lutes, D. Ebert, and E. Delp, "Use of technology in children's dietary assessment," Eur. J. Clin. Nutrit., vol. 63, pp. S50-S57, 2009.
2. F. Zhu, M. Bosch, I. Woo, S. Kim, C. Boushey, D. Ebert, and E. Delp, "The use of mobile devices in aiding dietary assessment and evaluation," IEEE J. Sel. Topics Signal Process., vol. 4, no. 4, pp. 756-766, Aug. 2010.
3. J. Carreira and C. Sminchisescu, "Constrained parametric min-cuts for automatic object segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3241-3248.
4. D. Hoiem, A. Efros, and M. Hebert, "Geometric context from a single image," in Proc. 10th IEEE Int. Conf. Comput. Vis., Oct. 2005, vol. 1, pp. 654-661.
5. J. Sivic, B. Russell, A. Efros, A. Zisserman, and W. Freeman, "Discovering objects and their location in images," in Proc. 10th IEEE Int. Conf. Comput. Vis., Oct. 2005, vol. 1, pp. 370-377.
6. A. Biem and S. Katagiri, "Feature extraction based on minimum classification error/generalized probabilistic descent method," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Minneapolis, MN, USA, Apr. 1993, pp. 275-278.
7. M. Bosch, "Visual feature modeling and refinement with application in dietary assessment," Ph.D. dissertation, Purdue Univ.,West Lafayette, IN, USA, May 2012.
8. M. Puri, Z. Zhu, Q. Yu, A. Divakaran, and H. Sawhney, "Recognition and volume estimation of food intake using a mobile device," in Proc. IEEE Workshop Appl. Comput. Vis., Dec. 2009, pp. 1-8.
9. S.Yang, M. Chen,D. Pomerleau, and R. Sukhankar, "Food recognition using statistics of pairwise local features," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., San Francisco, CA, USA, Jun. 2010, pp. 2249-2256.
10. T. Joutou and K. Yanai, "A food image recognition system with multiple kernel learning," in Proc. Int. Conf. Image Process., Cairo, Egypt, Nov. 2009, pp. 285-288.
11. K. Kitamura, T. Yamasaki, and K. Aizawa, "Foodlog: Capture, analysis and retrieval of personal food images via web," in Proc. ACM Multimedia Workshop Multimedia Cook. Eat. Activit., Beijing, China, Nov. 2009, pp. 23-30.
12. M. Chen, Y. Yang, C. Ho, S. Wang, S. Liu, E. Chang, C. Yeh, and M. Ouhyoung, "Automatic chinese food identification and quantity estimation," in Proc. Spec. Interest Group Graph. Interact. Techn. Asia, Singapore, Dec. 2012, pp. 1-4.
13. F. Zhu, M. Bosch, T. Schap, N. Khanna, D. Ebert, C. Boushey, and E. Delp, "Segmentation assisted food classification for dietary assessment," in Proc. IS&T/SPIE Conf. Computat. Imag. IX, Jan. 2011, vol. 7873.
14. F. Zhu, M. Bosch, and E. Delp, "An image analysis system for dietary assessment and evaluation," in Proc. Int. Conf. Image Process., Sep. 2010, Hong Kong, pp. 1853-1856.
15. F. Zhu, A. Mariappan, D. Kerr, C. Boushey, K. Lutes, D. Ebert, and E. Delp, "Technology-assisted dietary assessment," in Proc. IS&T/SPIE Conf. Computat. Imag. VI, San Jose, CA, USA, Jan. 2008, vol. 6814.
16. F. Zhu, "Multilevel image segmentation with application in dietary assessment and evaluation," Ph.D. dissertation, Purdue University, West Lafayette, IN, USA, Dec. 2011.
17. F. Zhu, M. Bosch, N. Khanna, C. Boushey, and E. Delp, "Multilevel segmentation for food classification in dietary assessment," in Proc. 7th Int. Symp. Image Signal Process. Anal., Dubrovnik, Croatia, Sep. 2011, pp. 337-342.
18. M. Bosch, F. Zhu, N. Khanna, C. Boushey, and E. Delp, "Combining global and local features for food identification and dietary assessment," in Proc. Int. Conf. Image Process., Brussels, Belgium, 2011.
19. B. Six, T. Schap, F. Zhu, A. Mariappan, M. Bosch, E. Delp, D. Ebert, D. Kerr, and C. Boushey, "Evidence-based development of a mobile telephone food record," J. Amer. Dietetic Assoc., pp. 74-79, Jan. 2010.
20. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
21. J. Canny, "A computational approach to edge detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679-698, Nov. 1986.
22. P. Kovesi, (2007). [Online]. Available: http://www.csse.uwa.edu.au/ pk/ research/matlabfns/
23. C. Xu, F.Zhu, N. Khanna,C.Boushey, andE. J.Delp, "Image enhancement and quality measures for dietary assessment using mobile devices," in Proc. IS&T/SPIE Conf. Computat. Imag. X, San Francisco, CA, USA, Jan. 2012, vol. 8296, pp. 1-10.
24. E. Sharon, A. Brandt, and R. Basri, "Fastmultiscale image segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2000, vol. 1, pp. 70-77.
25. S.Yu, "Segmentation using multiscale cues," in Proc. IEEE Conf.Comput. Vis. Pattern Recognit., 27Jun.-2 Jul. 2004, vol. 1, pp. 247-254.
26. T. Cour, F. Benezit, and J. Shi, "Spectral segmentation with multiscale graph decomposition," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2005, vol. 2, pp. 1124-1131.
27. C. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 379-423, 623-656, Jul.-Oct. 1948.
28. B. Manjunath, P. Salembier, and T. Sikora, Introduction to MPEG-7: Multimedia Content Description Interface. New York, NY, USA: Wiley, 2002.
29. M. Bosch, F. Zhu, N. Khanna, C. Boushey, and E. Delp, "Food texture descriptors based on fractal and local gradient information," in Proc. 19th Eur. Signal Process. Conf., Barcelona, Spain, Sep. 2011, pp. 764-768.
30. R. Haralick, K. Shanmugam, and I. Dinstein, "Texture features for image classification," IEEE Trans. Syst., Man, Cybern., vol. SMC-3, no. 6, pp. 610-621, Nov. 1973.
31. K. Falconer, Fractal Geometry: Mathematical Foundations and Applications. London, U.K.: Wiley, 1990.
32. Y. Xu, H. Ling, and C. Fermuller, "A projective invariant for textures," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., New York, NY, USA, 2006, vol. 2, pp. 1932-1939.
33. Y. Xu, H. Ji, and C. Fermuller, "Viewpoint invariant texture description using fractal analysis," Int. J. Comput. Vis., vol. 83, no. 1, pp. 85-100, 2009.
34. B. Manjunath and W. Y. Ma, "Texture features for browsing and retrieval of image data," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 8, pp. 837-842, Aug. 1996.
35. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 2, no. 60, pp. 91-110, 2004.
36. H. Bay, A. Ess, L. Tuytelaars, and T. Van Gool, "Surf: Speeded up robust features," in Proc. Int. Conf. Comput. Vis. Image Understand., 2008, vol. 110, no. 3, pp. 346-359.
37. W. Freeman and E. H. Adelson, "The design and use of steerable filters," IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 9, pp. 891-906, Sep. 1991.
38. S. Prasad and L. Bruce, "Decision fusion with confidence-based weight assignment for hyperspectral target recognition," IEEE Trans. Geosci. Remote Sens., vol. 46, no. 5, pp. 1448-1456, May 2008.
39. B. Topcu and H. Erdogan, "Decision fusion for patch-based face recognition," in Proc. 20th Int. Conf. Pattern Recognit., Washington, DC, USA, 2010, pp. 1348-1351.
40. L. Xu, A. Krzyzak, and C. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Trans. Syst., Man Cybern., vol. 22, no. 3, pp. 418-435, May/Jun. 1992.
41. R. Duda and P. Hart, Pattern Classification and Scene Analysis. New York, NY, USA: Wiley, 1973.
42. V. Vapnik, The Nature of Statistical Learning Theory. New York, NY, USA: Springer-Verlag, 1995.
43. S. Canu, Y. Grandvalet, V. Guigue, and A. Rakotomamonjy, "SVM and kernel methods MATLAB toolbox," 2005.
44. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramidmatching for recognizing natural scene categories," in Proc. 2006 Int. Conf. Comput. Vis. Pattern Recognit., New York, NY, USA, 2004, pp. 2169-2176.
45. G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray, "Visual categorization with bags of keypoints," presented at the 2004 Int. Worksh. Statist. Learn. Comput. Vis., Prague, Czech Republic, 2004.
46. D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in Computer Vision and Pattern Recognition, 2006, pp. 2161-2168.
47. M. Khabou, L. Hermi, and M. Rhouma, "Shape recognition using eigenvalues of theDirichlet Laplacian," Pattern Recogn., vol. 40, no. 1, pp. 141-153, Jan. 2007.
48. S. Divvala, A. Efros, and M. Hebert, "Can similar scenes help surface layout estimation?" CMU Robotics Institute, Pittsburgh, PA, Tech. Rep., 2008, paper 273.
49. F. Estrada and A. Jepson, "Benchmarking image segmentation algorithms," Int. J. Comput. Vis., vol. 85, no. 2, pp. 167-181, 2009.
50. D. Martin and C. Fowlkers, "The berkeley segmentation database and benchmark." [Online]. Available: http://www.cs.berkeley.edu/projects/ vision/grouping/segbench/
51. F. Perronnin, J. Sanchez, and T.Mensink, "Improving the fisher kernel for large-scale image classification," in Proc. Eur. Conf. Comput. Vis., Crete, Greece, Sep. 2010.
52. A. Vedaldi and B. Fulkerson. (2008). "VLFeat: An open and portable library of computer vision algorithms," [Online]. Available: http://www.vlfeat.org/
53. S. Prasad, L. Bruce, and J. Ball, "A multi-classifier and decision fusion framework for robust classification of mammographic masses," in Proc. IEEE Eng. Med. Biol. Conf., Vancouver, Canada, Aug. 2008, pp. 3048-3051.
54. K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman, "The devil is in the details: An evaluation of recent feature encoding methods," presented at the Brit. Mach. Vis. Conf., Dundee, U.K., Aug. 2011.