Learning semantic and visual similarity for endomicroscopy video retrieval

IEEE Transactions on Medical Imaging

Volumn 31, Issue 6, 2012, Pages 1276-1288

  Andre, Barbara ^a   Vercauteren, Tom ^a   Buchner, Anna M ^b   Wallace, Michael B ^c   Ayache, Nicholas ^d  

^a Mauna Kea Technologies   (France)

^b HOSPITAL OF THE UNIVERSITY OF PENNSYLVANIA   (United States)

^c MAYO CLINIC   (United States)

^d INRIA   (France)

Author keywords

Bag of visual words (BoW); content based image retrieval (CBIR); endomicroscopy; semantic and visual similarity; semantic gap; similarity learning

Indexed keywords

BAG-OF-VISUAL-WORDS; CONTENT-BASED IMAGE RETRIEVAL; ENDOMICROSCOPY; SEMANTIC GAP; SIMILARITY LEARNING; VISUAL SIMILARITY;

COMPUTER VISION; CONTENT BASED RETRIEVAL; DIAGNOSIS; DISTANCE EDUCATION; SEMANTICS; SUPPORT VECTOR MACHINES;

VISUAL LANGUAGES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; CAPSULE ENDOSCOPY; COLON TUMOR; COMPUTER ASSISTED DIAGNOSIS; HOSPITAL INFORMATION SYSTEM; HUMAN; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION RETRIEVAL; METHODOLOGY; PATHOLOGY; REPRODUCIBILITY; SEMANTICS; SENSITIVITY AND SPECIFICITY; VIDEO MICROSCOPY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CAPSULE ENDOSCOPY; COLONIC NEOPLASMS; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; MICROSCOPY, VIDEO; PATTERN RECOGNITION, AUTOMATED; RADIOLOGY INFORMATION SYSTEMS; REPRODUCIBILITY OF RESULTS; SEMANTICS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE;

EID: 84861873452     PISSN: 02780062     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMI.2012.2188301     Document Type: Article

References (33)

1. H. Müller, J. Kalpathy-Cramer, C. E. Kahn, W. Hatt, S. Bedrick, and W. R. Hersh, "Overview of the ImageCLEFmed 2008 medical image retrieval task," in CLEF, 2008, pp. 512-522, 1.
2. U. Avni, H. Greenspan, E. Konen, M. Sharon, and J. Goldberger, "X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words," IEEE Trans. Med. Imag., vol. 30, no. 3, pp. 733-746, 2011, 1.
3. A. W. M. Smeulders, M. Worring, S. Santini, A. Gupta, and R. Jain, "Content-based image retrieval at the end of the early years," IEEE Trans. Pattern Anal.Mach. Intell., vol. 22, no. 12, pp. 1349-1380, Dec. 2000. (Pubitemid 32088465)
4. C. B.Akgül, D. L. Rubin, S. Napel, C. F. Beaulieu,H.Greenspan, and B. Acar, "Content-based image retrieval in radiology: Current status and future directions," J. Digital Imag., vol. 24, no. 2, pp. 208-222, 2011
5. O. Khalid, S. Radaideh, O. W. Cummings,M. J. O'Brien, J. R. Goldblum, and D. K. Rex, "Reinterpretation of histology of proximal colon polyps called hyperplastic in 2001," World J. Gastroenterol., vol. 15, no. 30, pp. 3767-3770, 2009, 2.
6. B. André, T. Vercauteren, M. B. Wallace, A. M. Buchner, and N. Ayache, "Endomicroscopic video retrieval using mosaicing and visual words," in Proc. Int. Symp. Biomed. Imag. (ISBI'10), 2010, pp. 1419-1422.
7. B. André, T. Vercauteren, A. M. Buchner,M. B.Wallace, and N. Ayache, "Retrieval evaluation and distance learning from perceived similarity between endomicroscopy videos," in Proc. MICCAI'11, 2011, pp. 289-296
8. B. André, T. Vercauteren, A. M. Buchner, M. B. Wallace, and N. Ayache, Learning semantic and visual similarity for endomicroscopy video retrieval INRIA, INRIA Tech. Rep. RR-7722, Aug. 2011 [Online]. Available: http://hal.inria.fr/inria-00618057/en/
9. A. M. Buchner, M. W. Shahid, M. G. Heckman, M. Krishna, M. Ghabril, M. Hasan, J. E. Crook, V. Gomez, M. Raimondo, T. Woodward, H. Wolfsen, and M. B. Wallace, "Comparison of probe based confocal laser endomicroscopy with virtual chromoendoscopy for classification of colon polyps," Gastroenterology, vol. 138, no. 3, pp. 834-842, 2009.
10. T. Vercauteren, A. Perchant, G.Malandain, X. Pennec, and N. Ayache, "Robust mosaicing with correction of motion distortions and tissue deformation for in vivo fibered microscopy," Med. Image Anal., vol. 10, no. 5, pp. 673-692, Oct. 2006, 3. (Pubitemid 44283801)
11. E. Dabizzi, M. W. Shahid, B. Qumseya, M. Othman, and M. B. Wallace, "Comparison between video and mosaics viewing mode of confocal laser endomicroscopy (pCLE) in patients with Barrett's esophagus," Gastroenterology (DDW 2011), 2011, 3.
12. Visual Similarity Scoring (VSS) [Online]. Available: http://smartatlas. maunakeatech.com
13. R. Kiesslich, J. Burg, M. Vieth, J. Gnaendiger, M. Enders, P. Delaney, A. Polglase, W. McLaren, D. Janell, S. Thomas, B. Nafe, P. R. Galle, and M. F.Neurath, "Confocal laser endoscopy for diagnosing intraepithelial neoplasias and colorectal cancer in vivo," Gastroenterology, vol. 127, no. 3, pp. 706-713, 2004. (Pubitemid 39208403)
14. J. Zhang, S. Lazebnik, and C. Schmid, "Local features and kernels for classification of texture and object categories:Acomprehensive study," Int. J. Comput. Vis., vol. 73, pp. 213-238, Jun. 2007. (Pubitemid 46181625)
15. T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three-dimensional textons," Int. J. Comput. Vis., vol. 43, pp. 29-44, June 2001. (Pubitemid 32720091)
16. F.-F. Li and P. Perona, "A Bayesian hierarchical model for learning natural scene categories," in Proc. CVPR'05, 2005, pp. 524-531, 4.
17. R. M. Haralick, "Statistical and structural approaches to texture," Proc. IEEE, vol. 67, pp. 786-804, 1979, 4.
18. GLCM texture features [Online]. Available: http://www.mathworks. com/matlabcentral/fileexchange/22187-glcm-texture-features
19. A. Oliva and A. Torralba, "Modeling the shape of the scene: A holistic representation of the spatial envelope," Int. J. Comput. Vis., vol. 42, no. 3, pp. 145-175, 2001, 4. (Pubitemid 32680801)
20. B. Poblete, B. Bustos, M. Mendoza, and J. M. Barrios, "Visual-semantic graphs: Using queries to reduce the semantic gap in web image retrieval," in Proc. ACM Inf. Knowledge Manage., 2010, pp. 1553-1556, 5.
21. H. Ma, J. Zhu, M. R. Lyu, and I. King, "Bridging the semantic gap between image contents and tags," IEEE Trans. Multimedia, vol. 12, no. 5, pp. 462-473, Aug. 2010.
22. J. C. Caicedo, J. G. Moreno, E. A. Niño, and F. A. González, "Combining visual features and text data for medical image retrieval using latent semantic kernels," in Proc. Multimedia Inf. Retrieval, 2010, pp. 359-366.
23. N. Rasiwasia, P. J. Moreno, and N. Vasconcelos, "Bridging the gap: Query by semantic example," IEEE Trans. Multimedia, vol. 9, no. 5, pp. 923-938, Aug. 2007. (Pubitemid 47171250)
24. N. Rasiwasia, J. C. Pereira, E. Coviello,G.Doyle, G. R. G. Lanckriet, R. Levy, and N. Vasconcelos, "A new approach to cross-modal multimedia retrieval," in ACM Multimedia, 2010, pp. 251-260, 5.
25. R. Kwitt, N. Rasiwasia, N. Vasconcelos, A. Uhl, M. Häfner, and F. Wrba, "Learning pit pattern concepts for gastroenterological training," in Proc. MICCAI'11, 2011, pp. 273-280.
26. L. Yang, R. Jin, L. Mummert, R. Sukthankar, A. Goode, B. Zheng, S. C. H. Hoi, and M. Satyanarayanan, "A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 1, pp. 30-44, Jan. 2010.
27. K. Q. Weinberger and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification," J. Mach. Learn. Res., vol. 10, pp. 207-244, 2009.
28. J. Philbin, M. Isard, J. Sivic, and A. Zisserman, "Descriptor learning for efficient retrieval," in Proc. ECCV'10, 2010, pp. 677-691.
29. T. Kailath, "The divergence and Bhattacharyya distance measures in signal selection," IEEE Trans. Commun. Technol., vol. 15, no. 1, pp. 52-60, Feb. 1967.
30. V. Barnett, Sample Survey Principles and Methods. London, U.K.: Hodder Arnold, 1991.
31. X.-L. Meng, R. Rosenthal, and D. B. Rubin, "Comparing correlated correlation coefficients," Psychol. Bull., vol. 111, no. 1, pp. 172-175, 1992.
32. E. R. DeLong, D. M. DeLong, and D. L. Clarke-Pearson, "Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach," Biometrics, vol. 44, no. 3, pp. 837-845, 1988.
33. B. André, T. Vercauteren, A. M. Buchner, M. W. Shahid, M. B. Wallace, and N. Ayache, "An image retrieval approach to setup difficulty levels in training systems for endomicroscopy diagnosis," in Proc. MICCAI'10, 2010, no. 6362, pp. 480-487.