Multimodal biomedical image retrieval using hierarchical classification and modality fusion

International Journal of Multimedia Information Retrieval

Volumn 2, Issue 3, 2013, Pages 159-173

  Rahman, Md Mahmudur ^a   You, Daekeun ^a   Simpson, Matthew S ^a   Antani, Sameer K ^a   Demner Fushman, Dina ^a   Thoma, George R ^a  

^a NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

Biomedical imaging; Classification; Content based image retrieval; Data fusion; Multi modal retrieval; Text retrieval

Indexed keywords

EID: 85014919777     PISSN: 21926611     EISSN: 2192662X     Source Type: Journal    
DOI: 10.1007/s13735-013-0038-4     Document Type: Article

References (35)

1. Demner-Fushman D, Antani SK, Simpson M, Thoma GR (2009) Annotation and retrieval of clinically relevant images. Int J Med Inf 78(12):e59–e67
2. Demner-Fushman D, Antani SK, Simpson M, Thoma GR (2012) Design and development of a multimodal biomedical information retrieval system. JCSE 6(2):168–177
3. Antin-Ozerkis D, Rubinowitz A (2008) Recognizing lung disease in patients with rheumatoid arthritis. Part 2: pleuropulmonary disease may be more common than you thought. J Respir Dis 29(8):318–324
4. Datta R, Joshi D, Li J, Wang JZ (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv 40(2): 1–60
5. Chen N (2006) A survey of indexing and retrieval of multimodal documents: text and images. Technical Report, 2006–505, Queen’s University
6. Ingwersen P (1996) Cognitive perspectives of information retrieval interaction: elements of a cognitive IR theory. J Doc 52(1): 3–50
7. Hersh WR et al (2004) TREC 2004 genomics track overview. The Thirteenth Text Retrieval Conference: TREC 2004. National Institute of Standards and Technology, Gaithersburg
8. Hearst MA, Divoli A, Buturu H, Ksikes A, Nakov P, Wooldridge MA et al (2007) Biotext search engine: beyond abstract search. Bioinformatics 23(16):2196–2197
9. Xu S, McCusker J, Krauthammer M (2008) Yale Image Finder (YIF): a new search engine for retrieving biomedical images. Bioinformatics 24(17):1968–1970
10. Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Reisetter Jr J, Khan CEKJ, Hersh WR (2010) Overview of the CLEF 2010 Medical Image Retrieval Track. In: CLEF 2012 Evaluation Labs and Workshop, Online Working Notes, Padua, Italy, 20–23 Sept 2010
11. Kalpathy-Cramer J, Müller H, Bedrick S, Eggel I, Garcia Seco de Herrera A, Tsikrika T (2011) Overview of the CLEF 2011 medical image classification and retrieval tasks. In: CLEF 2011 Working Notes, Amsterdam, The Netherlands
12. Müller H, Herrera A, Kalpathy-Cramer J, Demner-Fushman D, Antani S, Ivan E (2012) Overview of the ImageCLEF 2012 Medical Image Retrieval and Classification Tasks. In: The Working Notes for the CLEF 2012 Labs and Workshop, Rome, Italy, 17–20 Sept 2012
13. Rahman MM, Antani SK, Thoma GR (2011) A learning-based similarity fusion and filtering approach for biomedical image retrieval using SVM classification and relevance feedback. IEEE Trans Inf Technol Biomed 15(4):640–646
14. Lehmann TM, Güld MO, Deselaers T, Keysers D, Schubert H, Spitzer K, Ney H, Wein BB (2005) Automatic categorization of medical images for content-based retrieval and data mining. Comput Med Imaging Graph 29:143–155
15. Fox EA, Shaw JA (1994) Combination of multiple searches. In: Proceedings of the 2nd Text Retrieval Conference (TREC-2), NIST Special Publication, pp 243–252
16. Lee JH (1995) Combining multiple evidence from different properties of weighting schemes. In: Proceedings of the 18th Annual ACM-SIGIR, pp 180–188
17. Simpson M, Rahman MM, Phadmis S, Apostolova E, Demner-Fushman D, Antani SK, Thoma GR (2011) Text- and content-based approaches to image modality classification and retrieval for the ImageCLEF 2011 medical retrieval track. In: The Working Notes for the CLEF 2011 Labs and Workshop, Amsterdam, 19–22 Sept 2011
18. Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS (1999) ASSERT: a physician-in-the-loop content-based image retrieval system for HRCT image databases. Comput Vis Image Underst 75:111–132
19. Hsu W, Antani S, Long LR, Neve L, Thoma GR (2008) SPIRS: a web-based image retrieval system for large biomedical databases. Int J Med Inf 78:13–24
20. Ide NC, Loane RF, Demner-Fushman D (2007) Essie: a concept-based search engine for structured biomedical text. J Am Med Inf Assoc 1(3):253–263
21. Baeza-Yates R, Ribeiro-Neto B (June 1999) Modern information retrieval. Addison-Wesley, Reading
22. Chang SF, Sikora T, Puri A (2001) Overview of the MPEG-7 standard. IEEE Trans Circ Syst Video Technol 11:688–695
23. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110
24. Manjunath BS, Ma WY (1996) Texture features for browsing and retrieval of large image data. IEEE Trans Pattern Anal Mach Intell (Special Issue on Digital Libraries) 18(8):837–842
25. Menp T (2003) The local binary pattern approach to texture analysis extensions and applications. PhD thesis, University of Oulu
26. Grubinger M, Clough P, Hanbury A, Müller H (2008) Lire: lucene image retrieval: an extensible java CBIR library. In: Proceedings of the 16th ACM international conference on Multimedia, Vancouver, British Columbia, Canada, pp 1085–1088
27. Müller H, Kalpathy-Cramer J, Demner-Fushman D, Antani S (2012) Creating a classification of image types in the medical literature for visual categorization. In: Advanced PACS-based Imaging Informatics and Therapeutic Applications, Proceedings of the SPIE, vol 8319, pp 83190P–83190P-12
28. Apostolova E, You D, Xue Z, Antani S, Demner-Fushman D, Thoma GR (2013) Image retrieval from scientific publications: text and image content processing to separate multi-panel figures. J Am Soc Inf Sci Technol. Published online in Wiley Online Library (wileyonlinelibrary.com). doi:10.1002/asi.2281
29. You D, Rahman MM, Antani SK, Demner-Fushman D, Thoma GR (2013) Text- and content-based biomedical image modality classification. In: SPIE Proceedings vol 8674 Medical Imaging, Advanced PACS-based Imaging Informatics and Therapeutic Applications. Maria Y. Law; William W, Boonn, Editors 86740L
30. Vapnik V (1998) Statistical learning theory. Wiley, New York
31. Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The WEKA data mining software: an update. SIGKDD Explor 11(1):10–18
32. Jung K, Kim KI, Jain AK (2004) Text information extraction in images and video: a survey. Pattern Recognit 37(5):977–997
33. Tsikrika T, Müller H, Kahn Jr CE (2012) Log analysis to understand medical professionals’ image searching behaviour. In: Proceedings of the 24th European Medical Informatics Conference (MIE2012)
34. Simpson M, You D, Rahman MdMM, Demner-Fushman D, Antani Sk, Thoma GR (2012) ITI’s participation in the ImageCLEF 2012 medical retrieval and classification tasks. In: The Working Notes for the CLEF 2012 Labs and Workshop, Amsterdam, Rome, Italy, 17–20 Sept 2012
35. Smucker MD, Allan J, Carterette B (2007) A comparison of statistical significance tests for information retrieval evaluation. In: Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, pp 623–632