Optical music recognition: state-of-the-art and open issues

International Journal of Multimedia Information Retrieval

Volumn 1, Issue 3, 2012, Pages 173-190

  Rebelo, Ana ^a   Fujinaga, Ichiro ^b   Paszkiewicz, Filipe ^a   Marcal, Andre R S ^c   Guedes, Carlos ^a   Cardoso, Jaime S ^a  

^a UNIVERSITY OF PORTO   (Portugal)

^b MCGILL UNIVERSITY   (Canada)

^c UNIVERSITY OF PORTO   (Portugal)

Author keywords

Computer music; Image processing; Machine learning; Music performance

Indexed keywords

EID: 84984956369     PISSN: 21926611     EISSN: 2192662X     Source Type: Journal    
DOI: 10.1007/s13735-012-0004-6     Document Type: Article

References (104)

1. Andronico A, Ciampa A (1995) On automatic pattern recognition and acquisition of printed music, 1982. In: Coüasnon B, Camillerapp J (eds) A way to separate knowledge from program in structured document analysis: application to optical music recognition. International conference on document analysis and recognition, pp 1092–1097
2. Bainbridge D (1997) An extensible optical music recognition system. In: Proceedings of the nineteenth Australasian computer science conference, pp 308–317
3. Bainbridge D, Bell T (2001) The challenge of optical music recognition. Comput Hum 35(2):95–121
4. Bainbridge D, Bell T (2003) A music notation construction engine for optical music recognition. Softw Pract Exp 33(2):173–200
5. Bellini P, Bruno I, Nesi P (2001) Optical music sheet segmentation. In: Proceedings of the first international conference on web delivering of music, pp 183–190
6. Bellini P, Bruno I, Nesi P (2007) Assessing optical music recognition tools. Comput Music J 31:68–93
7. Bellini P, Bruno I, Nesi P (2008) Optical music recognition: architecture and algorithms. In: Interactive multimedia music technologies. IGI Global, Hershey, pp 80–110
8. Bernsen J (2005) Dynamic thresholding of grey-level images, 1986. In: Bieniecki W, Grabowski S (eds) Multi-pass approach to adaptive thresholding based image segmentation. In: Proceedings of the 8th international IEEE conference CADSM
9. Blostein D, Baird HS (1992) A critical survey of music image analysis. In: Baird HS, Bunke H, Yamamoto K (eds) Structured document image analysis. Springer, Berlin, pp 405–434
10. Brink AD, Pendock NE (1996) Minimum cross-entropy threshold selection. Pattern Recognit 29(1):179–188
11. Burgoyne JA, Ouyang Y, Himmelman T, Devaney J, Pugin L, Fujinaga I (2009) Lyric extraction and recognition on digital images of early music sources. In: Proceedings of the 10th International Society for Music, information retrieval, pp 723–727
12. Burgoyne JA, Pugin L, Eustace G, Fujinaga I (2007) A comparative survey of image binarisation algorithms for optical recognition on degraded musical sources. In: Proceedings of the 8th International Society for Music, information retrieval, pp 509–512
13. Byrd D, Schindele M (2006) Prospects for improving OMR with multiple recognizers. In: Proceedings of the 7th International Society for Music, information retrieval, pp 41–47
14. Capela A, Cardoso JS, Rebelo A, Guedes C (2008) Integrated recognition system for music scores. In: Proceedings of the international computer music conference
15. Cardoso JS, Capela A, Rebelo A, Guedes C (2008) A connected path approach for staff detection on a music score. In: Proceedings of the 15th IEEE international conference on image processing, pp 1005–1008
16. Cardoso JS, Capela A, Rebelo A, Guedes C, Pinto da Costa JF (2009) Staff detection with stable paths. IEEE Trans Pattern Anal Mach Intell 31(6):1134–1139
17. Cardoso JS, Rebelo A (2010) Robust staffline thickness and distance estimation in binary and gray-level music scores. In: Proceedings of The twentieth international conference on pattern recognition, pp 1856–1859
18. Carter NP (1992) Automatic recognition of printed music in the context of electronic publishing, 1989. A critical survey of music image analysis. In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405–434
19. Chen Q, Sun Q, Heng P, Xia D (2008) A double-threshold image binarization method based on edge detector. Pattern Recognit 41(4):1254–1267
20. Choudhury G, Droetboom M, DiLauro T, Fujinaga I, Harrington B (2000) Optical music recognition system within a large-scale digitization project. In: Proceedings of the International Society for Music information retrieval
21. Coüasnon B (1996) Segmentation et reconnaissance de documents guidTes par la connaissance a priori: application aux partitions musicales. PhD thesis, Universit de Rennes
22. Coüasnon B, Brisset P, Stephan I (1995) Using logic programming languages for optical music recognition. In: Proceedings of the third international conference on the practical application of prolog, pp 115–134
23. Coüasnon B, Camillerapp J (1993) Using grammars to segment and recognize music scores. In: Proceedings of DAS-94: international association for pattern recognition workshop on document analysis systems, pp 15–27
24. Coüasnon B, Camillerapp J (1995) A way to separate knowledge from program in structured document analysis: application to optical music recognition. In: Proceedings of the third international conference on document analysis and recognition, pp 1092–1097
25. Dalitz C (2009) Reject options and confidence measures for knn classifiers. In: Dalitz C (ed) Schriftenreihe des Fachbereichs Elektrotechnik und Informatik Hochschule Niederrhein, vol 8. Shaker Verlag, Maastricht, pp 16–38
26. Dalitz C, Droettboom M, Czerwinski B, Fujigana I (2008) A comparative study of staff removal algorithms. IEEE Trans Pattern Anal Mach Intell 30:753–766
27. Damm D, Fremerey C, Kurth F, Müller M, Clausen M (2008) Multimodal presentation and browsing of music. In: Proceedings of the 10th international conference on multimodal interfaces. ACM, pp 205–208
28. Dan L (1996) Final year project report automatic optical music recognition, technical report
29. de Albuquerque MP, Esquef IA, Gesualdi Mello AR (2004) Image thresholding using tsallis entropy. Pattern Recognit Lett 25(9):1059–1065
30. Desaedeleer AF (2006) Reading sheet music. Master’s thesis, Imperial College London, Technology and Medicine, University of London
31. Droettboom M, Fujinaga I, MacMillan K (2002) Optical music interpretation. In: Proceedings of the joint IAPR international workshop on structural, syntactic, and statistical pattern recognition. Springer, Berlin, pp 378–386
32. Dutta A, Pal U, Fornés A, Lladós J (2010) An efficient staff removal approach from printed musical documents. In: Proceedings of the 20th international conference on pattern recognition. IEEE Computer Society, pp 1965–1968
33. Ferrand M, Leite JA, Cardoso A (1999) Hypothetical reasoning: an application to optical music recognition. In: Proceedings of the Appia-Gulp-Prode’99 joint conference on declarative programming, pp 367–381
34. Fornés A, Escalera S, Lladós J, Sánchez G, Radeva P, Pujol O (2007) Handwritten symbol recognition by a boosted blurred shape model with error correction. In: Proceedings of the 3rd Iberian conference on pattern recognition and image analysis, part I. Springer, Berlin, pp 13–21
35. Fornés A, Sánchez G (2005) Primitive segmentation in old handwritten music scores. In: Liu W, Llads J (eds) Graphics recognition. Ten years review and future perspectives. Lecture notes in computer science, vol 3926. Springer, Berlin, pp 279–290
36. Fornés A, Lladós J, Sánchez G, Bunke H (2008) Writer identification in old handwritten music scores. In: Proceedings of the 2008 the eighth IAPR international workshop on document analysis systems. IEEE Computer Society, pp 347–353
37. Fornés A, Lladós J, Sánchez G, Bunke H (2009) On the use of textural features for writer identification in old handwritten music scores. In: Proceedings of the 2009 10th international conference on document analysis and recognition. IEEE Computer Society, pp 996–1000
38. Fremerey C, Müller M, Kurth F, Clausen M (2008) Automatic mapping of scanned sheet music to audio recordings. In: Proceedings of the 9th International Society for Music, information retrieval, pp 413–418
39. Friel N, Molchanov I (1999) A new thresholding technique based on random sets. Pattern Recognit 32(9):1507–1517
40. Fujinaga I (1996) Exemplar-based learning in adaptive optical music recognition system. In: Proceedings of the international computer music conference, pp 55–60
41. Fujinaga I (2004) Staff detection and removal. In: George S (ed) Visual perception of music notation: on-line and off-line recognition. Idea Group Inc., Hershey, pp 1–39
42. Gatos B, Pratikakis I, Perantonis SJ (2004) An adaptive binarisation technique for low quality historical documents. In: Document analysis systems VI. Lecture notes in computer science, vol 3163. Springer, Berlin, pp 102–113
43. Genfang C, Wenjun Z, Qiuqiu W (2009) Pick-up the musical information from digital musical score based on mathematical morphology and music notation. In: Proceedings of the 2009 first international workshop on education technology and computer science. IEEE Computer Society, pp 1141–1144
44. George S (2004) Lyric recognition and christian music. In: George S (ed) Visual perception of music notation: on-line and off-line recognition. Idea Group Inc., Hershey, pp 198–225
45. Goecke R (2003) Building a system for writer identification on handwritten music scores. In: Proceedings of the IASTED international conference on signal processing, pattern recognition, and applications. Acta Press, Anaheim, pp 205–255
46. Grandvalet Y, Rakotomamonjy A, Keshet J, Canu S (2008) Support vector machines with a reject option. In: Koller D, Schuurmans D, Bengio Y, Bottou L (eds) Advances in neural information processing systems. MIT Press, Cambridge, pp 537–544
47. Homenda W (2005) Optical music recognition: the case study of pattern recognition. In: Kurzynski M, Puchala E, Wozniak M, zolnierek A (eds) Computer recognition systems. Advances in soft computing, vol 30. Springer, Heidelberg, pp 835–842
48. Homenda W, Luckner M (2006) Automatic knowledge acquisition: recognizing music notation with methods of centroids and classifications trees. In: Proceedings of the international joint conference on neural networks, pp 3382–3388
49. Huang LK, Wang MJJ (1995) Image thresholding by minimizing the measures of fuzziness. Pattern Recognit 28(1):41–51
50. Jain AK, Zongker D (1997) Representation and recognition of handwritten digits using deformable templates. IEEE Trans Pattern Anal Mach Intell 19(12):1386–1391
51. Jones G, Ong B, Bruno I, Ng K (2008) Optical music imaging: music document digitisation, recognition, evaluation, and restoration. In: Interactive multimedia music technologies. IGI Global, Hershey, pp 50–79
52. Kapur J, Sahoo P, Wong A (1985) A new method for gray-level picture thresholding using the entropy of the histogram. Comput Vis Graph Image Process 29(3):273–285
53. Kassler M (2009) Optical character recognition of printed music: a review of two dissertations, 1972. In: Vrist B (ed) Optical music recognition for structural information from high-quality scanned music, technical report
54. Khashman A, Sekeroglu B (2007) A novel thresholding method for text separation and document enhancement. In: Proceedings of the 11th panhellenic conference on informatics
55. Knopke I, Byrd D (2007) Towards musicdiff: a foundation for improved optical music recognition using multiple recognizers. In: Proceedings of the 8th International Society for Music, information retrieval, pp 123–124
56. Kurth F, Müller M, Fremerey C, Chang Y, Clausen M (2007) Automated synchronization of scanned sheet music with audio recordings. In: Proceedings of the 8th International Society for Music, information retrieval, pp 261–266
57. Leplumey I, Camillerapp J, Lorette G (1993) A robust detector for music staves. In: Proceedings of the international conference on document analysis and recognition, pp 902–905
58. Luth N (2002) Automatic identification of music notations. In: Proceedings of the first international symposium on cyber worlds. IEEE Computer Society, pp 203–210
59. MacMillan K, Droettboom M, Fujinaga I (2002) Gamera: optical music recognition in a new shell. In Proceedings of the international computer music conference, pp 482–485
60. Mahoney JV (1992) Automatic analysis of music score images, 1982. A critical survey of music image analysis. In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405–434
61. Miyao H, Haralick RM (2000) Format of ground truth data used in the evaluation of the results of an optical music recognition system. In: IAPR workshop on document analysis systems, pp 497–506
62. Miyao H, Nakano Y (1996) Note symbol extraction for printed piano scores using neural networks. IEICE Trans Inform Syst E79-D:548–554. ISSN: 0916–8532
63. Miyao H, Okamoto M (2004) Stave extraction for printed music scores using DP matching. J Adv Comput Intell Intell Inform 8:208–215
64. Ng K (2004a) Optical music analysis for printed music score and handwritten manuscript. In: George S (ed) Visual perception of music notation: on-line and off-line recognition. Idea Group Inc., Hershey, pp 1–39
65. Ng K (2004b) Optical music analysis for printed music score and handwritten music manuscript. In: George S (ed) Visual perception of music notation: on-line and off-line recognition. Idea Group Inc., Hershey, pp 108–127
66. Ng K, Boyle R (1996) Recognition and reconstruction of primitives in music scores. Image Vis Comput 14(1):39–46
67. Ng K, Boyle R, Cooper D (1995) Domain knowledge enhancement of optical music score recognition, technical report
68. Ng K, Cooper D, Stefani E, Boyle R, Bailey N (1999) Embracing the composer: optical recognition of handwritten manuscripts. In: Proceedings of the international computer music conference, pp 500–503
69. Niblack W (2003) An introduction to digital image processing, 1986. Comparison of some thresholding algorithms for text/background segmentation in difficult document images. In: Leedham G, Yan C, Takru K, Tan J, Mian L (eds) Proceedings of the seventh international conference on document analysis and recognition, pp 859–864
70. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66
71. Pal N, Pal S (2004) Entropic thresholding, 1989. In: Sezgin M, Sankur B (eds) Survey over image thresholding techniques and quantitative performance evaluation. J Electron Imaging 13(1):146–165
72. Pinto T, Rebelo A, Giraldi G, Cardoso JS (2011) Music score binarization based on domain knowledge. In: Pattern recognition and image analysis. Lecture notes in computer science, vol 6669. Springer, Heidelberg, pp 700–708
73. Prerau D (1992) Computer pattern recognition of standard engraved music notation, 1970. A critical survey of music image analysis. In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405–434
74. Prerau D (1992) Optical music recognition using projections, 1988. A critical survey of music image analysis. In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405–434
75. Pruslin D (1992) Automatic recognition of sheet music, 1966. A critical survey of music image analysis. In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405–434
76. Pugin L (2006) Optical music recognition of early typographic prints using Hidden Markov models. In: Proceedings of the International Society for Music, information retrieval, pp 53–56
77. Pugin L, Burgoyne J, Fujinaga I (2007a) Goal-directed evaluation for the improvement of optical music recognition on early music prints. In: Proceedings of the 7th ACM/IEEE-CS joint conference on digital libraries. ACM, pp 303–304
78. Pugin L, Burgoyne JA, Fujinaga I (2007b) MAP adaptation to improve optical music recognition of early music documents using Hidden Markov models. In: Proceedings of the 8th International Society for Music, information retrieval, pp 513–516
79. Randriamahefa R, Cocquerez JP, Fluhr C, Pepin F, Philipp S (1993) Printed music recognition. In: Proceedings of the second international conference on document analysis and recognition, pp 898–901
80. Read G (1969) Music notation: a manual of modern practice, 2 edn. Taplinger, New York. ISBN: 0-8008-5459-4
81. Rebelo A (2008) New methodologies torwads an automatic optical recognition of handwritten musical scores. Master’s thesis, School of Sciences, University of Porto
82. Rebelo A, Capela A, Pinto da Costa JF, Guedes C, Carrapatoso E, Cardoso JS (2007) A shortest path approach for staff line detection. In: Proceedings of the third international conference on automated production of cross media content for multi-channel, distribution, pp 79–85
83. Rebelo A, Capela G, Cardoso JS (2010) Optical recognition of music symbols: a comparative study. Int J Document Anal Recognit 13:19–31
84. Rebelo A, Paszkiewicz F, Guedes C, Marcal A, Cardoso JS (2011) A method for music symbols extraction based on musical rules. In: Bridges: mathematical connections in art, music, and science, pp 81–88
85. Reed KT, Parker JR (1996) Automatic computer recognition of printed music. In: Proceedings of the 13th international conference on pattern recognition, vol 3, pp 803–807
86. Ridler T, Calvard S (1995) Picture thresholding using an iterative selection method, 1978. In: Venkateswarlu N (ed) Implementation of some image thresholding algorithms on a connection machine-200. Pattern Recognit Lett 16(7):759–768
87. Riley J, Fujinaga I (2003) Recommended best practices for digital image capture of musical scores. OCLC Syst Serv 19(2):62–69
88. Roach JW, Tatem JE (1992) Using domain knowledge in low-level visual processing to interpret handwritten music: an experiment, 1988. A critical survey of music image analysis. In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405–434
89. Rossant F, Bloch I (2007) Robust and adaptive OMR system including fuzzy modeling, fusion of musical rules, and possible error detection. EURASIP J Appl Signal Process 2007(1):160
90. Sahoo P, Wilkins C, Yeager J (1997) Threshold selection using renyi’s entropy. Pattern Recognit 30(1):71–84
91. Sezan M (1985) A peak detection algorithm and its application to histogram-based image data reduction. Graph Models Image Process 29:47–59
92. Sezgin M, Sankur B (2004) Survey over image thresholding techniques and quantitative performance evaluation. J Electron Imaging 13(1):146–165
93. Sheridan S, George S (2004) Defacing music score for improved recognition. In: Abraham G, Rubinstein BIP (eds) Proceedings of the second Australian undergraduate students’ computing conference. Australian undergraduate students’ computing conference, pp 1–7
94. Sousa R, Mora B, Cardoso JS (2009) An ordinal data method for the classification with reject option. In: Proceedings of the eighth international conference on machine learning and applications, pp 746–750
95. Szwoch M (2005) A robust detector for distorted music staves. In: Computer analysis of images and patterns. Lecture notes in computer science. Springer, Berlin, pp 701–708
96. Szwoch M (2007) Guido: a musical score recognition system. In: Proceedings of the ninth international conference on document analysis and recognition, vol 2. IEEE Computer Society, pp 809–813
97. Szwoch M (2008) Using musicxml to evaluate accuracy of omr systems. In: Stapleton G, Howse J, Lee J (eds) Diagrammatic representation and inference. Lecture notes in computer science, vol 5223. Springer, Berlin, pp 419–422
98. Tardón LJ, Sammartino S, Barbancho I, Gómez V, Oliver A (2009) Optical music recognition for scores written in white mensural notation. EURASIP J Image Video Process. Article ID: 843401. ISSN: 1687–5176
99. Taubman G (2005) Musichand: a handwritten music recognition system, technical report
100. Toyama F, Shoji K, Miyamichi J (2006) Symbol recognition of printed piano scores with touching symbols. In: Proceedings of the international conference on pattern recognition. IEEE Computer Society, pp 480–483
101. Trier O, Jain A (1995) Goal-directed evaluation of binarization methods. IEEE Trans Pattern Anal Mach Intell 17(12):1191–1201
102. Trier O, Taxt T (1995) Evaluation of binarization methods for document images. IEEE Trans Pattern Anal Mach Intell 17(3):312–315
103. Tsai D-M (1995) A fast thresholding selection procedure for multimodal and unimodal histograms. Pattern Recognit Lett 16(6):653–666
104. Yanowitz S, Bruckstein A (1989) A new method for image segmentation. Comput Vis Graph Image Process 46(1):82–95