Automatic chord estimation from audio: A review of the state of the art

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 2, 2014, Pages 556-575

  McVicar, Matt ^a   Santos Rodríguez, Raúl ^a   Ni, Yizhao ^a   De Bie, Tijl ^a  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

Author keywords

Expert systems; Knowledge based systems; Machine learning; Music information retrieval; Supervised learning

Indexed keywords

DATA PROCESSING; EXPERT SYSTEMS; FEATURE EXTRACTION; INFORMATION RETRIEVAL; KNOWLEDGE BASED SYSTEMS; LEARNING SYSTEMS; SUPERVISED LEARNING; ARTIFICIAL INTELLIGENCE; DATA MINING;

EVALUATION STRATEGIES; MODEL TRAINING; MODELING STRATEGY; MUSIC INFORMATION RETRIEVAL; SOFTWARE IMPLEMENTATION; STATE OF THE ART;

RESEARCH; SEARCH ENGINES;

EID: 84897934386     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2013.2294580     Document Type: Article

References (105)

1. Various, The Real Book 6th ed. Milwaukee, WI, USA, Hal LeonardCorp., 2004.
2. J. Weil, T. Sikora, J. Durrieu, and G. Richard, "Automatic generation of lead sheets from polyphonic music signals," in Proc. 10th Int. Soc. Music Inf. Retrieval Conf., 2009, pp. 603-608.
3. D. Ellis and G. Poliner, "Identifying 'cover songs' wisth chroma features and dynamic programming beat tracking," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2007, pp. 1429-1433.
4. E. Gómez and P. Herrera, "The song remains the same: Identifying versions of the same piece using tonal descriptors," in Proc. 7th Int.Soc. Music Inf. Retrieval, 2006, pp. 180-185.
5. B. Catteau, J. Martens, and M. Leman, "A probabilistic framework for audio-based tonal key and chord recognition," in Proc. 30th Annu.Conf. Gesellschaft fur Klassikation, 2007, pp. 637-644, Springer.
6. A. Shenoy and Y. Wang, "Key, chord, and rhythm tracking of popular music recordings," J. Comput. Music, vol. 29, no. 3, pp. 75-86, 2005. (Pubitemid 41294792)
7. K. Lee and M. Slaney, "Acoustic chord transcription and key extraction from audio using key-dependent HMMs trained on synthesized audio,"IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 2, pp. 291-301, Feb. 2008.
8. V. Zenz and A. Rauber, "Automatic chord detection incorporating beat and key detection," in Proc. IEEE Int. Conf. Signal Process. Commun., 2007, pp. 1175-1178.
9. C. Perez-Sancho, D. Rizo, and J. Inesta, "Genre classification using chords and stochastic language models," Connect. Sci., vol. 21, no. 2-3,pp. 145-159, 2009.
10. T. O'Hara, "Inferring the meaning of chord sequences via lyrics," in Proc. 2nd Worskshop Music Recommendation Discovery collocated with ACM-RecSys, 2011, p. 34.
11. M. Mauch, H. Fujihara, and M. Goto, "Integrating additional chord information into HMM-based lyrics-to-audio alignment," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 200-210, Jan. 2012.
12. M. Mauch, H. Fujihara, and M. Goto, "Lyrics-to-audio alignment and phrase-level segmentation using incomplete internet-style chord annotations,"in Proc. 7th Sound Music Comput. Conf., 2010, pp. 9-16.
13. J. Burgoyne, J. Wild, and I. Fujinaga, "An expert ground truth set for audio chord recognition and music analysis," in Proc. Int. Conf. Music Inf. Retrieval, 2011, pp. 633-638.
14. M. McVicar, Y. Ni, R. Santos-Rodriguez, and T. De Bie, "Using online chord databases to enhance chord recognition," J. New Music Res., vol. 40, no. 2, pp. 139-152, 2011.
15. R. Macrae and S. Dixon, "Guitar tab mining, analysis and ranking," in Proc. 12th Int. Soc. Music Inf. Retrieval Conf., 2011, pp. 453-458.
16. C. Harte, "Towards automatic extraction of harmony information from music signals," Ph.D. dissertation, Univ. of London, London, U.K., 2010.
17. M. Mauch, "Automatic chord transcription from audio using computational models of musical context," Ph.D. dissertation, Queen Mary Univ. of London, London, U.K., 2010.
18. K. Noland and M. Sandler, "Influences of signal processing, tone profiles, and chord progressions on a model for estimating the musical keyfrom audio," J. Comput. Music, vol. 33, no. 1, pp. 42-56, 2009.
19. C. Harte, M. Sandler, S. Abdallah, and E. Gómez, "Symbolic representation of musical chords: A proposed syntax for text annotations," in Proc. Int. Conf. Music Inf. Retrieval, 2005, pp. 66-71.
20. G. Wakefield, "Mathematical representation of joint time-chroma distributions,"in Proc. Int. Symp. Opt. Sci., Eng. Instrum., 1999, vol. 99,pp. 18-23.
21. R. Shepard, "Circularity in judgments of relative pitch," J. Acoust. Soc. Amer., vol. 36, p. 2346, 1964.
22. C. Chafe, Techniques for Note Identification in Polyphonic Music. Stanford, CA, USA: CCRMA, Dept. of Music, Stanford Univ., 1985.
23. C. Chafe and D. Jaffe, "Source separation and note identification in polyphonic music," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1986, vol. 11, pp. 1289-1292. (Pubitemid 17485180)
24. K. Martin, A blackboard system for automatic transcription of simplepolyphonic music Mass. Inst. of Technol. Media Lab. Perceptual Comput. Sec., Tech. Rep., no. 385, 1996.
25. K. Kashino and N. Hagita, "A music scene analysis system with the MRF-based information integration scheme," in Proc. 13th Int. Conf. Pattern Recogn., 1996, vol. 2, pp. 725-729.
26. J. Bello, G. Monti, and M. Sandler, "Techniques for automatic music transcription," in Proc. Int. Symp. Music Inf. Retrieval, 2000, pp.23-25.
27. C. Raphael, "Automatic transcription of piano music," in Proc. 3rd Int.Conf. Music Inf. Retrieval, 2002, vol. 2, pp. 13-17.
28. T. Fujishima, "Realtime chord recognition of musical sound: A system using Common Lisp Music," in Proc. Int. Comput. Music Conf., 1999, pp. 464-467.
29. D. Deutsch, The Psychology of Music. New York, NY, USA: Academic, 1999.
30. W. Heisenberg, "Über den anschaulichen inhalt der quantentheoretischen kinematik und mechanik," Zeitschrift für Physik A Hadrons and Nuclei, vol. 43, no. 3, pp. 172-198, 1927.
31. J. Brown, "Calculation of a Constant-Q spectral transform," J. Acoust.Soc. Amer., vol. 89, no. 1, pp. 425-434, 1991.
32. S. Nawab, S. Ayyash, and R. Wotiz, "Identification of musical chords using Constant-Q spectra," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2001, vol. 5, pp. 3373-3376. (Pubitemid 32838670)
33. T. Yoshioka, T. Kitahara, K. Komatani, T. Ogata, and H. Okuno, "Automatic chord transcription with concurrent recognition of chord symbols and boundaries," in Proc. 5th Int. Conf. Music Inf. Retrieval, 2004.
34. J. Bello and J. Pickens, "A robust mid-level representation for harmonic content in music signals," in Proc. 6th Int. Soc. Music Inf. Retrieval, 2005, pp. 304-311.
35. M. Mauch, K. Noland, and S. Dixon, "Using musical structure to enhance automatic chord transcription," in Proc. 10th Int. Conf. Music Inf. Retrieval, 2009, pp. 231-236.
36. Y. Ni, M. McVicar, R. Santos-Rodriguez, and T. De Bie, "An end-to-end machine learning system for harmonic analysis of music,"IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 6, pp.1771-1783, Aug. 2012.
37. R. Chen, W. Shen, A. Srinivasamurthy, and P. Chordia, "Chord recognition using duration-explicit hidden markov models," in Proc. 13th Int. Soc. Music Inf. Retrieval, 2012, pp. 445-450.
38. S. Pauws, "Musical key extraction from audio," in Proc. 5th Int. Soc.Music Inf. Retrieval, 2004, vol. 4, pp. 66-69.
39. N. Ono, K. Miyamoto, J. Le Roux, H. Kameoka, and S. Sagayama, "Separation of a monaural audio signal into harmonic/percussive components by complementary diffusion on spectrogram," in Proc. Euro.Signal Process. Conf., 2008, pp. 445-450.
40. J. Reed, Y. Ueda, S. Siniscalchi, Y. Uchiyama, S. Sagayama, and C.Lee, "Minimum classification error training to improve isolated chord recognition," in Proc. 10th Int. Soc. Music Inf. Retrieval, 2009, pp.609-614.
41. Y. Ueda, Y. Uchiyama, T. Nishimoto, N. Ono, and S. Sagayama,"HMM- based approach for automatic chord detection using refined acoustic features," in Proc. IEEE Int. Conf. Acoust. Speech, SignalProcess., 2010, pp. 5518-5521.
42. K. Lee and M. Slaney, "Automatic chord recognition from audio using an HMM with supervised learning," in Proc. 7th Int. Soc. Music Inf. Retrieval, 2006, pp. 133-137.
43. H. Papadopoulos and G. Peeters, "Large-scale study of chord estimation algorithms based on chroma representation and HMM," in Proc. Int. Workshop Content-Based Multimedia Indexing, 2007, pp. 53-60.
44. M. Mauch and S. Dixon, "Simultaneous estimation of chords and musical context from audio," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 6, pp. 1280-1289, Aug. 2010.
45. M. Varewyck, J. Pauwels, and J. Martens, "A novel chroma representation of polyphonic music based on multiple pitch tracking techniques,"in Proc. 16th Int. Conf. Multimedia, 2008, pp. 667-670.
46. C. Lawson and R. Hanson, Solving Least Squares Problems. Philadelphia, PA, USA: Soc. for Ind. Math., 1995, vol. 15.
47. A. Sheh and D. Ellis, "Chord segmentation and recognition using em-trained Hidden Markov Models," in Proc. 4th Int. Soc. Music Inf. Retrieval, 2003, pp. 183-189.
48. C. Harte and M. Sandler, "Automatic chord identification using a quantised chromagram," in Proc. Audio Eng. Soc., 2005, pp. 291-301.
49. C. Harte, M. Sandler, and M. Gasser, "Detecting harmonic change in musical audio," in Proc. 1st Workshop Audio Music Comput. Multimedia, 2006, pp. 21-26.
50. M. T. Smith, Audio engineer's reference book. Abingdon, U.K.: Focal Press, 1999.
51. O. Lartillot and P. Toiviainen, "A MATLAB toolbox for musical feature extraction from audio," in Proc. 10th Int. Conf. Digital Audio Effects, Bordeaux, France, 2007, pp. 237-244.
52. M. Goto and Y. Muraoka, "Real-time beat tracking for drumless audio signals: Chord change detection for musical decisions," Speech Commun., vol. 27, no. 3, pp. 311-335, 1999.
53. T. Cho and J. Bello, "A feature smoothing method for chord recognition using recurrence plots," in Proc. 12th Int. Soc. Music Inf. Retrieval Conf., 2011, pp. 651-656.
54. H. Papadopoulos and G. Peeters, "Simultaneous estimation of chord progression and downbeats froman audio file," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2008, pp. 121-124.
55. T. Cho, R. Weiss, and J. Bello, "Exploring common variations in state of the art chord recognition systems," in Proc. Sound Music Comput. Conf., 2010, vol. 1.
56. E. Chew, "Towards a mathematical model of tonality," Ph.D. dissertation, Mass. Inst. of Technol., Cambridge, MA, USA, 2000.
57. K. Lee and M. Slaney, "A unified system for chord transcription and key extraction using Hidden Markov Models," in Proc. Int. Conf. Music Inf. Retrieval, 2007, pp. 245-250.
58. K. Lee, "A system for automatic chord transcription from audio using genre-specific Hidden Markov Models," Adaptive Multimedial Retrieval: Retrieval, User, and Semantics, pp. 134-146, 2008.
59. K. Sumi, K. Itoyama, K. Yoshii, K. Komatani, T. Ogata, and H. Okuno,"Automatic chord recognition based on probabilistic integration of chord transition and bass pitch estimation," in Proc. Int. Conf. Music Inf. Retrieval, 2008, pp. 39-44.
60. M. Mauch and S. Dixon, "A discrete mixture model for chord labelling," in Proc. 9th Int. Conf. Music Inf. Retrieval, 2008, pp.45-50.
61. G. Cabral, F. Pachet, J. Briot, and S. Paris, "Automatic X traditional descriptor extraction: The case of chord recognition," in Proc. 6th Int.Conf. Music Inf. Retrieval, 2005, pp. 444-449.
62. B. Su and S. Jeng, "Multi-timbre chord classification using wavelet transform and self-organized map neural networks," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2001, vol. 5, pp. 3377-3380. (Pubitemid 32838671)
63. S. Kullback and R. Leibler, "On information and sufficiency," Ann. Math. Stat., vol. 22, no. 1, pp. 79-86, 1951.
64. L. Oudre, Y. Grenier, and C. Févotte, "Template-based chord recognition: Influence of the chord types," in Proc. 10th Int. Soc. Music Inf. Retrieval Conf., 2009, pp. 153-158.
65. L. Oudre, Y. Grenier, and C. Févotte, "Chord recognition using measures of fit, chord templates and filtering methods," in Proc. IEEE Workshop Appl. Signal Process. Audio Acoust., 2009, pp. 9-12.
66. W. de Haas, J. Magalhães, and F. Wiering, "Improving audio chord transcription by exploiting harmonic and metric knowledge," in Proc.13th Int. Soc. Music Inf. Retrieval, 2012.
67. L. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
68. T. Moon, "The expectation-maximization algorithm," IEEE Signal Process. Mag., vol. 13, no. 6, pp. 47-60, Nov. 1996.
69. T. Cho and J. Bello, "Real-time implementation of HMM-based chord estimation in musical audio," in Proc. Int. Comput. Music Conf., 2009,pp. 16-21.
70. A. Stark and M. Plumbley, "Real-time chord recognition for live performance," in Proc. Int. Comput. Music Conf., 2009, vol. 8, pp.585-593.
71. R. Scholz, E. Vincent, and F. Bimbot, "Robust modelling of musical chord sequences using probabilistic N-grams," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2009, pp. 53-56.
72. E. Unal, P. Georgiou, S. Narayanan, and E. Chew, "Statistical modeling and retrieval of polyphonic music," in Proc. 9th IEEE Workshop Multimedia Signal Process., 2007, pp. 405-409.
73. M. Mauch, S. Dixon, and C. Harte, "Discovering chord idioms through Beatles and Real Book songs," in Proc. 8th Int. Soc. Music Inf. Retrieval,2007, pp. 255-258.
74. M. Khadkevich and M. Omologo, "Use of hidden Markov models and factored language models for automatic chord recognition," in Proc. Int. Soc. Music Inf. Retrieval Conf., 2009, pp. 561-566.
75. K. Yoshii and M. Goto, "A vocabulary-free infinity-gram model for nonparametric Bayesian chord progression analysis," in Proc. 12th Int. Soc. Music Inf. Retrieval Conf., 2011, pp. 645-650.
76. J. Burgoyne, L. Pugin, C. Kereliuk, and I. Fujinaga, "A cross-validated study ofmodelling strategies for automatic chord recognition in audio,"in Proc. 8th Int. Conf. Music Inf. Retrieval, 2007, p. 251.
77. A. Weller, D. Ellis, and T. Jebara, "Structured prediction models for chord transcription of music audio," in Proc. Int. Conf. Mach. Learn. Appl., 2009, pp. 590-595.
78. M. McVicar, Y. Ni, R. Santos-Rodriguez, and T. De Bie, "Using online chord databases to enhance chord recognition," J. New Music Res., vol. 40, no. 2, pp. 139-152, 2011.
79. N. Jiang, P. Grosche, V. Konz, and M. Müller, "Analyzing chroma feature types for automated chord recognition," in Proc. 42nd Audio Eng. Soc. Conf., 2011, pp. 1-10.
80. J. Burgoyne and L. Saul, "Learning harmonic relationships in digital audio with Dirichlet-based Hidden Markov Models," in Proc. Int. Conf. Music Inf. Retrieval, 2005, pp. 438-443.
81. Y. Ni, M. Mcvicar, R. Santos-Rodrguez, and T. De Bie, "Using hypergenre training to explore genre information for automatic chord estimation,"in Proc. 13th Int. Soc. Music Inf. Retrieval, 2012, pp. 109-114.
82. M. Mauch and S. Dixon, "Approximate note transcription for the improved identification of difficult chords," in Proc. 11th Int. Soc. Music Inf. Retrieval Conf., 2010, pp. 135-140.
83. A. Berenzweig, B. Logan, D. Ellis, and B. Whitman, "A large-scale evaluation of acoustic and subjective music-similarity measures," J. Comput. Music, vol. 28, no. 2, pp. 63-76, 2004.
84. W. de Haas and J. Burgoyne, Parsing the Billboard chord transcriptions Univ. of Utrecht, Utrecht, The Netherlands, Tech. Rep., 2012.
85. F. Lerdahl, Tonal pitch space. New York, NY, USA: Oxford Univ. Press, 2005.
86. D. Ellis and A. Weller, "The 2010LabROSA chord recognition system,"in Proc. 11th Int. Soc. Music Inf. Retrieval (MIREX submission), 2010.
87. Y. Ni, M. McVicar, R. Santos-Rodrguez, and T. De Bie, "Harmony progression analyzer for MIREX 2012," in Proc. 13th Int. Soc. Music Inf. Retrieval (MIREX submission), 2012.
88. R. Macrae and S. Dixon, "A guitar tablature score follower," in Proc. IEEE Int. Conf. Multimedia Expo, 2010, pp. 725-726.
89. M. Barthet, A. Anglade, G. Fazekas, S. Kolozali, and R. Macrae,"Music recommendation for music learning: Hotttabs, a multimedia guitar tutor," in Proc. 2nd Worskshop Music Recommendation Discovery collocated with ACM-RecSys, 2011, p. 7.
90. M. McVicar and T. De Bie, "Enhancing chord recognition accuracy using web resources," in Proc. 3rd Int. Workshop Mach. Learn. Music, 2010, pp. 41-44.
91. J. Pauwels and G. Peeters, "Evaluating automatically estimated chord sequences," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2013, pp. 749-753.
92. V. Konz, M. Müller, and S. Ewert, "A multi-perspective evaluation framework for chord recognition," in Proc. 11th Int. Conf. Music Inf. Retrieval, 2010, pp. 9-14.
93. Y. Ni, M. McVicar, R. Santos-Rodriguez, and T. De Bie, "Understanding effects of subjectivity in measuring chord estimation accuracy," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no.12, pp. 2607-2615, Dec. 2013.
94. A. Wang and J. Smith, "System and methods for recognizing sound and music signals in high noise and distortion," U.S. patent 6,990,453, Jan. 24, 2006, III.
95. D. Temperley, "A unified probabilistic model for polyphonic music analysis," J. New Music Res., vol. 38, no. 1, pp. 3-18, 2009.
96. C. Cannam, C. Landone, and M. Sandler, "Sonic Visualiser: An open source application for viewing, analysing, and annotating music audiofiles," in Proc. ACM Multimedia 2010 Int. Conf., Oct. 2010, pp.1467-1468.
97. H. Papadopoulos and G. Tzanetakis, "Modeling chord and key structure with Markov logic," in Proc. 13th Int. Soc. Music Inf. Retrieval, 2012, pp. 121-126.
98. M. Bartsch and G. Wakefield, "To catch a chorus: Using chroma-based representations for audio thumbnailing," in Proc. Appl. Signal Process. Audio Acoust., 2001, pp. 15-18.
99. J. Paulus, M. Müller, and A. Klapuri, "State of the art report: Audio-based music structure analysis," in Proc. 11th Int. Soc. Music Inf. Retrieval Conf., 2010, pp. 625-36.
100. K. J. Pallesen, E. Brattico, C. Bailey, A. Korvenoja, J. Koivisto, A. Gjedde, and S. Carlson, "Emotion processing of major, minor, and dissonant chords," Ann. New York Acad. Sci., vol. 1060, no. 1, pp.450-453, 2005.
101. E. Schmidt and Y. Kim, "Modeling musical emotion dynamics with Conditional Random Fields," in Proc. 12th Int. Soc. Music Inf. Retrieval, 2011, pp. 777-782.
102. H.-T. Cheng, Y.-H. Yang, Y.-C. Lin, I.-B. Liao, and H.H. Chen, "Automatic chord recognition for music classification and retrieval," in Proc. IEEE Int. Conf. Multimedia Expo, 2008, pp. 1505-1508.
103. G. Linden, B. Smith, and J. York, "Amazon.com recommendations: Item-to-item collaborative filtering," IEEE Internet Comput., vol. 7,no. 1, pp. 76-80, Jan./Feb. 2003.
104. B. McFee and G. Lanckriet, "Metric learning to rank," in Proc. 27th Int. Conf. Mach. Learn., Haifa, Israel, 2010, pp. 775-782.
105. K. Kosta, M. Marchini, and H. Purwins, "Unsupervised chord-sequence generation from an audio example," in Proc. 13th Int. Soc. Music Inf. Retrieval, 2012, pp. 481-486.