Word level automatic alignment of music and lyrics using vocal synthesis

ACM Transactions on Multimedia Computing, Communications and Applications

Volumn 6, Issue 3, 2010, Pages

  Maddage, Namunu C ^a   Sim, Khe Chai ^b   Li, Haizhou ^b  

^a RMIT UNIVERSITY   (Australia)

^b INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

Music indexing; Music information modeling; Music lyrics alignment; Rhythm analysis; Singing voice synthesis

Indexed keywords

MUSIC INDEXING; MUSIC INFORMATION; MUSIC-LYRICS ALIGNMENT; RHYTHM ANALYSIS; SINGING-VOICE SYNTHESIS;

ALIGNMENT; AUDIO ACOUSTICS; INDEXING (OF INFORMATION); KNOWLEDGE REPRESENTATION;

COMPUTER MUSIC;

EID: 77956388808     PISSN: 15516857     EISSN: 15516865     Source Type: Journal    
DOI: 10.1145/1823746.1823753     Document Type: Conference Paper

References (35)

1. ROYAL SCHOOLS OF MUSIC. 1949. Rudiments and Theory of Music. The associated board of the Royal Schools of Music. London.
2. BARTSCH, M. A. AND WAKEFIELD, G. H. 2004. Singing voice identification using spectral envelop estimation. IEEE Trans. Audio Speech Lang. Proc. 12, 2, 100-109.
3. BERENZWEIG, A. L. AND ELLIS, D. P. W. 2001. Location singing voice segments within music signals. In Proceedings of IEEE Workshop on Applications of Signal processing to Audio and Acoustics (WASPAA).
4. BROWN, J. C. AND PUCKETTE, M. S. 1992. An efficient algorithm for the calculation of a constant Q transform. J. Acoustic Soc. Amer. 2, 5, 1933-1941
5. CHEN. K., GAO, S., ZHU, Y. AND SUN, Q. 2006. Popular song and lyrics synchronization and its application to music information retrieval. In Proceeding of Multimedia Networking and Computing.
6. DANNENBERG, R. B. 1984. An on-line algorithm for real-time accompaniment. In Proceedings of the International Computer Music Conference. 193-198.
7. DEUTSCH, D. 1988. The perceived height of octave-related complexes. J. Acous. Soc. Amer. 80, 5, 1346-1353.
8. DUXBURG, C., SANDLER, M. AND DAVIES, M. 2002. A hybrid approach to musical note onset detection. In Proceedings of the International Conference on Digital Audio Effects (DAFx).
9. ELLIS, D. P. W. AND POLINER, G. E. 2006. Identifying 'cover songs' with chroma features and dynamic programming beat tracking. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
10. FLETCHER, H. 1931. Some physical characteristics of speech and music. J. Acousti. Soc. Amer. 3, 2, 1-26.
11. FUJIHARA, H., GOTO, M., OGATA, J., KOMATANI, K., OGATA, T. AND OKUNO, H. G. 2006. Automatic synchronization between lyrics and music CD recordings based on viterbi alignment of segregated vocal signals. In Proceedings of the IEEE International Symposium on Multimedia (ISM).
12. FURINI, M. AND ALBORESI, L. 2004. Audio-text synchronization inside MP3 files: A new approach and its implementation. In Proceedings of the IEEE Consumer Communications & Networking.
13. GRUBB, L. AND DANNENBERG, R. B. 1997. A stochastic method of tracking a vocal performer. In Proceedings of the International Computer Music Conference (ICMC). 301-308.
14. HAMON, C. MOULINE, E., AND CHARPENTIER, F. 1989. A diphone synthesis system based on time-domain prosodic modifications of speech. In Proceedings of the IEEE International Conference on Acoustics Speech Signal Processing (ICASSP). 238-241.
15. HU, N., DANNENBERG, R. B. AND TZANETAKIS, G. 2003. Polyphonic audio matching and alignment for music retrieval. In Proceedings of the IEEE Workshop on Application of Signal Processing to Audio and Acoustics.
16. INOUE, W., HASHIMOTO, S. AND OHTERU, S. 1994. Adaptive karaoke system-human singing accompaniment based on speech recognition. In Proceedings of the International Computer Music Conference (ICMC). 70-77.
17. ISKANDAR, D., WANG, Y., KAN, M. Y., AND LI, H. 2006. Syllabic level automatic synchronization of music signals and text lyrics. In Proceedings of the ACM Multimedia Conference (ACM MM). 659-662.
18. JOHN, R. D., JOHN, H. L., AND JOHN, G. P. 1999. Discrete-Time Processing of Speech Signals. IEEE Press.
19. JOURDAIN, R. 1997. Music, The Brain, and Ecstasy: How Music Captures Our Imagination. HarperCollins.
20. KATAYOSE, H., KANOMORI, T., KAMEI, K., NAGASHIMA, Y., SATO, K., INOKUCHI, S., AND SIMURA, S. 1993. Virtual performer. In Proceedings of the International Computer Music Conference (ICMC), 138-145.
21. KIM, Y. E. 2003. Singing voice analysis/synthesis. PhD. Thesis, Massachusetts institute of Technology.
22. KORST, J., AND GELEIJNSE, G. 2006. Efficient lyrics retrieval and alignment. In Proceedings of Philips Symposium on Intelligent Algorithms.
23. LOSCOS, A., CANO, P., AND BONADA, J. 1999. Low-delay singing voice alignment to text. In Proceedings of the International Computer Music Conference (ICMC).
24. MADDAGE, N. C., LI, H., AND KANKANHALLI, M. S. 2006. Music structure based vector space retrieval. In Proceedings of ACM Special Interest Group on Information Retrieval (ACM SIGIR) Conference, 67-74.
25. MAKHOUL, J. 1975. Linear Prediction: A tutorial review. Proc. IEEE, 63, 4, 561-580.
26. NWE, T. L. AND WANG, Y. 2004. Automatic detection of vocal segments in popular songs. In Proceedings of the 5th International Symposium/Conference of Music Information Retrieval (ISMIR).
27. RABINER, L. R. AND JUANG, B. H. 1993. Fundamentals of Speech Recognition. Prentice-Hall.
28. SAKEO, H. AND CHIBA, S. 1978. Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Audio Speech, Lang. Proce. 26, 1, 43-49.
29. SHEH, A. AND ELLIS, D. P. W. 2003. Chord segmentation and recognition using EM-trained hidden Markov models. In Proceedings of the International Conference on Music Information (ISMIR).
30. STEVENS, S. S., VOLKMANN, J. AND NEWMAN, E. B. 1937. A scale for the measurement of the psychological magnitude pitch. J. Acoust. Soc. Amer. 8, 3, 185-190.
31. TAYLOR, P. A., BLACK, A. W., AND CALEY, R. J. 1998. The architecture of the festival speech synthesis system. In Proceedings of the 3rd International Workshop on Speech Synthesis.
32. TSAI, W. H., WANG, H. M., RODGERS, D., CHENG, S. S. AND YU, H. M. 2004. Blind clustering of popular music recordings based on singer voice characteristics. In Proceedings of the International Symposium of Music Information Retrieval (ISMIR).
33. WANG, Y., KAN, M. Y., NWE, T. L., SHENOY, A. AND YIN, J. 2004. LyricAlly: Automatic synchronization of acoustic musical signals and textual lyrics. In Proceedings of the ACM Multimedia Conference. 212-219.
34. WONG, C. H., SZETO, W. M. AND WONG, K. H. 2006. Automatic lyrics alignment for Cantonese popular music. In Multimedia Systems.
35. YOUNG, S., EVERMANN, G., GALES, M., HAIN, T., KERSHAW, D., LIU, X., MOORE, G., ODELL, J., OLLASON, D., POVEY, D., VALTCHEV, V., AND WOODLAND, P. 2006. The HTK Book Version 3.4. Department of Engineering, University of Cambridge.