Multi-pitch streaming of harmonic sound mixtures

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 1, 2014, Pages 138-150

  Duan, Zhiyao ^a   Han, Jinyu ^b   Pardo, Bryan ^c  

^a University of Rochester   (United States)

^b Gracenote Inc   (United States)

^c Northwestern University   (United States)

Author keywords

Cochannel speech; Constrained clustering; Multi pitch analysis; Pitch streaming; Timbre tracking

Indexed keywords

ACOUSTIC GENERATORS; CLUSTERING ALGORITHMS; ESTIMATION; HARMONIC ANALYSIS; CONSTRAINED OPTIMIZATION;

CO-CHANNEL SPEECH; CONCURRENT SOUNDS; CONSTRAINED CLUSTERING; DISCRETE CEPSTRUM; HARMONIC STRUCTURES; MULTI PITCHES; MULTI-PITCH ESTIMATIONS; STREAMING APPROACH; TIME-FREQUENCY RELATIONSHIPS;

CONTINUOUS SPEECH RECOGNITION;

EID: 84897936477     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2013.2285484     Document Type: Article

References (51)

1. , A. Klapuri and M. Davy, Eds., Signal Processing Methods for Music Transcription . New York, NY, USA: Springer, 2006.
2. Z. Duan, Y. Zhang, C. Zhang, and Z. Shi, "Unsupervised single-channel music source separation by average harmonic structure modeling," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 4, pp. 766-778, May 2008.
3. J. Han and C.-W. Chen, "Improving melody extraction using probabilistic latent component analysis," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2011, pp. 33-36.
4. M. Cooke, J. R. Hershey, and S. Rennie, "Monaural speech separation and recognition challenge," Comput. Speech Lang., vol. 24, pp. 1-15, 2010.
5. D.-n. Jiang, W. Zhang, L.-q. Shen, and L.-h. Cai, "Prosody analysis andmodeling for emotional speech synthesis," in Proc. IEEE Int. Conf. Audio, Speech, Signal Process. (ICASSP), 2005, pp. 281-284.
6. E. C. Cherry, "Some experiments on the recognition of speech, with one and two ears," J. Acoust. Soc. Amer., vol. 25, pp. 975-979, 1953.
7. T. Tolonen and M. Karjalainen, "Acomputationally efficient multipitch analysis model," IEEE Trans. Speech Audio Process., vol. 8, no. 6, pp. 708-716, Nov. 2000.
8. A. de Cheveigné and H. Kawahara, "Multiple period estimation and pitch perceptionmodel," Speech Commun., vol. 27, pp. 175-185, 1999.
9. M. Davy, S. J. Godsill, and J. Idier, "Bayesian analysis of polyphonic western tonal music," J. Acoust. Soc. Amer., vol. 119, pp. 2498-2517, 2006.
10. R. C. Maher and J. W. Beauchamp, "Fundamental frequency estimation of musical signals using a two-way mismatch procedure," J. Acoust. Soc. Amer., vol. 95, no. 4, pp. 2254-2263, 1994.
11. M. Goto, "A real-time music-scene-description system: Predominant-f0 estimation for detecting melody and bass lines in real-world audio signals," Speech Commun., vol. 43, no. 4, pp. 311-329, 2004.
12. C. Yeh, A. Röbel, and X. Rodet, "Multiple fundamental frequency estimation of polyphonic music signals," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2005, pp. 225-228.
13. A. Pertusa and J. M. Inesta, "Multiple fundamental frequency estimation using gaussian smoothness," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2008, pp. 105-108.
14. A. Klapuri, "Multiple fundamental frequency estimation by summing harmonic amplitudes," in Proc. ISMIR, 2006, pp. 216-221.
15. V. Emiya, R. Badeau, and B. David, "Multipitch estimation of quasi-harmonic sounds in colored noise," in Proc. Int. Conf. Digital Audio Effects (DAFx), 2007.
16. S. Saito, H. Kameoka, K. Takahashi, T. Nishimoto, and S. Sagayama, "Specmurt analysis of polyphonic music signals," IEEE Trans. Speech Audio Process., vol. 16, no. 3, pp. 639-650, 2008.
17. Z. Duan, B. Pardo, and C. Zhang, "Multiple fundamental frequency estimation by modeling spectral peaks and non-peak regions," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 8, pp. 2121-2133, Nov. 2010.
18. M. Wu, D. Wang, and G. J. Brown, "A multipitch tracking algorithm for noisy speech," IEEE Trans. Speech Audio Process., vol. 11, no. 3, pp. 229-241, May 2003.
19. F. Sha and L. Saul, "Real-time pitch determination of one or more voices by nonnegative matrix factorization," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2005, pp. 1233-1240.
20. F. Bach and M. Jordan, "Discriminative training of hidden Markov models for multiple pitch tracking," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2005, pp. 489-492.
21. Z. Jin and D. Wang, "HMM-based multipitch tracking for noisy and reverberant speech," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 5, pp. 1091-1102, Jul. 2011.
22. M. Ryynanen and A. Klapuri, "Polyphonic music transcription using note event modeling," in Proc. IEEE Workshop Applicat. Signal Process. Audio Acoust. (WASPAA), 2005, pp. 319-322.
23. G. E. Poliner and D. P. W. Ellis, "A discriminative model for polyphonic piano transcription," EURASIP J. Adv. Signal Process., 2007, DOI:10.1155/2007/48317.
24. H. Kameoka, T. Nishimoto, and S. Sagayama, "A multipitch analyzer based on harmonic temporal structured clustering," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 3, pp. 982-994, Mar. 2007.
25. W.-C. Chang, A. W. Y. Su, C. Yeh, A. Robel, and X. Rodet, "Multiple-f0 tracking based on a high-order HMM model," in Proc. Int. Conf. Digital Audio Effects (DAFx), 2008.
26. J. Le Roux, H. Kameoka, N. Ono, A. de Cheveigne, and S. Sagayama, "Single and multiple f0 contour estimation through parametric spectrogram modeling of speech in noisy environments," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 4, pp. 1135-1145, Jul. 2007.
27. Z. Duan, J. Han, and B. Pardo, "Song-level multi-pitch tracking by heavily constrained clustering," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2010, pp. 57-60.
28. K. Kashino and H. Murase, "A sound source identification system for ensemblemusic based on template adaptation andmusic streamextraction," Speech Commun., pp. 337-349, 1999.
29. E. Vincent, "Musical source separation using time-frequency source priors," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 91-98, 2006. (Pubitemid 43863456)
30. M. Bay, A. F. Ehmann, J. W. Beauchamp, P. Smaragdis, and J. S. Downie, "Second fiddle is important too: Pitch tracking individual voices in polyphonic music," in Proc. Int. Soc. Music Inf. Retrieval Conf. (ISMIR), 2012, pp. 319-324.
31. M. Wohlmayr, M. Stark, and F. Pernkopf, "A probabilistic interaction model for multipitch tracking with factorial hidden Markov models," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 4, pp. 799-810, May 2011.
32. K. Hu and D. Wang, "An unsupervised approach to cochannel speech separation," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 1, pp. 122-131, Jan. 2013.
33. A. Bregman, Auditory Scene Analysis: The Perceptual Organization of Sound . Cambridge, MA, USA: MIT Press, 1990.
34. R. J. McAulay and T. F. Quatieri, "Speech analysis/synthesis based on a sinusoidal representation," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, no. 4, pp. 744-754, Aug. 1986.
35. P. Depalle, G. Garcia, and X. Rodet, "Tracking of partials for additive sound synthesis using hiddenmarkovmodels," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1993, pp. 225-228.
36. M. Lagrange and G. Tzanetakis, "Sound source tracking and formation using normalized cuts," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2007, pp. 61-64.
37. K. Wagstaff and C. Cardie, "Clustering with instance-level constraints," in Proc. Int. Conf. Mach. Learn. (ICML), 2000, pp. 1103-1110.
38. K. Wagstaff, C. Cardie, S. Rogers, and S. Schroedl, "Constrained k-means clustering with background knowledge," in Proc. Int. Conf. Mach. Learn. (ICML), 2001, pp. 577-584.
39. I. Davidson, S. Ravi, and M. Ester, "Efficient incremental constrained clustering," in Proc. ACM Conf. Knowl. Discov. Data Mining (KDD), 2007, pp. 240-249.
40. A. Klapuri, "Analysis of musical instrument sounds by source-filter-decay model," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2007, pp. 53-56.
41. J. J. Burred, A. Röbel, and T. Sikora, "Dynamic spectral envelope-modeling for timbre analysis of musical instrument sounds," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 3, pp. 663-674, Mar. 2010.
42. D. P. W. Ellis, PLP and RASTA (and MFCC, and inversion) in Matlab, 2005, online web resource.
43. Z. Duan and B. Pardo, "Soundprism: An online system for score-informed source separation of music audio," IEEE J. Sel. Topics Signal Process., vol. 5, no. 6, pp. 1205-1215, Dec. 2011.
44. T. Galas and X. Rodet, "An improved cepstral method for deconvolution of source-filter systems with discrete spectra: Application to musical sounds," in Proc. Int. Comput. Music Conf. (ICMC), 1990, pp. 82-84.
45. D. Schwarz, "Spectral envelopes in sound analysis and synthesis," Master's thesis, Univ. Stuttgart Fakultät Informatik, Stuttgart, Germany, 1998.
46. O. Cappé, J. Laroche, and E. Moulines, "Regularized estimation of cepstrum envelope from discrete frequency points," in Proc. IEEE Workshop Applicat. Signal Process. Audio Acoust. (WASPAA), 1995, pp. 213-216.
47. A. de Cheveigné and H. Kawahara, "Yin, a fundamental frequency estimator for speech and music," J. Acoust. Soc. Amer., vol. 111, pp. 1917-1930, 2002.
48. G. Pirker, M. Wohlmayr, S. Petrik, and F. Pernkopf, "A pitch tracking corpus with evaluation on multipitch tracking scenario," in Proc. Interspeech, 2011, pp. 1509-1512.
49. J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, D. Pallett, and N. Dahlgren, "The DARPA TIMIT acoustic-phonetic continuous speech corpus CDROM," NTIS, order number PB01-100354, 1993, now available from LDC.
50. P. Boersma, "Praat, a system for doing phonetics by computer," Glot Int., vol. 5, no. 9/10, pp. 341-345, 2001.
51. G. Hu and D. Wang, "A tandem algorithm for pitch estimation and voiced speech segregation," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 8, pp. 2067-2079, Nov. 2010.