Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription

Proceedings of the 29th International Conference on Machine Learning, ICML 2012

Volumn 2, Issue , 2012, Pages 1159-1166

  Boulanger Lewandowski, Nicolas ^a   Bengio, Yoshua ^a   Vincent, Pascal ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

DATA SETS; HIGH-DIMENSIONAL; LANGUAGE MODEL; POLYPHONIC MUSIC; PROBABILISTIC MODELS; SYMBOLIC SEQUENCE;

COMPUTATIONAL LINGUISTICS; LEARNING SYSTEMS; PROBABILITY DISTRIBUTIONS; RECURRENT NEURAL NETWORKS; TRANSCRIPTION;

COMPUTER MUSIC;

EID: 84867129058     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Allan, M. and Williams, C.K.I. Harmonising chorales by probabilistic inference. In NIPS 17, pp. 25-32, 2005.
2. Bay, M., Ehmann, A.F., and Downie, J.S. Evaluation of multiple-F0 estimation and tracking systems. In ISMIR, 2009.
3. Bengio, Y. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1-127, 2009.
4. Bengio, Y. and Bengio, S. Modeling high-dimensional discrete data with multi-layer neural networks. In NIPS 12, pp. 400-406, 2000.
5. Bengio, Y., Simard, P., and Frasconi, P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. on Neural Networks, 5(2):157-166, 1994.
6. Cemgil, A.T. Bayesian music transcription. PhD thesis, Radboud University Nijmegen, 2004.
7. Eck, D. and Schmidhuber, J. Finding temporal structure in music: Blues improvisation with LSTM recurrent networks. In NNSP, pp. 747-756, 2002.
8. Hinton, G.E. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8): 1771-1800, 2002.
9. Hsu, E., Pulli, K., and Popovic, J. Style translation for human motion. In SIGGRAPH, pp. 1082-1089, 2005.
10. Larochelle, H. and Murray, I. The neural autoregressive distribution estimator. JMLR: W&CP, 15:29-37, 2011.
11. Lavrenko, V. and Pickens, J. Polyphonic music modeling with random fields. In ACM MM, pp. 120-129, 2003.
12. Li, Y. and Wang, D.L. Pitch detection in polyphonic music using instrument tone models. ICASSP, 2:481, 2007.
13. Martens, J. and Sutskever, I. Learning recurrent neural networks with Hessian-free optimization. In ICML 28, 2011.
14. Mnih, V., Larochelle, H., and Hinton, G. Conditional restricted Boltzmann machines for structured output prediction. In UAI 27, 2011.
15. Mozer, M.C. Neural network music composition by prediction. Connection Science, 6(2):247-280, 1994.
16. Nam, J., Ngiam, J., Lee, H., and Slaney, M. A classification-based polyphonic piano transcription approach using learned feature representations. In ISMIR, 2011.
17. Paiement, J.F., Bengio, S., and Eck, D. Probabilistic models for melodic prediction. Artificial Intelligence, 173 (14):1266-1274, 2009.
18. Poliner, G.E. and Ellis, D.P.W. A discriminative model for polyphonic piano transcription. JASP, 2007(1):154-164, 2007.
19. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. Learning internal representations by error propagation. In Parallel Dist. Proc., pp. 318-362. MIT Press, 1986.
20. Salakhutdinov, R. and Murray, I. On the quantitative analysis of deep belief networks. In ICML 25, 2008.
21. Schrauwen, B. and Buesing, L. A hierarchy of recurrent networks for speech recognition. In NIPS 21, 2009.
22. Smolensky, P. Information processing in dynamical systems: Foundations of harmony theory. In Parallel Dist. Proc., pp. 194-281. MIT Press, 1986.
23. Sutskever, I. and Hinton, G.E. Learning multilevel distributed representations for high-dimensional sequences. In AISTATS, pp. 544-551, 2007.
24. Sutskever, I., Hinton, G., and Taylor, G. The recurrent temporal restricted Boltzmann machine. In NIPS 20, pp. 1601-1608, 2008.
25. Taylor, G.W., Hinton, G.E., and Roweis, S.T. Modeling human motion using binary latent variables. In NIPS 19, pp. 1345, 2007.
26. Welling, M., Rosen-Zvi, M., and Hinton, G. Exponential family harmoniums with an application to information retrieval. In NIPS 17, pp. 1481-1488, 2005.