Improved pre-training of deep belief networks using sparse encoding symmetric machines

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2012, Pages 4165-4168

  Plahl, Christian ^a   Sainath, Tara N ^b   Ramabhadran, Bhuvana ^b   Nahamoo, David ^b  

^a RWTH AACHEN UNIVERSITY   (Germany)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Deep belief network; neural network feature extraction; pre training; sparse representation

Indexed keywords

ALTERNATIVE METHODS; BROADCAST NEWS; CODEBOOKS; COMPUTATIONAL COSTS; DEEP BELIEF NETWORKS; OBJECTIVE FUNCTIONS; PRE-TRAINING; RESTRICTED BOLTZMANN MACHINE; SPARSE REPRESENTATION; STOPPING CRITERIA;

ENCODING (SYMBOLS); FEATURE EXTRACTION; SIGNAL PROCESSING; SPEECH RECOGNITION;

BAYESIAN NETWORKS;

EID: 84867584652     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2012.6288836     Document Type: Conference Paper

References (9)

1. M. Sundermeyer et.al, "The RWTH 2010 quaero ASR evaluation system for English, French, and German," in Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Prague, Czech Republic, May 2011, pp. 2212-2215.
2. Geoffrey E. Hinton, Simon Osindero, and Yee-Whye Teh, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, no. 7, pp. 1527-1554, July 2006.
3. Abdel-rahman Mohamed, Dong Yu, and Li Deng, "Investigation of full-sequence training of deep belief networks for speech recognition," in Interspeech, Makuhari, Japan, Sept. 2010, pp. 1692-1695.
4. Frank Seide, Li Gang, and Yu Dong, "Conversational Speech Transcription using context-dependent Deep Neural Network," in Interspeech, Florence, Italy, Aug. 2011, pp. 437-440.
5. Tara N. Sainath, Brian Kingsbury, Bhuvana Ramabhadran, Petr Fousek, and P. Novak, "Making Deep Belief Networks effective for Large Vocabulary Continuous Speech Recognition," in Proc. IEEE Automatic Speech Recognition and Understanding Workshop, Hawaii, USA, Dec. 2011.
6. Geoffrey E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Computations, vol. 14, no. 8, pp. 1771-1800, Aug. 2002.
7. Marc'Aurelio Ranzato, Y-Lan Boureau, and Yann LeCun, "Sparse feature learning for deep belief networks," in Advances in Neural Information Processing Systems, 2007.
8. Lori F. Lamel, Robert H. Kassel, and Stephanie Seneff, "Speech database development: Design and analysis of the acousticphonetic corpus," in Proceedings of the DARPA Speech Recognition Workshop, 1986, pp. 100-110.
9. Hagen Soltau, Georg Soan, and Brian Kingsbury, "The IBM Atilla Speech Recognition Toolkit," in IEEE Workshop on Spoken Language Technology, Berkley, CA, USA, May 2010, pp. 97-102.