Linearly augmented deep neural network

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn 2016-May, Issue , 2016, Pages 5085-5089

  Ghahremani, Pegah ^a   Droppo, Jasha ^b   Seltzer, Michael L ^b  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

acoustic modeling; deep network; Linear Augmented Network; pre training

Indexed keywords

EID: 84973305023     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2016.7472646     Document Type: Conference Paper

References (17)

1. Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdelrahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " Signal Processing Magazine, IEEE, vol. 29, no. 6, pp. 82-97, 2012.
2. Yoshua Bengio, Aaron Courville, and Pierre Vincent, "Representation learning: A review and new perspectives, " Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, no. 8, pp. 1798-1828, 2013.
3. Yoshua Bengio, Pascal Lamblin, Dan Popovici, Hugo Larochelle, et al., "Greedy layer-wise training of deep networks, " Advances in neural information processing systems, vol. 19, pp. 153, 2007.
4. Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu, "Deeply-supervised nets, " arXiv preprint arXiv: 1409. 5185, 2014.
5. Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio, "Maxout networks, " arXiv preprint arXiv: 1302. 4389, 2013.
6. Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio, "Fitnets: Hints for thin deep nets, " arXiv preprint arXiv: 1412. 6550, 2014.
7. Dong Yu, Frank Seide, Gang Li, and Li Deng, "Exploiting sparseness in deep neural networks for large vocabulary speech recognition, " in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, 2012, pp. 4409-4412.
8. Jian Xue, Jinyu Li, and Yifan Gong, "Restructuring of deep neural network acoustic models with singular value decomposition., " in INTERSPEECH, 2013, pp. 2365-2369.
9. John S Garofolo, Lori F Lamel, William M Fisher, Jonathon G Fiscus, and David S Pallett, "Darpa timit acoustic-phonetic continous speech corpus cd-rom. nist speech disc 1-1. 1, " NASA STI/Recon Technical Report N, vol. 93, pp. 27403, 1993.
10. Xavier Glorot and Yoshua Bengio, "Understanding the difficulty of training deep feedforward neural networks, " in International conference on artificial intelligence and statistics, 2010, pp. 249-256.
11. Li Deng, Xiaodong He, and Jianfeng Gao, "Deep stacking networks for information retrieval, " in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. IEEE, 2013, pp. 3153-3157.
12. Dong Yu, Adam Eversole, Mike Seltzer, Kaisheng Yao, Zhiheng Huang, Brian Guenter, Oleksii Kuchaiev, Yu Zhang, Frank Seide, Huaming Wang, et al., "An introduction to computational networks and the computational network toolkit, " Tech. Rep., Tech. Rep. MSR, Microsoft Research, 2014, http: //codebox/cntk, 2014.
13. Vinod Nair and Geoffrey E Hinton, "Rectified linear units improve restricted boltzmann machines, " in Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, pp. 807-814.
14. Jonathan G Fiscus, Jerome Ajot, and John S Garofolo, "The rich transcription 2007 meeting recognition evaluation, " in Multimodal Technologies for Perception of Humans, pp. 373-389. Springer, 2008.
15. Pawel Swietojanski, Arnab Ghoshal, and Steve Renals, "Hybrid acoustic models for distant and multichannel large vocabulary speech recognition, " in Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on. IEEE, 2013, pp. 285-290.
16. Geoffrey E Hinton, Simon Osindero, and Yee-Whye Teh, "A fast learning algorithm for deep belief nets, " Neural computation, vol. 18, no. 7, pp. 1527-1554, 2006.
17. Thomas Hain, Luká Burget, John Dines, Philip N Garner, Frantisek Grézl, Asmaa El Hannani, Marijn Huijbregts, Martin Karafiat, Mike Lincoln, and VincentWan, "Transcribing meetings with the amida systems, " Audio, Speech, and Language Processing, IEEE Transactions on, vol. 20, no. 2, pp. 486-498, 2012.