Deep learning applications and challenges in big data analytics

Journal of Big Data

Volumn 2, Issue 1, 2015, Pages

  Najafabadi, Maryam M ^a   Villanustre, Flavio ^b   Khoshgoftaar, Taghi M ^a   Seliya, Naeem ^a   Wald, Randall ^a   Muharemagic, Edin ^c  

^a FLORIDA ATLANTIC UNIVERSITY   (United States)

^b LexisNexis Risk Solutions   (United States)

^c LexisNexis Risk Solutions   (United States)

Author keywords

Big data; Deep learning

Indexed keywords

ABSTRACTING; BIG DATA; CLUSTERING ALGORITHMS; DATA MINING; DEEP LEARNING; INDEXING (OF INFORMATION); MEDICAL PROBLEMS; SEMANTICS; SUPERVISED LEARNING;

CYBER SECURITY; DATA ANALYTICS; DEEP LEARNING; DOMAIN-SPECIFIC INFORMATION; FRAUD DETECTION; GOOGLE+; MEDICAL INFORMATICS; MICROSOFT; SECURITY FRAUDS; SEMANTIC INDEXING;

LEARNING ALGORITHMS;

EID: 85013977220     PISSN: None     EISSN: 21961115     Source Type: Journal    
DOI: 10.1186/s40537-014-0007-7     Document Type: Article

References (60)

1. Domingos P (2012) A few useful things to know about machine learning. Commun ACM 55(10)
2. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference On. IEEE Vol. 1. pp 886–893
3. Lowe DG (1999) Object recognition from local scale-invariant features. In: Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference On. IEEE Computer Society Vol. 2. pp 1150–1157
4. Bengio Y, LeCun Y: Scaling learning algorithms towards, AI.In Large Scale Kernel Machines Edited by: Bottou L, Chapelle O, DeCoste D, Weston J. MIT Press, Cambridge, MA; 2007, 321–360. [http://www.iro.umontreal.ca/~lisa/pointeurs/bengio+lecun_chapter2007.pdf] http://www.iro.umontreal.ca/~lisa/pointeurs/bengio+lecun_chapter2007.pdf http://www.iro.umontreal.ca/~lisa/pointeurs/bengio+lecun_chapter2007.pdf
5. Bengio Y, Courville A, Vincent P: Representation learning: A review and new perspectives. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2013,35(8):1798–1828. doi:10.1109/TPAMI.2013.50 doi:10.1109/TPAMI.2013.50 10.1109/TPAMI.2013.50
6. Arel I, Rose DC, Karnowski TP: Deep machine learning-a new frontier in artificial intelligence research [research frontier]. IEEE Comput Intell 2010, 5: 13–18. 10.1109/MCI.2010.938364
7. Hinton GE, Osindero S, Teh Y-W: A fast learning algorithm for deep belief nets. Neural Comput 2006,18(7):1527–1554. 10.1162/neco.2006.18.7.1527
8. Bengio Y, Lamblin P, Popovici D, Larochelle H2007. Greedy layer-wise training of deep networks, Vol. 19. Bengio Y, Lamblin P, Popovici D, Larochelle H2007. Greedy layer-wise training of deep networks, Vol. 19.
9. Larochelle H, Bengio Y, Louradour J, Lamblin P: Exploring strategies for training deep neural networks. J Mach Learn Res 2009, 10: 1–40.
10. Salakhutdinov R, Hinton GE (2009) Deep boltzmann machines. In: International Conference on, Artificial Intelligence and Statistics. JMLR.org. pp 448–455
11. Goodfellow I, Lee H, Le QV, Saxe A, Ng AY (2009) Measuring invariances in deep networks. In: Advances in Neural Information Processing Systems. Curran Associates, Inc. pp 646–654
12. Dahl G, Ranzato M, Mohamed A-R, Hinton GE (2010) Phone recognition with the mean-covariance restricted boltzmann machine. In: Advances in Neural Information Processing Systems. Curran Associates, Inc. pp 469–477
13. Hinton G, Deng L, Yu D, Mohamed A-R, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath T, Dahl G, Kingsbury B: Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Signal Process Mag IEEE 2012,29(6):82–97. 10.1109/MSP.2012.2205597
14. Seide F, Li G, Yu D (2011) Conversational speech transcription using context-dependent deep neural networks. In: INTERSPEECH. ISCA. pp 437–440
15. Mohamed A-R, Dahl GE, Hinton G: Acoustic modeling using deep belief networks. Audio Speech Lang Process IEEE Trans 2012,20(1):14–22. 10.1109/TASL.2011.2109382
16. Dahl GE, Yu D, Deng L, Acero A: Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. Audio Speech Lang Process IEEE Trans 2012,20(1):30–42. 10.1109/TASL.2011.2134090
17. Krizhevsky A, Sutskever I, Hinton G (2012) Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems. Curran Associates, Inc. Vol. 25. pp 1106–1114
18. Mikolov T, Deoras A, Kombrink S, Burget L, Cernock`y J (2011) Empirical evaluation and combination of advanced language modeling techniques. In: INTERSPEECH. ISCA. pp 605–608
19. Socher R, Huang EH, Pennin J, Manning CD, Ng A (2011) Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In: Advances in Neural Information Processing Systems. Curran Associates, Inc. pp 801–809
20. Bordes A, Glorot X, Weston J, Bengio Y (2012) Joint learning of words and meaning representations for open-text semantic parsing. In: International Conference on Artificial Intelligence and Statistics. JMLR.org. pp 127–135
21. National Research Council: Frontiers in Massive Data Analysis. The National Academies Press, Washington, DC; 2013.
22. Dumbill E: What Is Big Data? An Introduction to the Big Data Landscape. In Strata 2012: Making Data Work. O’Reilly, Santa Clara, CA O’Reilly; 2012.
23. Khoshgoftaar TM (2013) Overcoming big data challenges. In: Proceedings of the 25th International Conference on Software Engineering and Knowledge Engineering, Boston, MA. ICSE. Invited Keynote Speaker
24. Bengio Y: Learning Deep Architectures for AI. Now Publishers Inc., Hanover, MA, USA; 2009.
25. Bengio Y: Deep learning of representations: Looking forward. In Proceedings of the 1st International Conference on Statistical Language and Speech Processing. SLSP’13. Springer, Tarragona, Spain; 2013:1–37. http://dx.doi.org/10.1007/978–3-642–39593–2_1 http://dx.doi.org/10.1007/978-3-642-39593-2_1 10.1007/978-3-642-39593-2_1
26. Hinton GE, Salakhutdinov RR (Science) Reducing the dimensionality of data with neural networks313(5786): 504–507. Hinton GE, Salakhutdinov RR (Science) Reducing the dimensionality of data with neural networks313(5786): 504–507.
27. Hinton GE, Zemel RS: Autoencoders, minimum description length, and helmholtz free energy. Adv Neural Inform Process Syst 1994, 6: 3–10.
28. Smolensky P (1986) Information processing in dynamical systems: foundations of harmony theory. In: Parallel Distributed Processing: Explorations in the Microstructure of Cognition. MIT Press. Vol. 1. pp 194–281
29. Hinton GE: Training products of experts by minimizing contrastive divergence. Neural Comput 2002,14(8):1771–1800. 10.1162/089976602760128018
30. Garshol LM (2013) Introduction to Big Data/Machine Learning. Online Slide Show,. http://www.slideshare.net/larsga/introduction-to-big-datamachine-learning., [http://www.slideshare.net/larsga/introduction-to-big-datamachine-learning]
31. Grobelnik M (2013) Big Data Tutorial. European Data Forum. http://www.slideshare.net/EUDataForum/edf2013-big-datatutorialmarkogrobelnik?related=1., Grobelnik M (2013) Big Data Tutorial. European Data Forum. http://www.slideshare.net/EUDataForum/edf2013-big-datatutorialmarkogrobelnik?related=1 Grobelnik M (2013) Big Data Tutorial. European Data Forum.
32. Salton G, Buckley C: Term-weighting approaches in automatic text retrieval. Inform Process Manag 1988,24(5):513–523. 10.1016/0306-4573(88)90021-0
33. Robertson SE, Walker S (1994) Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In: Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. pp 232–241. Springer-Verlag New York, Inc
34. Hinton G, Salakhutdinov R: Discovering binary codes for documents by learning deep generative models. Topics Cogn Sci 2011,3(1):74–91. 10.1111/j.1756-8765.2010.01109.x
35. Salakhutdinov R, Hinton G: Semantic hashing. Int J Approximate, Reasoning 2009,50(7):969–978. 10.1016/j.ijar.2008.11.006
36. Ranzato M, Szummer M (2008) Semi-supervised learning of compact document representations with deep=networks. In: Proceedings of the 25th International Conference on Machine Learning. ACM. pp 792–799
37. Mikolov T, Chen K, Dean J (2013) Efficient estimation of word representations in vector space. CoRR: Computing Research Repository: 1–12. abs/1301.3781
38. Dean J, Corrado G, Monga R, Chen K, Devin M, Le Q, Mao M, Ranzato M, Senior A, Tucker P, Yang K, Ng A (2012) Large scale distributed deep networks. In: Bartlett P, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ (eds)Advances in Neural Information Processing Systems, 1232–1240. http://books.nips.cc/papers/files/nips25/NIPS2012_0598.pdf., Dean J, Corrado G, Monga R, Chen K, Devin M, Le Q, Mao M, Ranzato M, Senior A, Tucker P, Yang K, Ng A (2012) Large scale distributed deep networks. In: Bartlett P, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ (eds)Advances in Neural Information Processing Systems, 1232–1240. http://books.nips.cc/papers/files/nips25/NIPS2012_0598.pdf
39. Mikolov T, Le QV, Sutskever I (2013) Exploiting similarities among languages for machine translation. CoRR: Comput Res Repository: 1–10. abs/1309.4168
40. Li G, Zhu H, Cheng G, Thambiratnam K, Chitsaz B, Yu D, Seide F (2012) Context-dependent deep neural networks for audio indexing of real-life data. In: Spoken Language Technology Workshop (SLT), 2012 IEEE. IEEE. pp 143–148
41. Zipern A (2001) A Quick Way to Search For Images on the Web. The New York Times. News Watch Article. http://www.nytimes.com/2001/07/12/technology/news-watch-a-quick-way-to-search-for-images-on-the-web.html., Zipern A (2001) A Quick Way to Search For Images on the Web. The New York Times. News Watch Article. http://www.nytimes.com/2001/07/12/technology/news-watch-a-quick-way-to-search-for-images-on-the-web.html
42. Cusumano MA: Google: What it is and what it is not. Commun ACM - Med Image Moeling 2005,48(2):15–17. doi:10.1145/1042091.1042107 doi:10.1145/1042091.1042107 10.1145/1042091.1042107
43. Lee H, Battle A, Raina R, Ng A (2006) Efficient sparse coding algorithms. In: Advances in Neural Information Processing Systems. MIT Press. pp 801–808
44. Le Q, Ranzato M, Monga R, Devin M, Chen K, Corrado G, Dean J, Ng A (2012) Building high-level features using large scale unsupervised learning. In: Proceeding of the 29th International Conference in Machine Learning, Edingburgh, Scotland
45. Freytag A, Rodner E, Bodesheim P, Denzler J: Labeling Examples that Matter: Relevance-Based Active Learning with Gaussian Processes. In 35th German Conference on Pattern Recognition (GCPR). Saarland University and Max-Planck-Institute for Informatics, Germany; 2013:282–291.
46. Socher R, Lin CC, Ng A, Manning C (2011) Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th International Conference on Machine Learning. Omnipress. pp 129–136
47. Kumar R, Talton JO, Ahmad S, Klemmer SR (2012) Data-driven web design. In: Proceedings of the 29th International Conference on Machine Learning. icml.cc/Omnipress
48. Le QV, Zou WY, Yeung SY, Ng AY (2011) Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. In: Computer Vision and Pattern Recognition (CVPR) 2011 IEEE Conference On. IEEE. pp 3361–3368
49. Zhou G, Sohn K, Lee H (2012) Online incremental feature learning with denoising autoencoders. In: International Conference on Artificial Intelligence and Statistics. JMLR.org. pp 1453–1461
50. Vincent P, Larochelle H, Bengio Y, Manzagol P-A (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th International Conference on Machine Learning. ACM. pp 1096–1103
51. Calandra R, Raiko T, Deisenroth MP, Pouzols FM: Learning deep belief networks from non-stationary streams. In Artificial Neural Networks and Machine Learning–ICANN 2012. Springer, Berlin Heidelberg; 2012:379–386. 10.1007/978-3-642-33266-1_47
52. Chen M, Xu ZE, Weinberger KQ, Sha F (2012) Marginalized denoising autoencoders for domain adaptation. In: Proceeding of the 29th International Conference in Machine Learning, Edingburgh, Scotland
53. Coates A, Ng A (2011) The importance of encoding versus training with sparse coding and vector quantization. In: Proceedings of the 28th International Conference on Machine Learning. Omnipress. pp 921–928
54. Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R (2012) Improving neural networks by preventing co-adaptation of feature detectors. CoRR: Comput Res Repository: 1–18. abs/1207.0580
55. Goodfellow IJ, Warde-Farley D, Mirza M, Courville A, Bengio Y (2013) Maxout networks. In: Proceeding of the 30th International Conference in Machine Learning, Atlanta, GA
56. Coates A, Huval B, Wang T, Wu D, Catanzaro B, Andrew N (2013) Deep learning with cots hpc systems. In: Proceedings of the 30th International Conference on Machine Learning. pp 1337–1345
57. Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: A deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11). pp 513–520
58. Chopra S, Balakrishnan S, Gopalan R (2013) Dlid: Deep learning for domain adaptation by interpolating betweendomains. In: Workshop on Challenges in Representation Learning, Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA
59. Suthaharan S: Big data classification: Problems and challenges in network intrusion prediction with machine learning. In ACM Sigmetrics: Big Data Analytics Workshop. ACM, Pittsburgh, PA; 2013.
60. Wang W, Lu D, Zhou X, Zhang B, Mu J: Statistical wavelet-based anomaly detection in big data with compressive sensing. EURASIP J Wireless Commun Netw 2013, 2013: 269. http://www.bibsonomy.org/bibtex/25e432dc7230087ab1cdc65925be6d4cb/dblp http://www.bibsonomy.org/bibtex/25e432dc7230087ab1cdc65925be6d4cb/dblp 10.1186/1687-1499-2013-269