Statistical Learning Theory and ELM for Big Social Data Analysis

IEEE Computational Intelligence Magazine

Volumn 11, Issue 3, 2016, Pages 45-55

  Oneto, Luca ^a   Bisio, Federica ^a   Cambria, Erik ^b   Anguita, Davide ^a  

^a UNIVERSITY OF GENOA   (Italy)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

CHARACTER RECOGNITION; DATA HANDLING; INFORMATION ANALYSIS; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; PUBLIC RELATIONS;

AMOUNT OF INFORMATION; BACKPROPAGATION NETWORK; EXTREME LEARNING MACHINE; GENERALIZATION PERFORMANCE; NATURAL LANGUAGE TEXT; SCIENTIFIC COMMUNITY; STATISTICAL LEARNING THEORY; TECHNOLOGICAL TOOLS;

BACKPROPAGATION;

EID: 84979680823     PISSN: 1556603X     EISSN: None     Source Type: Journal    
DOI: 10.1109/MCI.2016.2572540     Document Type: Article

References (96)

1. S. Mills, S. Lucas, L. Irakliotis, M. Rappa, T. Carlson, and B. Perlowitz, (2012). Demystifying big data: A practical guide to transforming the business of government. [Online]. Technical report. Available: http://www. ibm. com/software/data/demystifying-big-data
2. E. Cambria, H. Wang, and B. White, "Guest editorial: Big social data analysis, " Knowledge-Based Syst., vol. 69, pp. 1-2, 2014.
3. C. Cortes and V. Vapnik, "Support-vector networks, " Machine Learn., vol. 20, no. 3, pp. 273-297, 1995.
4. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, UK: Clarendon Press, 1995.
5. T. Cover and P. Hart, "Nearest neighbor pattern classification, " IEEE Trans. Inform. Theory, vol. 13, no. 1, pp. 21-27, 1967.
6. L. Breiman, "Random forests, " Machine Learn., vol. 45, no. 1, pp. 5-32, 2001.
7. X. Wu, X. Zhu, G. Q. Wu, and W. Ding, "Data mining with big data, " IEEE Trans. Knowl. Data Eng., vol. 26, no. 1, pp. 97-107, 2014.
8. Y. You, S. L. Song, H. Fu, A. Marquez, M. M. Dehnavi, K. Barker, K. W. Cameron, A. P. Randles, and G. Yang, "Mic-svm: Designing a highly efficient support vector machine for advanced modern multi-core and many-core architectures, " in Proc. IEEE Int. Parallel and Distributed Process. Symp., 2014.
9. G. Huang, G. B. Huang, S. Song, and K. You, "Trends in extreme learning machines: A review, " Neural Networks, vol. 61, pp. 32-48, 2015.
10. J. Xin, Z. Wang, C. Chen, L. Ding, G. Wang, and Y. Zhao, "ELM: Distributed extreme learning machine with mapreduce, " World Wide Web, vol. 17, no. 5, pp. 1189-1204, 2014.
11. Q. He, T. Shang, F. Zhuang, and Z. Shi, "Parallel extreme learning machine for regression based on mapreduce, " Neurocomputing, vol. 102, pp. 52-58, 2013.
12. S. Huang, B. Wang, J. Qiu, J. Yao, G. Wang, and G. Yu, "Parallel ensemble of online sequential extreme learning machine based on mapreduce, " in Proc. ELM-2014, 2015.
13. L. J. Cao, S. S. Keerthi, C. J. Ong, J. Q. Zhang, U. Periyathamby, X. J. Fu, and H. P. Lee, "Parallel sequential minimal optimization for the training of support vector machines, " IEEE Trans. Neural Networks, vol. 17, no. 4, pp. 1039-1049, 2006.
14. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica, "Resilient distributed datasets: A fault-Tolerant abstraction for in-memory cluster computing, " in Proc. USENIX Conf. Networked Systems Design and Implementation, 2012.
15. K. Olukotun, "Beyond parallel programming with domain specific languages, " in Proc. Symp. Principles and Practice of Parallel Programming, 2014.
16. A. Akusok, K. M. Bjork, Y. Miche, and A. Lendasse, "High-performance extreme learning machines: A complete toolbox for big data applications, " IEEE Access, vol. 3, pp. 1011-1025, 2015.
17. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica, "Spark: Cluster computing with working sets, " in Proc. USENIX Conf. Hot Topics in Cloud Computing, 2010.
18. J. L. Reyes-Ortiz, L. Oneto, and D. Anguita, "Big data analytics in the cloud: Spark on hadoop vs MPI/openMP on Beowulf, " Procedia Comp. Sci., vol. 53, pp. 121-130, Aug. 2015.
19. D. Agrawal, S. Das, and A. El Abbadi, "Big data and cloud computing: current state and future opportunities, " in Proc. Int. Conf. Extending Database Technology, 2011.
20. L. Breiman, "Statistical modeling: The two cultures (with comments and a rejoinder by the author), " Stat. Sci., vol. 16, no. 3, pp. 199-231, 2001.
21. V. Dhar, "Data science and prediction, " Commun. ACM, vol. 56, no. 12, pp. 64-73, 2013.
22. V. N. Vapnik, "An overview of statistical learning theory, " IEEE Trans. Neural Networks, vol. 10, no. 5, pp. 988-999, 1999.
23. V. Koltchinskii, "Rademacher penalties and structural risk minimization, " IEEE Trans. Inform. Theory, vol. 47, no. 5, pp. 1902-1914, 2001.
24. P. L. Bartlett, O. Bousquet, and S. Mendelson, "Local rademacher complexities, " Ann. Stat., vol. 33, no. 4, pp. 1497-1537, 2005.
25. O. Bousquet and A. Elisseeff, "Stability and generalization, " J. Machine Learn. Res, vol. 2, pp. 499-526, 2002.
26. L. Oneto, A. Ghio, S. Ridella, and D. Anguita, "Fully empirical and data-dependent stability-based bounds, " IEEE Trans. Cybernetics, vol. 45, no. 9, pp. 1913-1926, 2015.
27. L. G. Valiant, "A theory of the learnable, " Commun. ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
28. P. L. Bartlett, S. Boucheron, and G. Lugosi, "Model selection and error estimation, " Machine Learn., vol. 48, no. 1-3, pp. 85-113, 2002.
29. J. Langford, "Tutorial on practical prediction theory for classification, " J. Machine Learn. Res., vol. 6, no. 1, pp. 273, 2006.
30. D. Anguita, A. Ghio, L. Oneto, and S. Ridella, "In-sample and out-of-sample model selection and error estimation for support vector machines, " IEEE Trans. Neural Network Learn. Syst., vol. 23, no. 9, pp. 1390-1406, 2012.
31. E. Cambria and B. White, "Jumping NLP curves: A review of natural language processing research, " IEEE Comput. Intell. Mag., vol. 9, no. 2, pp. 48-57, 2014.
32. E. Cambria, B. Schuller, B. Liu, H. Wang, and C. Havasi, "Knowledge-based approaches to concept-level sentiment analysis, " IEEE Intell. Syst., vol. 28, no. 2, pp. 12-14, 2013.
33. E. Cambria, B. Schuller, B. Liu, H. Wang, and C. Havasi, "Statistical approaches to concept-level sentiment analysis, " IEEE Intell. Syst., vol. 28, no. 3, pp. 6-9, 2013.
34. A. Gangemi, V. Presutti, and D. Reforgiato, "Frame-based detection of opinion holders and topics: A model and a tool, " IEEE Comput. Intell. Mag., vol. 9, no. 1, pp. 20-30, 2014.
35. C. Strapparava and A. Valitutti, "WordNet-Affect: An affective extension of WordNet, " in Proc. Int. Conf. Language Resources and Evaluation, 2004.
36. E. Cambria, D. Olsher, and D. Rajagopal, "SenticNet 3: A common and common-sense knowledge base for cognition-driven sentiment analysis, " in Proc. AAAI, Quebec, 2014, pp. 1515-1521.
37. S. Poria, E. Cambria, A. Gelbukh, F. Bisio, and A. Hussain, "Sentiment data flow analysis by means of dynamic linguistic patterns, " IEEE Comput. Intell. Mag., vol. 10, no. 4, pp. 26-36, 2015.
38. D. Tang, F. Wei, B. Qin, T. Liu, and M. Zhou, "Coooolll: A deep learning system for twitter sentiment classification, " in Proc. 8th Int. Workshop on Semantic Evaluation, 2014.
39. C. N. dos Santos and M. Gatti, "Deep convolutional neural networks for sentiment analysis of short texts, " in Proc. Int. Conf. Computational Linguistics, 2014.
40. D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, and B. Qin, "Learning sentiment-specific word embedding for Twitter sentiment classification, " in Proc. Annu. Meeting of the Association for Computational Linguistics, 2014.
41. S. Poria, E. Cambria, and A. Gelbukh, "Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis, " in Proc. EMNLP, 2015, pp. 2539-2544.
42. V. Cherkassky, "The nature of statistical learning theory, " IEEE Trans. Neural Networks, vol. 8, no. 6, pp. 1564-1564, 1997.
43. S. Madden, "From databases to big data, " IEEE Internet Comput., no. 3, pp. 4-6, 2012.
44. A. G. Shoro and T. R. Soomro, "Big data analysis: Apache spark perspective, " Global J. Comp. Sci. Technol., vol. 15, no. 1, 2015.
45. L. Rosasco, E. De Vito, A. Caponnetto, M. Piana, and A. Verri, "Are loss functions all the same?, " Neural Computat., vol. 16, no. 5, pp. 1063-1076, 2004.
46. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer, 1996.
47. E. Cambria and G. B. Huang, "Extreme learning machines, " IEEE Intell. Syst., vol. 28, no. 6, pp. 30-59, 2013.
48. G. B. Huang, Q. Y. Zhu, and C. K. Siew, "Extreme learning machine: Theory and applications, " Neurocomputing, vol. 70, no. 1, pp. 489-501, 2006.
49. S. Ridella, S. Rovetta, and R. Zunino, "Circular backpropagation networks for classification, " IEEE Trans. Neural Networks, vol. 8, no. 1, pp. 84-97, 1997.
50. G. B. Huang, Q. Y. Zhu, and C.-K. Siew, "Extreme learning machine: A new learning scheme of feedforward neural networks, " in Proc. IEEE Int. Joint Conf. Neural Networks, 2004.
51. G. B. Huang, L. Chen, and C. K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes, " IEEE Trans. Neural Networks, vol. 17, no. 4, pp. 879-892, 2006.
52. F. Bisio, P. Gastaldo, R. Zunino, and E. Cambria, "A learning scheme based on similarity functions for affective common-sense reasoning, " in Proc. IJCNN, 2015, pp. 2476-2481.
53. G. B. Huang, H. Zhou, X. Ding, and R. Zhang, "Extreme learning machine for regression and multiclass classification, " IEEE Trans. Syst. Man Cybern. B, vol. 42, no. 2, pp. 513-529, 2012.
54. R. S. Xin, J. Rosen, M. Zaharia, M. J. Franklin, S. Shenker, and I. Stoica, "Shark: Sql and rich analytics at scale, " in Proc. ACM SIGMOD Int. Conf. Management of Data, 2013.
55. H. Furuta, T. Kameda, Y. Fukuda, and D. M. Frangopol, "Life-cycle cost analysis for infrastructure systems: Life cycle cost vs. safety level vs. service life, " in Proc. Life-Cycle Performance of Deteriorating Structures: Assessment, Design and Management, 2004.
56. A. G. Carlyle, S. L. Harrell, and P. M. Smith, "Cost-effective HPC: The community or the cloud?" in Proc. IEEE Int. Conf. Cloud Computing Technology and Science, 2010.
57. H. Karau, A. Konwinski, P. Wendell, and M. Zaharia, Learning Spark. Sebastopol, CA: OReilly Media, 2015.
58. A. Tarantola, Inverse Problem Theory and Methods for Model Parameter Estimation. Philadelphia, PA: SIAM, 2005.
59. R. E. Megginson, An Introduction to Banach Space Theory. New York: Springer, 2012, vol. 183.
60. H. Zou, T. Hastie, and R. Tibshirani, "On the degrees of freedom of the lasso, " Ann. Stat., vol. 35, no. 5, pp. 2173-2192, 2007.
61. C. De Mol, E. De Vito, and L. Rosasco, "Elastic-net regularization in learning theory, " J. Complex., vol. 25, no. 2, pp. 201-230, 2009.
62. V. N. Vapnik, Statistical Learning Theory. New York: Wiley-Interscience, 1998.
63. R. Rifkin, G. Yeo, and T. Poggio, "Regularized least-squares classification, " Nato Sci. Ser. Sub Ser. III Comp. Syst. Sci., vol. 190, pp. 131-154, 2003.
64. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers, " Neural Process. Lett., vol. 9, no. 3, pp. 293-300, 1999.
65. R. E. Fan, K. W. Chang, C. J. Hsieh, X. R. Wang, and C. J. Lin, "LIBLINEAR: A library for large linear classification, " J. Machine Learn. Res., vol. 9, pp. 1871-1874, 2008.
66. R. Caruana, S. Lawrence, and G. Lee, "Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping, " in Proc. Neural Information Processing Systems, 2001.
67. L. Prechelt, "Automatic early stopping using cross validation: quantifying the criteria, " Neural Networks, vol. 11, no. 4, pp. 761-767, 1998.
68. S. Gopalani and R. Arora, "Comparing apache spark and map reduce with performance analysis using k-means, " Int. J. Comp. Appl., vol. 113, no. 1, 2015.
69. S. Arlot and A. Celisse, "A survey of cross-validation procedures for model selection, " Stat. Survey., vol. 4, no. 2010, pp. 40-79, 2010.
70. D. A. McAllester, "Pac-bayesian stochastic model selection, " Machine Learn., vol. 51, no. 1, pp. 5-21, 2003.
71. R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection, " in Proc. Int. Joint Conf. Artificial Intelligence, San Francisco, CA, 1995, pp. 1137-1143.
72. I. Guyon, A. Saffari, G. Dror, and G. Cawley, "Model selection: Beyond the Bayesian/frequentist divide, " J. Machine Learn. Res., vol. 11, pp. 61-87, Mar. 2010.
73. D. Anguita, A. Ghio, L. Oneto, and S. Ridella, "In-sample model selection for support vector machines, " in Proc. Int. Joint Conf. Neural Networks, San Jose, CA, 2011, pp. 1154-1161.
74. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. London, UK: Chapman & Hall, 1993.
75. L. Oneto, A. Ghio, S. Ridella, and D. Anguita, "Global rademacher complexity bounds: From slow to fast convergence rates, " Neural Processing Letter, vol. 43, no. 2, pp. 567-602, Apr. 2016.
76. L. Oneto, A. Ghio, S. Ridella, and D. Anguita, "Local rademacher complexity: Sharper risk bounds with and without unlabeled samples, " Neural Networks, vol. 65, pp. 115-125, May 2015.
77. D. A. McAllester, "Some pac-bayesian theorems, " in Proc. ACM Computational Learning Theory, New York, 1998, pp. 230-234.
78. G. Lever, F. Laviolette, and J. Shawe-Taylor, "Tighter pac-bayes bounds through distribution-dependent priors, " Theoretic. Comp. Sci., vol. 473, pp. 4-28, 2013.
79. S. Floyd and M. Warmuth, "Sample compression, learnability, and the vapnik-chervonenkis dimension, " Machine Learn., vol. 21, no. 3, pp. 269-304, 1995.
80. T. Poggio, R. Rifkin, S. Mukherjee, and P. Niyogi, "General conditions for predictivity in learning theory, " Nature, vol. 428, no. 6981, pp. 419-422, 2004.
81. A. Inoue and L. Kilian, "In-sample or out-of-sample tests of predictability: Which one should we use?" Econometric Rev., vol. 23, no. 4, pp. 371-402, 2005.
82. F. Cheng, J. Yu, and H. Xiong, "Facial expression recognition in jaffe dataset based on gaussian process classification, " IEEE Trans. Neural Networks, vol. 21, no. 10, pp. 1685-1690, 2010.
83. D. Anguita, A. Ghio, S. Ridella, and D. Sterpi, "K-fold cross validation for error rate estimate in support vector machines, " in Proc. Int. Conf. Data Mining, 2009.
84. W. Hoeffding, "Probability inequalities for sums of bounded random variables, " J. Am. Stat. Assoc., vol. 58, no. 301, pp. 13-30, 1963.
85. P. L. Bartlett and S. Mendelson, "Rademacher and gaussian complexities: Risk bounds and structural results, " J. Machine Learn. Res., vol. 3, pp. 463-482, Mar. 2003.
86. A. Rakhlin, S. Mukherjee, and T. Poggio, "Stability results in learning theory, " Analysis Appl, vol. 3, no. 4, pp. 397-417, 2005.
87. A. Kleiner, A. Talwalkar, P. Sarkar, and M. I. Jordan, "A scalable bootstrap for massive data, " J. R. Stat. Soc. B Stat. Methodol., vol. 76, no. 4, pp. 795-816, 2014.
88. A. Kleiner, A. Talwalkar, P. Sarkar, and M. I. Jordan, "The big data bootstrap, " in Proc. Int. Conf. Machine Learning, 2012.
89. A. Kleiner, A. Talwalkar, P. Sarkar, and M. I. Jordan, "Bootstrapping big data, " in Proc. Advances in Neural Information Processing Systems, Workshop: Big Learning: Algorithms, Systems, and Tools for Learning at Scale, 2011.
90. L. Oneto, B. Pilarz, A. Ghio, and D. Anguita, "Model selection for big data: Algorithmic stability and bag of little bootstraps on gpus, " in Proc. Eur. Symp. Artificial Neural Networks, Computational Intelligence and Machine Learning, 2015.
91. R. Dietrich, M. Opper, and H. Sompolinsky, "Statistical mechanics of support vector networks, " Phys. Rev. Lett., vol. 82, no. 14, pp. 2975, 1999.
92. E. Cambria, "Affective computing and sentiment analysis, " IEEE Intell. Syst., vol. 31, no. 2, pp. 102-107, 2016.
93. E. Cambria, P. Gastaldo, F. Bisio, and R. Zunino, "An ELM-based model for affective analogical reasoning, " Neurocomputing, vol. 149, no. A, pp. 443-455, Feb. 2015.
94. E. Cambria, J. Fu, F. Bisio, and S. Poria, "AffectiveSpace 2: Enabling affective intuition for concept-level sentiment analysis, " in Proc. AAAI, Austin, TX, 2015, pp. 508-514.
95. E. Cambria and A. Hussain, Sentic Computing: A Common-Sense-Based Framework for Concept-Level Sentiment Analysis. Cham, Switzerland: Springer, 2015.
96. G. Huang, E. Cambria, K. Toh, B. Widrow, and Z. Xu, "New trends of learning in computational intelligence [guest editorial], " IEEE Comput. Intell. Mag., vol. 10, no. 2, pp. 16-17, 2015.