High-Performance Extreme Learning Machines: A Complete Toolbox for Big Data Applications

IEEE Access

Volumn 3, Issue , 2015, Pages 1011-1025

  Akusok, Anton ^a   Bjork, Kaj Mikael ^b   Miche, Yoan ^c   Lendasse, Amaury ^a  

^a UNIVERSITY OF IOWA   (United States)

^b ARCADA UNIVERSITY OF APPLIED SCIENCES   (Finland)

^c NOKIA SIEMENS NETWORKS   (Finland)

Author keywords

Artificial neural networks; Computer applications; Feedforward neural networks; High performance computing Software; Learning systems; Machine learning; Neural networks; Open source software; Performance analysis; Prediction methods; Predictive models; Radial basis function networks; Scientific computing; Supervised learning; Utility programs

Indexed keywords

APPLICATION PROGRAMS; ARTIFICIAL INTELLIGENCE; COMPUTER APPLICATIONS; FEEDFORWARD NEURAL NETWORKS; KNOWLEDGE ACQUISITION; LEARNING ALGORITHMS; LEARNING SYSTEMS; NATURAL SCIENCES COMPUTING; NEURAL NETWORKS; OPEN SOURCE SOFTWARE; OPEN SYSTEMS; RADIAL BASIS FUNCTION NETWORKS; SOFTWARE ENGINEERING; SUPERVISED LEARNING; UTILITY PROGRAMS;

BIG DATA APPLICATIONS; EXTREME LEARNING MACHINE; HIGH PERFORMANCE COMPUTING; MACHINE LEARNING PROBLEM; PERFORMANCE ANALYSIS; PREDICTION METHODS; PREDICTIVE MODELS; SOFTWARE AND HARDWARES;

BIG DATA;

EID: 84959859043     PISSN: None     EISSN: 21693536     Source Type: Journal    
DOI: 10.1109/ACCESS.2015.2450498     Document Type: Article

References (43)

1. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: A new learning scheme of feedforward neural networks, " in Proc. IEEE Int. Joint Conf. Neural Netw., vol. 2. Jul. 2004, pp. 985-990.
2. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: Theory and applications, " Neurocomputing, vol. 70, nos. 1-3, pp. 489-501, 2006. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0925231206000385
3. G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, "Extreme learning machine for regression and multiclass classi-cation, " IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 513-529, Apr. 2012. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/21984515
4. E. Cambria et al., "Extreme learning machines [trends & controversies], " IEEE Intell. Syst., vol. 28, no. 6, pp. 30-59, Nov./Dec. 2013. [Online]. Available: http://sentic.net/extreme-learning-machines.pdf
5. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes, " IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006. [Online]. Available: http://ieeexplore.ieee.org/xpls/abs-all.jsp?arnumber=1650244
6. G.-B. Huang, "What are extreme learning machines? Filling the gap between frank Rosenblatt's dream and John von Neumann's puzzle, " Cognit. Comput., vol. 7, no. 3, pp. 263-278, 2015.
7. G.-B. Huang, "An insight into extreme learning machines: Random neurons, random features and kernels, " Cognit. Comput., vol. 6, no. 3, pp. 376-390, 2014.
8. H. White, "An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks, " in Proc. Int. Joint Conf. Neural Netw. (IJCNN), vol. 2. 1989, pp. 451-455.
9. H. White, Approximate Nonlinear Forecasting Methods (Handbook of Economic Forecasting), vol. 1, G. Elliott, C. W. J. Granger, and A. Timmermann, Eds. Amsterdam, The Netherlands: Elsevier, 2006, ch. 9, pp. 459-512. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S1574070605010098
10. Y.-H. Pao, G.-H. Park, and D. J. Sobajic, "Learning and generalization characteristics of the random vector functional-link net, " Neurocomputing, vol. 6, no. 2, pp. 163-180, 1994. [Online]. Available: http://www. sciencedirect.com/science/article/pii/0925231294900531
11. W. F. Schmidt, M. A. Kraaijveld, and R. P. W. Duin, "Feedforward neural networks with random weights, " in Proc. 11th IAPR Int. Conf. Pattern Recognit., vol. 2. Aug./Sep. 1992, pp. 1-4.
12. B. Igelnik and Y.-H. Pao, "Stochastic choice of basis functions in adaptive function approximation and the functional-link net, " IEEE Trans. Neural Netw., vol. 6, no. 6, pp. 1320-1329, Nov. 1995.
13. S. Haykin, Neural Networks: A Comprehensive Foundation. Englewood Cliffs, NJ, USA: Prentice-Hall, 2004.
14. F. Rosenblatt, "The perceptron: A probabilistic model for information storage and organization in the brain, " Psychol. Rev., vol. 65, no. 6, pp. 386-408, Nov. 1958. [Online]. Available: http://www.ncbi. nlm.nih.gov/pubmed/13602029
15. C. Cortes and V. Vapnik, "Support-vector networks, " Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995. [Online]. Available: http://www. springerlink.com/index/K238JX04HM87J80G.pdf
16. G.-B. Huang, Q.-Y. Zhu, K. Z. Mao, C.-K. Siew, P. Saratchandran, and N. Sundararajan, "Can threshold networks be trained directly?" IEEE Trans. Circuits Syst. II, Exp. Briefs, vol. 53, no. 3, pp. 187-191, Mar. 2006. [Online]. Available: http://ieeexplore.ieee.org/xpls/abs-all. jsp?arnumber=1605431
17. G. Tsoumakas and I. Katakis, "Multi-label classi-cation: An overview, " Int. J. Data Warehousing Mining, vol. 3, no. 3, pp. 1-13, 2007.
18. Y. Zhai, Y.-S. Ong, and I.W. Tsang, "The emerging big dimensionality, "' IEEE Comput. Intell. Mag., vol. 9, no. 3, pp. 14-26, Aug. 2014.
19. A. Akusok, Y. Miche, J. Hegedus, R. Nian, and A. Lendasse, "A two-stage methodology using K-NN and false-positive minimizing ELM for nominal data classi-cation, " Cognit. Comput., vol. 6, no. 3, pp. 432-445, 2014.
20. A. Akusok, A. Grigorievskiy, A. Lendasse, and Y. Miche, "Image-based classi-cation of websites, " in Machine Learning Reports, vol. 18, T. Villmann and F.-M. Schleif, Eds. Saarbrücken, Germany: GCPR, Sep. 2013, pp. 25-34.[Online]. Available: http://www.techfak.uni-bielefeld.de/~fschleif/mlr/mlr-02-2013.pdf
21. A. Akusok, Y. Miche, J. Karhunen, K.-M. Bjork, R. Nian, and A. Lendasse, "Arbitrary category classi-cation of websites based on image content, " IEEE Comput. Intell. Mag., vol. 10, no. 2, pp. 30-41, May 2015.
22. G.-B. Huang, Z. Bai, L. L. C. Kasun, and C. M. Vong, "Local receptive-elds based extreme learning machine, " IEEE Comput. Intell. Mag., vol. 10, no. 2, pp. 18-29, May 2015.
23. N.-Y. Liang, G.-B. Huang, P. Saratchandran, and N. Sundararajan, "A fast and accurate online sequential learning algorithm for feedforward networks, " IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1411-1423, Nov. 2006.
24. A. van Schaik and J. Tapson, "Online and adaptive pseudoinverse solutions for ELM weights, " Neurocomputing, vol. 149, pp. 233-238, Feb. 2015. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0925231214011485
25. M. van Heeswijk et al., "Adaptive ensemble models of extreme learning machines for time series prediction, " in Proc. 19th Int. Conf. Artif. Neural Netw. (ICANN), 2009, pp. 305-314. [Online]. Available: http://dx.doi.org/10.1007/978-3-642-04277-5-31
26. M. van Heeswijk, Y. Miche, E. Oja, and A. Lendasse, "GPU-accelerated and parallelized ELM ensembles for large-scale regression, " Neurocomputing, vol. 74, no. 16, pp. 2430-2437, Sep. 2011.
27. Y. Miche, P. Bas, C. Jutten, O. Simula, and A. Lendasse, "A methodology for building regression models using extreme learning machine: OP-ELM, " in Proc. ESANN, 2008, pp. 247-252. [Online]. Available: http://dblp.uni-trier.de/db/conf/esann/esann2008.html#MicheBJSL08
28. Y. Miche, A. Sorjamaa, P. Bas, O. Simula, C. Jutten, and A. Lendasse, "OP-ELM: Optimally pruned extreme learning machine, " IEEE Trans. Neural Netw., vol. 21, no. 1, pp. 158-162, Jan. 2010. [Online]. Available: http://www.ncbi.nlm.nih.gov/pubmed/20007026
29. Y. Miche, M. van Heeswijk, P. Bas, O. Simula, and A. Lendasse, "TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization, " Neurocomputing, vol. 74, no. 16, pp. 2413-2421, Sep. 2011. [Online]. Available: http://linkinghub.elsevier.com/retrieve/pii/S092523121100258X
30. Q. Yu, Y. Miche, E. Eirola, M. van Heeswijk, E. Séverin, and A. Lendasse, "Regularized extreme learning machine for regression with missing data, " Neurocomputing, vol. 102, pp. 45-51, Feb. 2013. [Online]. Available: http://www.scopus.com/inward/record.url?eid=2-s2.0-84870244730 &partnerID=40&md5=4bc9805ef198a3de44fd2c2a976834b5
31. W. Zong, G.-B. Huang, and Y. Chen, "Weighted extreme learning machine for imbalance learning, " Neurocomputing, vol. 101, pp. 229-242, Feb. 2013. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0925231212006479
32. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks, " IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
33. C. R. Rao and S. K. Mitra, "Generalized inverse of a matrix and its applications, " in Proceedings of the Sixth Berkeley Symposium on Math-ematical Statistics and Probability: Theory of Statistics, vol. 1. Berkeley, CA, USA: Univ. California Press, 1972, pp. 601-620. [Online]. Available: http://projecteuclid.org/euclid.bsmsp/1200514113
34. A. N. Tikhonov, "Solution of incorrectly formulated problems and the regularization method, " Soviet Math. Doklady, vol. 4, pp. 1035-1038, 1963.
35. A. E. Hoerl and R. W. Kennard, "Ridge regression: Applications to nonorthogonal problems, " Technometrics, vol. 12, no. 1, pp. 69-82, 1970.
36. T. Similä and J. Tikka, "Multiresponse sparse regression with application to multidimensional scaling, " in Proc. 15th Int. Conf. Artif. Neural Netw., Formal Models Appl. (ICANN), 2005, pp. 97-102. [Online]. Available: http://dl.acm.org/citation.cfm?id=1986079.1986097
37. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression, " Ann. Statist., vol. 32, no. 2, pp. 407-499, 2004.
38. T. Simila and J. Tikka, "Common subset selection of inputs in multiresponse regression, " in Proc. Int. Joint Conf. Neural Netw. (IJCNN), 2006, pp. 1908-1915.
39. M. Lichman. (2013). UCI Machine Learning Repository. [Online]. Available: http://archive.ics.uci.edu/ml
40. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines, " ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, 2011, Art. ID 27.
41. C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics), vol. 4, M. Jordan, J. Kleinberg, and B. Schölkopf, Eds. New York, NY, USA: Springer-Verlag, 2006, no. 4. [Online]. Available: http://www.library.wisc.edu/selectedtocs/bg0137.pdf
42. C. E. Rasmussen, "Gaussian processes in machine learning, " in Advanced Lectures on Machine Learning (Lecture Notes in Computer Science), O. Bousquet, U. von Luxburg, and G. Rätsch, Eds. Berlin, Germany: Springer-Verlag, 2004, pp. 63-71. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-28650-9-4
43. S. L. Phung, A. Bouzerdoum, and D. Chai, "Skin segmentation using color pixel classi-cation: Analysis and comparison, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 1, pp. 148-154, Jan. 2005.