Local receptive fields based extreme learning machine

IEEE Computational Intelligence Magazine

Volumn 10, Issue 2, 2015, Pages 18-29

  Huang, Guang Bin ^a   Bai, Zuo ^a   Kasun, Liyanaarachchi Lekamalage Chamara ^a   Vong, Chi Man ^b  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b UNIVERSITY OF MACAU   (Macao)

Author keywords

[No Author keywords available]

Indexed keywords

ARCHITECTURE; FEEDFORWARD NEURAL NETWORKS; ITERATIVE METHODS; KNOWLEDGE ACQUISITION; NETWORK ARCHITECTURE; NETWORK LAYERS; OBJECT RECOGNITION; PROBABILITY DISTRIBUTIONS;

CONTINUOUS PROBABILITY DISTRIBUTION; EXTREME LEARNING MACHINE; FEATURE LEARNING; GENERAL ARCHITECTURES; LOCAL CONNECTIONS; RECEPTIVE FIELDS; SINGLE-HIDDEN LAYER FEEDFORWARD NEURAL NETWORKS; UNIVERSAL APPROXIMATION;

DEEP LEARNING;

EID: 84928106753     PISSN: 1556603X     EISSN: 15566048     Source Type: Journal    
DOI: 10.1109/MCI.2015.2405316     Document Type: Article

References (66)

1. C. P. Chen and C.-Y. Zhang, "Data-intensive applications, challenges, techniques and technologies: A survey on big data," Inform. Sci., vol. 275, pp. 314-347, Aug. 2014.
2. Z.-H. Zhou, N. Chawla, Y. Jin, and G. Williams, "Big data opportunities and challenges: Discussions from data analytics perspectives," IEEE Comput. Intell. Mag., vol. 9, pp. 62-74, Nov. 2014.
3. Y. Zhai, Y.-S. Ong, and I. Tsang, "The emerging 'big dimensionality,'" IEEE Comput. Intell. Mag., vol. 9, pp. 14-26, Aug. 2014.
4. I. Arnaldo, K. Veeramachaneni, A. Song, and U. O'Reilly, "Bring your own learner: A cloud-based, data-parallel commons for machine learning," IEEE Comput. Intell. Mag., vol. 10, pp. 20-32, Feb. 2015.
5. L. L. C. Kasun, H. Zhou, G.-B. Huang, and C. M. Vong, "Representational learning with extreme learning machine for big data," IEEE Intell. Syst., vol. 28, no. 6, pp. 31-34, 2013.
6. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagation errors," Nature, vol. 323, pp. 533-536, Oct. 1986.
7. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: A new learning scheme of feedforward neural networks," in Proc. Int. Joint Conf. Neural Networks, July 2004, vol. 2, pp. 985-990.
8. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70, pp. 489-501, Dec. 2006.
9. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Networks, vol. 17, no. 4, pp. 879-892, 2006.
10. G.-B. Huang and L. Chen, "Convex incremental extreme learning machine," Neurocomputing, vol. 70, pp. 3056-3062, Oct. 2007.
11. G.-B. Huang and L. Chen, "Enhanced random search based incremental extreme learning machine," Neurocomputing, vol. 71, pp. 3460-3468, Oct. 2008.
12. G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, "Extreme learning machine for regression and multiclass classification," IEEE Trans. Syst. Man, Cybern. B, vol. 42, no. 2, pp. 513-529, 2012.
13. G.-B. Huang, "An insight into extreme learning machines: Random neurons, random features and kernels," Cognitive Comput., vol. 6, no. 3, pp. 376-390, 2014.
14. Z.-H. You, Y.-K. Lei, L. Zhu, J. Xia, and B. Wang, "Prediction of proteinprotein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis," BMC Bioinform., vol. 14, suppl. 8, p. S10, 2013.
15. Y. Song, J. Crowcroft, and J. Zhang, "Automatic epileptic seizure detection in EEGs based on optimized sample entropy and extreme learning machine," J. Neurosci. Methods, vol. 210, no. 2, pp. 132-146, 2012.
16. Y. Zhang and P. Zhang, "Optimization of nonlinear process based on sequential extreme learning machine," Chem. Eng. Sci., vol. 66, no. 20, pp. 4702-4710, 2011.
17. Y. Yang, Y. Wang, and X. Yuan, "Bidirectional extreme learning machine for regression problem and its learning effectiveness," IEEE Trans. Neural Networks Learning Syst., vol. 23, no. 9, pp. 1498-1505, 2012.
18. Z. Yan and J. Wang, "Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks," IEEE Trans. Neural Networks Learning Syst., vol. 25, no. 3, pp. 457-469, 2014.
19. A. Nizar, Z. Dong, and Y. Wang, "Power utility nontechnical loss analysis with extreme learning machine method," IEEE Trans. Power Syst., vol. 23, no. 3, pp. 946-955, 2008.
20. R. Minhas, A. A. Mohammed, and Q. M. J. Wu, "Incremental learning in human action recognition based on snippets," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 11, pp. 1529-1541, 2012.
21. Y. Zhou, J. Peng, and C. L. P. Chen, "Extreme learning machine with composite kernels for hyperspectral image classification," IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing, 2014, to be published.
22. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, "What is the best multistage architecture for object recognition?" in Proc. IEEE Int. Conf. Computer Vision, 2009, pp. 2146-2153.
23. Z. Bai, G.-B. Huang, D. Wang, H. Wang, and M. B. Westover, "Sparse extreme learning machine for classification," IEEE Trans. Cybern., vol. 44, no. 10, pp. 1858-1870, 2014.
24. S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, "Face recognition: A convolutional neural-network approach," IEEE Trans. Neural Networks, vol. 8, no. 1, pp. 98-113, 1997.
25. J. Hawkins and S. Blakeslee, On Intelligence. New York: Macmillan, 2007.
26. G. Huang, G.-B. Huang, S. Song, and K. You, "Trends in extreme learning machines: A review," Neural Netw., vol. 61, no. 1, pp. 32-48, 2015.
27. Y.-H. Pao, G.-H. Park, and D. J. Sobajic, "Learning and generalization characteristics of the random vector functional-link net," Neurocomputing, vol. 6, pp. 163-180, Apr. 1994.
28. C. P. Chen, "A rapid supervised learning neural network for function interpolation and approximation," IEEE Trans. Neural Networks, vol. 7, no. 5, pp. 1220-1230, 1996.
29. C. Chen, S. R. LeClair, and Y.-H. Pao, "An incremental adaptive implementation of functional-link processing for function approximation, time-series prediction, and system identification," Neurocomputing, vol. 18, no. 1, pp. 11-31, 1998.
30. C. P. Chen and J. Z. Wan, "A rapid learning and dynamic stepwise updating algorithm for f lat neural networks and the application to time-series prediction," IEEE Trans. Syst. Man, Cybern. B, vol. 29, no. 1, pp. 62-72, 1999.
31. Y.-H. Pao, Adaptive Pattern Recognition and Neural Networks. Boston, MA: Addison-Wesley Longman Publishing Co., Inc., 1989.
32. B. Igelnik and Y.-H. Pao, "Stochastic choice of basis functions in adaptive function approximation and the functional-link net," IEEE Trans. Neural Networks, vol. 6, no. 6, pp. 1320-1329, 1995.
33. H. White, "An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks," in Proc. Int. Conf. Neural Networks, 1989, pp. 451-455.
34. T.-H. Lee, H. White, and C. W. J. Granger, "Testing for neglected nonlinearity in time series models: A comparison of neural network methods and standard tests," J. Econometrics, vol. 56, pp. 269-290, 1993.
35. M. B. Stinchcombe and H. White, "Consistent specification testing with nuisance parameters present only under the alternative," Econometric Theory, vol. 14, no. 3, pp. 295-324, 1998.
36. H. White, "Approximate nonlinear forecasting methods," in Handbook of Economics Forecasting, G. Elliott, C. W. J. Granger, and A. Timmermann, Eds. New York: Elsevier, 2006, pp. 460-512.
37. G. Huang, S. Song, J. N. Gupta, and C. Wu, "Semi-supervised and unsupervised extreme learning machines," IEEE Trans. Cybern., vol. 44, no. 12, pp. 2405-2417, 2014.
38. A. Rahimi and B. Recht, "Uniform approximation of functions with random bases," in Proc. 46th Annu. Allerton Conf. Communication, Control, Computing, Sept. 2008, pp. 555-561.
39. G.-B. Huang, Q.-Y. Zhu, K. Z. Mao, C.-K. Siew, P. Saratchandran, and N. Sundararajan, "Can threshold networks be trained directly?" IEEE Trans. Circuits Syst. II, vol. 53, no. 3, pp. 187-191, 2006.
40. W. F. Schmidt, M. A. Kraaijveld, and R. P. Duin, "Feedforward neural networks with random weights," in Proc. 11th IAPR Int. Conf. Pattern Recognition Methodology Systems, 1992, pp. 1-4.
41. B. Widrow, A. Greenblatt, Y. Kim, and D. Park, "The no-prop algorithm: A new learning algorithm for multilayer neural networks," Neural Netw., vol. 37, pp. 182-188, Jan. 2013.
42. G.-B. Huang, X. Ding, and H. Zhou, "Optimization method based extreme learning machine for classification," Neurocomputing, vol. 74, pp. 155-163, Dec. 2010.
43. C. R. Rao and S. K. Mitra, Generalized Inverse of Matrices and Its Applications. vol. 7, New York: Wiley, 1971.
44. B. Frénay and M. Verleysen, "Using SVMs with randomised feature spaces: An extreme learning approach," in Proc. 18th European Symp. Artificial Neural Networks, Apr. 28-30, 2010, pp. 315-320.
45. C. Cortes and V. Vapnik, "Support vector networks," Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
46. T. Poggio, S. Mukherjee, R. Rifkin, A. Rakhlin, and A. Verri, "b," A.I. Memo No. 2001-011, CBCL Memo 198, Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 2001.
47. I. Steinwart, D. Hush, and C. Scovel, "Training SVMs without offset," J. Machine Learning Res., vol. 12, pp. 141-202, Feb. 2011.
48. P. L. Bartlett, "The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network," IEEE Trans. Inform. Theory, vol. 44, no. 2, pp. 525-536, 1998.
49. D. H. Hubel and T. N. Wiesel, "Receptive fields and functional architecture of monkey striate cortex," J. Physiol., vol. 195, no. 1, pp. 215-243, 1968.
50. D. Scherer, A. Müller, and S. Behnke, "Evaluation of pooling operations in convolutional architectures for object recognition," in Proc. Int. Conf. Artificial Neural Networks, 2010, pp. 92-101.
51. M. Pal, A. E. Maxwell, and T. A. Warner, "Kernel-based extreme learning machine for remote-sensing image classification," Remote Sensing Lett., vol. 4, no. 9, pp. 853-862, 2013.
52. J. Butcher, D. Verstraeten, B. Schrauwen, C. Day, and P. Haycock, "Reservoir computing and extreme learning machines for non-linear time-series data analysis," Neural Netw., vol. 38, pp. 76-89, Feb. 2013.
53. W. Zheng, Y. Qian, and H. Lu, "Text categorization based on regularization extreme learning machine," Neural Comput. Applicat., vol. 22, nos. 3-4, pp. 447-456, 2013.
54. P. Kara and R. C. Reid, "Efficacy of retinal spikes in driving cortical responses," J. Neurosci., vol. 23, no. 24, pp. 8547-8557, 2003.
55. D. L. Sosulski, M. L. Bloom, T. Cutforth, R. Axel, and S. R. Datta, "Distinct representations of olfactory information in different cortical centres," Nature, vol. 472, no. 7342, pp. 213-216, 2011.
56. O. Barak, M. Rigotti, and S. Fusi, "The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off," J. Neurosci., vol. 33, no. 9, pp. 3844-3856, 2013.
57. Q. V. Le, J. Ngiam, Z. Chen, D. Chia, P. W. Koh, and A. Y. Ng, "Tiled convolutional neural networks," in Proc. Advances Neural Information Processing Systems, 2010, pp. 1279-1287.
58. A. Saxe, P. W. Koh, Z. Chen, M. Bhand, B. Suresh, and A. Y. Ng, "On random weights and unsupervised feature learning," in Proc. Int. Conf. Machine Learning, 2011, pp. 1089-1096.
59. D. Cox and N. Pinto, "Beyond simple features: A large-scale feature search approach to unconstrained face recognition," in Proc. IEEE Int. Conf. Automatic Face Gesture Recognition Workshops, 2011, pp. 8-15.
60. A. Coates, A. Y. Ng, and H. Lee, "An analysis of single-layer networks in unsupervised feature learning," in Proc. Int. Conf. Artificial Intelligence Statistics, 2011, pp. 215-223.
61. D. W. Marquaridt, "Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation," Technometrics, vol. 12, no. 3, pp. 591-612, 1970.
62. Y. LeCun, F. J. Huang, and L. Bottou, "Learning methods for generic object recognition with invariance to pose and lighting," in Proc. Int. Conf. Computer Vision Pattern Recognition, July 2004, vol. 2, pp. II-97-104.
63. V. Nair and G. E. Hinton, "3D object recognition with deep belief nets," in Proc. Advances Neural Information Processing Systems, 2009, pp. 1339-1347.
64. Y. Bengio, "Learning deep architectures for AI," Foundations Trends R Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
65. M. A. Ranzato, C. Poultney, S. Chopra, and Y. LeCun, "Efficient learning of sparse representations with an energy-based model," in Proc. Advances Neural Information Processing Systems, 2006, pp. 1137-1144.
66. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations," in Proc. Int. Conf. Machine Learning, 2009, pp. 609-616.