A comprehensive survey on functional link neural networks and an adaptive PSO-BP learning for CFLNN

Neural Computing and Applications

Volumn 19, Issue 2, 2010, Pages 187-205

  Dehuri, Satchidananda ^a   Cho, Sung Bae ^b  

^a FAKIR MOHAN UNIVERSITY   (India)

^b Yonsei University   (South Korea)

Author keywords

Back propagation learning; Chebyshev functional link neural network; Classification; Functional link neural networks; Particle swarm optimization

Indexed keywords

BACKPROPAGATION LEARNING; CHANNEL EQUALIZATION; CHEBYSHEV FUNCTIONAL LINK NEURAL NETWORKS; COMPREHENSIVE PERFORMANCE; EXTENSIVE SIMULATIONS; FUNCTIONAL LINK NEURAL NETWORK; HIGHER ORDER NEURAL NETWORK; LEARNING SCHEMES;

CLASSIFICATION (OF INFORMATION); DATA MINING; ELECTRIC LOAD FORECASTING; NEURAL NETWORKS; PARTICLE SWARM OPTIMIZATION (PSO); SURVEYS;

BACKPROPAGATION;

EID: 80054727848     PISSN: 09410643     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00521-009-0288-5     Document Type: Article

References (140)

1. Haykin S (1999) Neural networks-a comprehensive foundation. Prentice Hall, Englewood Cliffs
2. Zhang GP (2007) Avoiding pitfalls in neural network research. IEEE Trans Syst Man Cybern Part C Appl Rev 37(1):3-16
3. Anders U, Korn O (1999) Model selection in neural networks. Neural Netw 12:309-323
4. Benitez JM, Castro JL, Requena I (1997) Are artificial neural networks black boxes? IEEE Trans Neural Netw 8(5):1156-1164
5. Cheng B, Titterington D (1994) Neural networks: a review from a statistical perspective. Stat Sci 9(1):2-54
6. McCulloch W, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 7:115-133
7. Giles CL, Maxwell T (1987) Learning invariance, and generalization in a higher order neural networks. Appl Opt 26(23):4972-4978
8. Belli MR, Conti M, Crippa P, Turchetti C (1999) Artificial neural networks as approximators of stochastic processes. Neural Netw 12(4-5):647-658
9. Castro JL, Mantas CJ, Benitez JM (2000) Neural networks with a continuous squashing function in the output are universal approximators. Neural Netw 13(6):561-563
10. Funahashi K (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183-192
11. Andrews R, Diederich J, Tickle AB (1995) Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl Based Syst 8(6):373-389
12. Castro JL, Requena I, Benitez JM (2002) Interpretation of artificial neural networks by means of fuzzy rules. IEEE Trans Neural Netw 13(1):101-116
13. Setiono R, Leow WK, Zurada J (2002) Extraction of rules from artificial neural networks for nonlinear regression. IEEE Trans Neural Network 13(3):564-577
14. Setiono R, Thong JYL (2004) An approach to generate rules from neural networks for regression problems. Eur J Oper Res 155:239-250
15. Gish H (1990) A probabilistic approach to the understanding and training of neural network classifiers. In: Proc IEEE international conference acoustic, speech signal process 3:1361-1364
16. Zhang GP (2000) Neural networks for classification: a survey. IEEE Trans Syst Man Cybern C 30(4):451-462
17. Michie D, Spiegelhalter DJ, Taylor CC (1994) Machine learning, neural and statistical classification. Ellis Horwood, New York
18. Adya M, Collopy F (1998) How effective are neural networks at forecasting and prediction? A review and evaluation. J Forecast 17:481-495
19. Callen JL, Kwan CCY, Yip PCY, Yuan Y (1996) Neural network forecasting of quarterly accounting earnings. Int J Forecast 12:475-482
20. Church KB, Curram SP (1996) Forecasting comsumers' expenditure: a comparison between econometric and neural network models. Int J Forecast 12:255-267
21. Connor JT, Martin RD, Atlas LE (1994) Recurrent neural networks and robust time series prediction. IEEE Trans Neural Netw 51(2):240-254
22. Cottrell M, Girard B, Girard Y, Mangeas M, Muller C (1995) Neural modeling for time series: a statistical stepwise method for weight elimination. IEEE Trans Neural Netw 6(6):1355-1364
23. Faraway JJ, Chatfield C (1998) Time series forecasting with neural networks: a comparative study using the airline data. Appl Stat 47:231-250
24. Fletcher D, Goss E (1993) Forecasting with neural networks-an application using bankruptcy data. Inf Manag 24:159-167
25. Gorr WL (1994) Research prospective on neural network forecasting. Int J Forecast 10:1-4
26. Hippert HS, Pedreira CE, Souza RC (2001) Neural networks for short-term forecasting: a review and evaluation. IEEE Trans Power Syst 16(1):44-55
27. Hu MY, Zhang GP, Jiang CX, Patuwo BE (1999) A crossvalidation analysis of neural network out-of-sample performance in exchange rate forecasting. Decis Sci 30:197-216
28. Kaastra I, Boyd M (1996) Designing a neural network for forecasting financial and economic time series. Neurocomputing 10:215-236
29. Maier HR, Dandy GC (2000) Neural networks for the prediction and forecasting of water resources variables: a review of modeling issues and applications. Environ Model Softw 15:101-124
30. Park YR, Murray TJ, Chen C (1996) Predicting sun spots using a layered perceptron neural network. IEEE Trans Neural Netw 7(2):501-505
31. Qi M, Zhang GP (2001) An investigation of model selection criteria for neural network time series forecasting. Eur J Oper Res 132:666-680
32. Kracha KA, Wagner U (1999) Applications of artificial neural networks in management science: a survey. J Retail Consum Serv 6:185-203
33. Wong BK, Bodnovich TA, Selvi Y (1997) Neural network applications in business: a review and analysis of the literature (1988-1995). Decis Support Syst 19:301-320
34. Flood I, Kartam N (1994) Neural network in civil engineering-I: principles and understanding. J Comput Civil Eng 8(2):131-148
35. Lu CN, Wu HT, Vemuri S (1993) Neural network based short term load forecasting. IEEE Trans Power Syst 8(1):336-342
36. Lisboa PJG (2002) A review of evidence of health benefit from artificial neural networks in medical intervention. Neural Netw 15:11-39
37. Protney LG, Watkins MP (2000) Foundations of clinical research: applications to practice. Prentice-Hall, Princeton
38. Hosseini-Nezhad SM, Yamashita TS, Bielefeld RA, Krug SE, Pao YH (1995) A neural network approach for the determination of interhospital transport mode. Comput Biomed Res 28(4):319-334
39. Tawfik H, Liatsis P (1997) Prediction of non-linear time series using higher order neural networks. In: Proceeding IWSSIP1997 conference, Poznan, Poland
40. Kaita T, Tomita S, Yamanaka J (2002) On a higher order neural network for distortion invariant pattern recognition. Pattern Recognit Lett 23:977-984
41. Ghosh J, Shin Y (1992) Efficient higher-order neural networks for classification and function approximation. Int J Neural Syst 3:323-350
42. Minsky M, Papert S (1969) Perceptrons. The MIT Press
43. Widrow B, Hoff ME (1960) Adaptive switching circuits. IRE WESCON Convention Record, pp 96-104
44. Widrow B, Lehr M (1990) 30 years of adaptive neural networks: perceptron, madaline, and back-propagation. Proc IEEE 78(9):1415-1442
45. Cover TM (1965) Geometrical and statistical properites of systems of linear inequalities with applications in pattern recognition. IEEE Trans Electron Comput 14:326-334
46. Hornik K et al (1989) Multi-layer feed-forward networks are universal approximators. Neural Netw 2:359-366
47. Giles CL, Maxwell T (1987) Learning, invariance and generalization in higher-order neural networks. Appl Opt 26(23):4972-4978
48. Pao YH (1989) Adaptive pattern recognition and neural network. Addison-Wesley, Reading, MA
49. Venkatesh SS, Baldi P (1991) Programmed interactions in higher order neural networks: maximal capacity. J Complex 7:316-337
50. Antyomov E, Pecht OY (2005) Modified higher order neural network for invariant pattern recognition. Pattern Recognit Lett 26:843-851
51. Misra BB, Dehuri S (2007) Functional link neural network for classification task in data mining. J Comput Sci 3(12):948-955
52. Mirea L, Marcu T (2002) System identification using functional link neural networks with dynamic structure. 15th Triennial World Congress, Barcelona, Spain
53. Cass R, Radl B (1996) Adaptive process optimization using functional link networks and evolutionary algorithms. Control Eng Pract 4(11):1579-1584
54. Pao Y-H, Philips SM (1995) The functional link net learning optimal control. Neurocomputing 9:149-164
55. Shin Y, Ghosh J (1995) Ridge polynomial networks. IEEE Trans Neural Netw 6(2):610-622
56. Shin Y, Ghosh J (1992) Approximation of multivariate functions using ridge polynomial networks. In: Proceedings of international joint conference on neural networks II, pp 380-385
57. Voutriaridis C, Boutalis YS, Mertzios G (2003) Ridge polynomial networks in pattern recognition. 4th EURASIP conference focused on video/image processing and multimedia communications, Croatia, pp 519-524
58. Shin Y, Ghosh J (1991) The pi-sigma networks: an efficient higher order neural network for pattern classification and function approximation. In: Proceedings of international joint conference on neural networks I, pp 13-18
59. Shin Y, Ghosh J (1992) Computationally efficient invariant pattern recognition with higher order pi-sigma networks. The University of Texas at Austin, Tech. Report
60. Shin Y, Ghosh J (1991) Realization of boolean functions using binary pi-sigma networks. In: Proceedings of conference on artificial neural networks in engineering, St. Louis
61. Hussain AJ, Liatsis P (2002) Recurrent pi-sigma networks for DPCM image coding. Neurocomputing 55:363-382
62. Xiong Y et al (2007) Training pi-sigma network by on-line gradient algorithm with penalty for small weight update. Neural Comput 19:3356-3368
63. Iyoda EM et al (2007) Image compression and reconstruction using pit-sigma neural networks. Soft Comput 11:53-61
64. Hussain AJ et al (2008) Physical time series prediction using recurrent pi-sigma neural networks. Int J Artif Intell Soft Comput 1(1):130-145
65. Nie Y, Deng W (2008) A hybrid genetic learning algorithm for pi-sigma neural network and the analysis of its convergence. In: Proceedings of fourth international conference on natural computation, IEEE Press, pp 19-23
66. Zhu Q, Cai Y, Liu L (1999) A global learning algorithm for a RBF network. Neural Netw 12:527-540
67. Li M, Tian J, Chen F (2008) Imrpoving multiclass pattern recognition with a co-evolutionary RBFNN. Pattern Recognit Lett 29:392-406
68. Dybowski R (1998) Classification of incomplete feature vectors by radial basis function networks. Pattern Recognit Lett 19:1257-1264
69. Leonardis A, Bischof H (1998) An efficient MDL based construction of RBF networks. Neural Netw 11:963-973
70. Chen S, Wu Y, Luk BL (1999) Combined genetic algorithm optimization and regularized orthogonal least square learning for radial basis function networks. IEEE Tran Neural Netw 10(5):1239-1243
71. Lee YC, Doolen G, Chen HH, Sun GZ, Maxwell T, Lee HY, Giles CL (1986) Machine learning using a higher order correlation network. Physica 22D:276-306
72. Peretto P, Niez JJ (1986) Long-term memory storage capacity of multiconnected neural networks. Biol Cybern 54:5363
73. Psaltis D, Park CH (1986) Nonlinear discriminant functions and associative memories. In: Denker JS (ed) Neural networks for computing. Amererican Institute of Physics, New York, pp 370-375
74. Gardner E (1987) Multiconnected neural-network models. J Phys A Math Gen 20:3453-3464
75. Abbott LF, Arian Y (1987) Storage capacity of generalized networks. Phys Rev A 36:5091-5094
76. Kamp Y, Hasler M (1990) Recursive neural networks for associative memory. Wiley, New York
77. Horn D, Usher M (1988) Capacities of multiconnected memory models. J Phys France 49:389-395
78. Guillermo V (1998) A distributed approach to neural network simulation program. Master thesis, The University of Texas at E1 Paso, TX
79. Zurada JM (1992) Introduction to artificial neural system. West Publishing Company, St. Paul, MN
80. Beale R, Jackson T (1991) Neural computing: an introduction. Hilger, Philadelphia, PA
81. Haring B, Kok JN (1995) Finding functional links for neural networks by evolutionary computation. In: Van de Merckt T et al (eds) BENELEARN1995, proceedings of the fifth Belgian-Dutch conference on machine learning, Brussels, Belgium, pp 71-78
82. Panagiotopoulos DA et al (1999) Planning with a functional neural network architecture. IEEE Trans Neural Netw 10(1):115-127
83. Patra JC et al (1999) Identification of non-linear dynamic systems using functional link artificial neural networks. IEEE IEEE Trans Syst Man Cyber Part B Cybern 29(2):254-262
84. Sierra A, Macias JA, Corbacho F (2001) Evolution of Functional Link Networks. IEEE Tranas Evol Comput 5(1):54-65
85. Marcu T, Koppen-Seliger B (2004) Dynamic functional link neural networks genetically evolved applied to system identification. In: Proceedings of ESANN'2004, Bruges (Belgium), pp 115-120
86. Patra JC, Pal NR (1995) A functional link neural network for adaptive channel equalization. Signal Process 43:181-195
87. Zhao H, Zhang J (2008) Functional link neural network cascaded with Chebyshev orthogonal polynomial for non-linear channel equalization. signal Process 88:1946-1957
88. Haring et al (1997) Feature selection for neural networks through functional links found by evolutionary computation. In: Liu X et al (eds) Adavnces in intelligent data analysis (IDA-97). LNCS 1280:199-210
89. Patra JC et al (2000) Modelling of an intelligent pressure sensor using functional link artificial neural networks. ISA Trans 39:15-27
90. Dehuri S et al (2008) Genetic feature selection for optimal functional link neural network in classification. In: Fyfe C et al (eds) IDEAL 2008, LNCS 5326:156-163
91. Majhi B et al (2005) An improved scheme for digital watermarking using functional link artificial neural network. J Comput Sci 1(2):169-174
92. Patra JC et al (2008) Functional link neural networks-based intelligent sensors for Harsh Environments. Sens Transducers J 90:209-220
93. Dash PK et al (1999) A functional link neural network for short term electric load forecasting. J Intell Fuzzy Syst 7:209-221
94. Krishnaiah D et al (2008) Application of ultrasonic waves coupled with functional link neural network for estimation of carrageenan concentration. Int J Phys Sci 3(4):90-96
95. Sing SN, Srivastava KN (2002) Degree of insecurity estimation in a power system using functional link neural network. ETEP 12(5):353-359
96. Abu-Mahfouz I-A (2005) A comparative study of three artificial neural networks for the detection and classification of gear faults. Int J Gen Syst 34(3):261-277
97. Hu Y-C, Tseng F-M (2007) Functional-link net with fuzzy integral for bankruptcy prediction. Neurocomputing 70:2959-2968
98. Park GH, Pao YH (2000) Unconstrained word-based approach for off-line script recognition using density based random vector functional link net. Neurocomputing 31:45-65
99. Hu Y-C (2008) Functional link nets with genetic algorithm based learning for robust non-linear interval regression analysis. Neurocomputing. doi:10.1016/J.neucom.2008.07.002
100. Chen CLP et al (1998) An incremental adaptive implementation of functional link processing for function approximation, time series prediction, and system identification. Neurocomputing 18:11-31
101. Weng W-D, Yen CT (2004) Reduced decision feed-back FLANN non-linear channel equaliser for digital communication systems. IEE Proc Commun 151(4):305-311
102. Hussain A et al (1997) A new adaptive functional link neural network based DFE for overcoming co-channel interference. IEEE IEEE Trans Commun 45(11):1358-1362
103. Patra JC et al (1999) Non-linear channel equalization for QAM signal constellation using artificial neural networks. IEEE Tranasactions on Systems, Man, Cybernetics-Part B: Cybernetics 29(2):262-271
104. Purwar S et al (2007) On-line system identification of complex systems using Chebyshev neural networks. Appl Soft Comput 7:364-372
105. Weng W-D et al (2007) A channel equalizer usi ng reduced decision feedback Chebyshev function link artificial neural networks. Inf Sci 177:2642-2654
106. Patra JC et al (2002) Non-linear dynamic system identification using Chebyshev functional link artificial neural networks. IEEE Trans Syst Man Cybern Part B Cybern 32(4):505-511
107. Fogel DB (2000) Evolutionary computation: towards a new philosophy of machine intelligence. IEEE Press, New York
108. Pearson DW et al (eds) (1995) Artificial neural networks and genetic algorithms. Springer Verlag
109. Suzuki J (1995) A Markov chain analysis on simple genetic algorithms. IEEE Trans Syst Man Cybern 25(4):6-659
110. Kennedy J, Eberhart RC (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks. Pisacataway, NJ, pp 1942-9148
111. Schaffer JD, Whitley D, Eshelman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Proceedings of international workshop on combinations of genetic algorithms and neural networks pp 1-37
112. Davidor Y (1990) Epistasis variance: suitability of a representation to genetic algorithms. Complex Syst 4:368-383
113. Eshelman LJ, Schaffer JD (1993) Real coded genetic algorithms and interval schemata. In: Whitley LD (ed) Foundation of genetic algorithms. Morgan Kaufmann, San Mateo, pp 187-202
114. Muhlenbein H, Schlierkamp-Voosen D (1993) Predictive models for the breeder genetic algorithm I. Continuous parameters optimization. Evol Comput 1(1):24-49
115. Schutte JF, Groenwold AA (2005) A study of global optimization using particle swarms. J Glob Optim 31(1):93-108
116. Ali MM, Kaelo P (2008) Improved particle swarm algorithms for global optimization. Appl Math Comput 196:578-593
117. Yu J, Wang S, Xi L (2008) Evolving artificial neural networks using an improved PSO and DPSO. Neurocomputing 71:1054-1060
118. Da Y, Ge XR (2005) An improved PSO-based ANN with simulated annealing technique. Neurocomput Lett 63:527-533
119. Blake CL, Merz CJ (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html
120. Pao Y-H, Phillips SM, Sobajic DJ (1992) Neural-net computing and intelligent control systems. Int J Control 56(2):263-289
121. Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw 4:251-257
122. Smith KA, Gupta JND (2002) Neural networks in business: techniques and applications. Idea Group, Hershey, PA
123. Lee TT, Jeng JT (1998) The Chebyshev polynomial based unified model neural networks for function approximations. IEEE Trans Syst Man Cybern Part B 28:925-935
124. Namatame A, Veda N (1992) Pattern classification with Chebyshev neural network. Int J Neural Netw 3:23-31
125. Klasser MS, Pao YH (1988) Characteristics of the functional link net: a higher order delta rule net. IEEE proceedings of 2nd annual international conference on neural networks, San Diago, CA
126. Pao YH, Takefuji Y (1992) Functional link net computing: theory, system, architecture and functionalities. IEEE Comput, pp 76-79
127. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Morgan Kaufmann, San Mateo
128. Kennedy J, Eberhart RC (1999) The particle swarm: social adaptation in information processing systems. In: Corne D, Dorigo M, Glover F (eds) New ideas in optimization. McGraw-Hill, Cambridge, UK, pp 379-387
129. Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of the IEEE international conference on evolutionary computation. IEEE Press, Pisacataway, NJ, pp 69-73
130. Shi Y, Eberhart RC (1998) Parameter selection in particle swarm optimization. Evolutionary Programming VII, LNCS, Springer, Berlin 1447:591-600
131. Forie PC, Groenwold AA (2002) The particle swarm optimization algorithm in size and shape optimization. Struct Multidiscipl Optim 23(4):259-267
132. Clerc M, Kennedy J (2002) The particle swarm explosion, stability and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58-73
133. Zhang JR et al (2007) A hybrid particle swarm optimizationback-propagation algorithm for feed-forward neural network training. Appl Math Comput 185:1026-1037
134. Ratnaweera A, Halgamuge SK, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time varying acceleration coefficients. IEEE Trans Evol Comput 8(3):240-255
135. Lippmann R (1987) An introduction to computing with neural networks. IEEE ASSP Mag 4:4-22
136. Preshelt L (1994) Proben1-a set of neural network benchmark problems and benchmarking rules. Technical Report 21/94, Universitat Karlsruhe, Germany
137. Ghosh A, Dehuri S, Ghosh S (2008) Multi-objective evolutionary algorithms for knowledge discovery from databases. Springer
138. Kriegel H-P et al (2007) Future trends in data mining. Data Mining Knowl Discov 15(1):87-97
139. Vellido A, Lisboa PJG, Vaughan J (1999) Neural networks in business: a survey of applications (1992-1998). Expert Syst Appl 17:51-70
140. Liatsis P, Hussain AJ (1999) Non-linear one dimensional DPCM image prediction using polynomial neural network. In: Proceedings of SPIE applications of artificial neural networks in image processing IV, San Jose, CA 3647:58-68