Modified Recursive Least Squares algorithm to train the Hybrid Multilayered Perceptron (HMLP) network

Applied Soft Computing Journal

Volumn 10, Issue 1, 2010, Pages 236-244

  Al Batah, Mohammad Subhi ^a   Mat Isa, Nor Ashidi ^a   Zamli, Kamal Zuhairi ^a   Azizli, Khairun Azizi ^b  

^a SCHOOL OF CHEMICAL ENGINEERING   (Malaysia)

^b SCHOOL OF MATERIALS AND MINERAL RESOURCES ENGINEERING   (Malaysia)

Author keywords

Artificial Neural Network; Hybrid Multilayered Perceptron; Modified Recursive Least Square; Multilayered Perceptron; Pattern recognition

Indexed keywords

ARTIFICIAL NEURAL NETWORK; BENCHMARK DATA; CLASSIFICATION PERFORMANCE; COMPARISON RESULT; CONVERGENCE RATES; FORGETTING FACTORS; HYBRID MULTILAYERED PERCEPTRON; LEARNING RATES; LINEAR CONNECTIONS; MACHINE-LEARNING; MODIFIED RECURSIVE LEAST SQUARE; MULTI-LAYERED; MULTILAYERED PERCEPTRON; PERCEPTRON; RECURSIVE LEAST SQUARE ALGORITHMS; RECURSIVE LEAST SQUARES; RECURSIVE LEAST SQUARES ALGORITHMS; RECURSIVE PREDICTION; RLS ALGORITHMS; UCI REPOSITORY;

BACKPROPAGATION; CLASSIFIERS; CONVERGENCE OF NUMERICAL METHODS; EDUCATION; MEAN SQUARE ERROR; NEURAL NETWORKS; PATTERN RECOGNITION; RECURSIVE FUNCTIONS; ROBOT LEARNING;

LEARNING ALGORITHMS;

EID: 70350087544     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2009.06.018     Document Type: Article

References (53)

1. A. Hoang, Supervised classifier performance on the UCI database, Master Thesis, Department of Computer Science, University of Adelaide, Australia, 1997.
2. Fayyad U. The KDD process for extracting useful knowledge from volumes of data. Communication of the ACM 39 (1996) 27-34
3. Takagi H. Introduction to fuzzy systems neural networks and genetic algorithms. In: Ruan D. (Ed). Intelligent Systems: Fuzzy Logic, Neural Networks and Genetic Algorithms (1997), Kluwer Academic Publishers, USA 1-33
4. Mukhopadhyay S., Tang C., Huang J., Yu M., and Palakal M. A comparative study of genetic sequence classification algorithms. Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, Neural Networks for Signal Processing (2002) 57-66
5. Wang S. Classification with incomplete survey data: a Hopfield neural network approach. Computers and Operations Research 32 10 (2005) 2583-2594
6. Bauer M., Buchtala O., Horeis T., Kern R., Sick B., and Wagner R. Technical data mining with evolutionary radial basis function classifiers. Applied Soft Computing 9 2 (2009) 765-774
7. Shirvany Y., Hayati M., and Moradian R. Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations. Applied Soft Computing 9 1 (2009) 20-29
8. Subudhi B., and Morris A.S. Soft computing methods applied to the control of a flexible robot manipulator. Applied Soft Computing 9 1 (2009) 149-158
9. Mandal S., Sivaprasad P.V., Venugopal S., and Murthy K.P.N. Artificial neural network modeling to evaluate and predict the deformation behavior of stainless steel type AISI 304L during hot torsion. Applied Soft Computing 9 1 (2009) 237-244
10. Yu L., Wang S., and Lai K.K. A neural-network-based nonlinear metamodeling approach to financial time series forecasting. Applied Soft Computing 9 2 (2009) 563-574
11. Obot O.U., and Uzoka F.-M.E. A framework for application of neuro-case-rule base hybridization in medical diagnosis (2009) 245-253
12. Li T., Luo Q., Sun C., and Zhang B. Exponential stability of recurrent neural networks with time-varying discrete and distributed delays. Nonlinear Analysis: Real World Applications 10 4 (2009) 2581-2589
13. Mat-Isa N.A., Mashor M.Y., and Othman N.H. An automated cervical pre-cancerous diagnostic system. Artificial Intelligence in Medicine 42 1 (2008) 1-11
14. Mat-Isa N.A., Al-Batah M.S., Zamli K.Z., Azizli K.A., Joret A., and Mat-Noor N.R. Suitable features selection for the HMLP and MLP networks to identify the shape of aggregate. Journal of Construction and Building Materials 22 3 (2008) 402-410
15. Al-Batah M.S., Mat-Isa N.A., Zamli K.Z., Md-Sani Z., and Azizli K.A. A novel aggregate classification technique using moment invariants and cascaded multilayered perceptron network. International Journal of Mineral Processing, Elsevier 92 1-2 (2009) 92-102
16. Mashor M.Y. Performance comparison between HMLP and MLP NETWORKS. Proceeding of the International Conference on Robotics, Vision and Parallel Processing for Automation (ROVPIA'99) (1999) 123-132
17. Geiger H. Storing and processing information in connectionist systems. In: Eckmiller R. (Ed). Advanced Neural Computers (1990), North-Holland, Amsterdam 271-277
18. Poggio T., and Girosi F. Networks for approximation and learning. Proceedings of the IEEE 78 9 (1990) 1481-1497
19. Sarle W.S. Neural networks and statistical models. Proceedings of the 19th Annual SAS Users Group International Conference. SAS Institute Inc., Cary, NC (1994) 1538-1550
20. M.Y. Mashor, System identification using radial basis function network, Ph.D. Thesis, University of Sheffield, UK, 1995.
21. Billings S.A., and Fung C.F. Recurrent radial basis function networks for adaptive noise cancellation. Neural Networks 8 2 (1995) 273-290
22. Billings S.A., Jamaluddin H.B., and Chen S. A comparison of the back propagation and recursive prediction error algorithms for training neural networks. Mechanical System and Signal Processing 5 3 (1991) 233-255
23. Billings S.A., Jamaluddin H.B., and Chen S. Properties of neural networks with applications to modeling non-linear dynamical systems. International Journal of Control 55 1 (1992) 193-224
24. Mashor M.Y. Hybrid multilayered perceptron networks. International Journal of Systems Science 31 6 (2000) 771-785
25. Mashor M.Y., Osman M.K., and Arshad M.R. 3D object recognition using 2D moments and HMLP network. Proceeding of the IEEE International Conference on Computer Graphics, Imaging and Visualization (2004) 126-130
26. Mat-Isa N.A., Mashor M.Y., and Othman N.H. Classification of cervical cancer cells using HMLP network with confidence percentage and confidence level analysis. International Journal of the Computer, the Internet and Management 11 1 (2004) 17-29
27. Mat-Sakim H.A., Mat-Isa N.A., Naguib R.N.G., and Sherbet G.V. Image cytometry data from breast lesions analyzed using hybrid networks. Proceedings of the 27th Annual International Conference of the Engineering in Medicine and Biology Society (IEEE-EMBS 2005) (2005) 2059-2062
28. Alhady S.S.N., Arshad M.R., and Mashor M.Y. Suitable features selection for HMLP network to identify R-wave in the ECG signal. Proceedings of the International Conference on Robotics, Vision, Information and Signal Processing (ROVISP 2005) (2005) 871-875
29. Haykin S. Neural Networks: A Comprehensive Foundation. 2nd edition (1998), Prentice Hall
30. Farhang-Boroujeny B. Adaptive Filters: Theory and Applications (1998), Wiley, New York
31. Leung S.H. Gradient-based variable forgetting factor RLS algorithm in time varying environments. IEEE Transactions on Signal Processing 53 8 (2005) 3141-3150
32. Lan H.Y., Yang S.Y., Tan F., and Zhao L.Q. Gradient-based variable forgetting factor RLS algorithm in MIMO-OFDM channel estimation. International Conference on Wireless Communications, Networking and Mobile Computing, WinCom2007 (2007) 468-471
33. Song S., Lim J.S., Baek S.J., and Sung K.M. Gauss Newton variable forgetting factor recursive least squares for time varying parameter tracking. Electronics Letters 36 11 (2000) 988-990
34. Azimi-Sadjadi M.R., and Citrin S. Fast learning process of multilayer neural nets using recursive least squares technique. Proceedings of the IEEE International Conference Neural Networks. Washington, DC (1989)
35. Azimi-Sadjadi M.R., Citrin S., and Sheedvash S. Supervised learning process of multilayer perceptron neural networks using fast recursive least squares. Proceedings of the IEEE International Conference Acoustic, Speech, Signal Processing (ICASSP'90). New Mexico (1990) 1381-1384
36. Azimi-Sadjadi M.R., and Liou R.J. Fast learning process of multilayer neural networks using recursive least squares method. IEEE Transaction on Signal Processing 40 2 (1992) 446-450
37. Bilski J., and Rutkowski L. A fast training algorithm for neural networks. IEEE Transaction on Circuits Systems II 45 6 (1998) 749-753
38. Mahmood K., and Zerguine A. Neural network-based decision feedback equalizer using a recursive least squares algorithm. Proceedings of the 8th International Symposium on Signal Processing and Its Applications, vol. 1 (2005) 82-85
39. Mahmood K., Zidouri A., and Zerguine A. Performance analysis of a RLS based MLP-DFE in time-invariant and time-varying channels. Digital Signal Processing 18 3 (2008) 307-320
40. D.J. Newman, S. Hettich, C.L. Blake, C.J. Merz, D.W. Aha, UCI Repository of Machine Learning Databases, Department of Information and Computer Science, University of California, Irvine, 1998 of Conference, http://archive.ics.uci.edu/ml/datasets.html.
41. P. Eklund, A. Hoang, A performance survey of public domain machine learning algorithms, Technical Report, School of Information Technology, Griffith University, 2002.
42. Eklund P., and Hoang A. A comparative study of public supervised classifier performance on the UCI database. Australian Journal of Intelligent Information Processing Systems 9 1 (2006) 1-39
43. Datasets Used for Classification: Comparison of Results, Nicolaus Copernicus University, Department of Informatics, 2008, http://www.fizyka.umk.pl/kmk/projects/datasets.html.
44. Cybenko G. Approximations by superposition of a sigmoidal function, mathematics of control. Signal and Systems 2 (1989) 303-314
45. Funahashi K. On the approximation realization on continuous mappings by neural networks. Neural Networks 2 (1989) 183-192
46. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning Internal Representation by Error Propagation Parallel Distributed Processing: Explorations in the Microstructures of Cognition vol. 1 (1986), MIT Press pp. 318-362
47. Rumelhart D.E., Hinton G.E., and Williams R.J. Learning representation by back-propagation errors. Nature (London) 323 (1986) 533-536
48. Haykin S. Adaptive Filter Theory. 3rd edition (1996), Prentice-Hall, Englewood Cliffs, NJ
49. R.A. Jacobs, Increased rates of convergence through learning rate adaptation, vol. 1, Technical Report in Neural Networks, University of Massachusetts, USA, 1988, pp. 295-307.
50. Duch W., Adamczak R., and Grabczewski K. A new methodology of extraction, optimization and application of crisp and fuzzy logical rules. IEEE Transactions on Neural Networks 12 2 (2001) 277-306
51. Domeniconi C., Peng J., and Gunopulos D. An adaptive metric for pattern classification. Advances in Neural Information Processing Systems vol. 13 (2000), MIT Press, Cambridge, MA
52. Michie D., Spiegelhalter D.J., and Taylor C.C. Machine Learning Neural and Statistical Classification (1994), Elis Horwood, London
53. S. Aeberhard, D. Coomans, O. Vel, Comparison of classifiers in high dimensional settings, Technical Report Number 92-02, Department of Computer Science and Department of Mathematics and Statistics, James Cook University of North Queensland, 1992.