Speeding up the scaled conjugate gradient algorithm and its application in neuro-fuzzy classifier training

Soft Computing

Volumn 14, Issue 4, 2010, Pages 365-378

  Cetişli, Bayram ^a   Barkana, Atalay ^b  

^a SULEYMAN DEMIREL UNIVERSITY   (Turkey)

^b ANADOLU UNIVERSITY   (Turkey)

Author keywords

Gradient estimation; Large scale problems; Neural network; Neuro fuzzy classifier; Speeding up learning; The scaled conjugate gradient algorithm

Indexed keywords

GRADIENT ESTIMATION; LARGE-SCALE PROBLEMS; NEURO-FUZZY CLASSIFIER; SPEEDING UP LEARNING; THE SCALED CONJUGATE GRADIENT ALGORITHM;

CLASSIFIERS; CONJUGATE GRADIENT METHOD; CONVERGENCE OF NUMERICAL METHODS; ESTIMATION; FUZZY SYSTEMS; LEARNING SYSTEMS; NEURAL NETWORKS;

LEARNING ALGORITHMS;

EID: 70350534585     PISSN: 14327643     EISSN: 14337479     Source Type: Journal    
DOI: 10.1007/s00500-009-0410-8     Document Type: Article

References (36)

1. Abraham A (2004) Meta learning evolutionary artificial neural networks. Neurocomputing 56:1-38. doi: 10.1016/S0925-2312(03)00369-2
2. Bazaraa MS, Sherali HD, Shetty CM (2006) Nonlinear programming, 3rd edn. Wiley, New York
3. Bishop CM (1996) Neural networks for pattern recognition. Oxford University Press, New York
4. Broyden CG (1967) Quasi-Newton methods and their applications to function minimization. Math Comput 21:368-381. doi: 10.2307/2003239
5. Castillo E, Guijarro-Berdiñas B, Fontenla-Romero O, Alonso-Betanzos A (2006) A very fast learning method for neural networks based on sensitivity analysis. J Mach Learn Res 7:1159-1182
6. Chuang CC, Jeng JT (2007) CPBUM neural networks for modeling with outliers and noise. Appl Soft Comput 7:957-967. doi: 10.1016/ j.asoc.2006.04.009
7. Demuth H, Beale M, Hagan M (2008) Neural network toolbox 6 user's guide. Mathworks Inc, Natick
8. Edmonson W, Principe J, Srinivasan K, Wang C (1998) A global least mean square algorithm for adaptive IIR filtering. IEEE trans. on circuits and systems-II. Analog Digit Signal Process 45(3):379-384. doi: 10.1109/ 82.664244
9. Haykin S (2001) Kalman filtering and neural networks. Wiley, New York
10. Jang JSR (1991) Fuzzy modelling using generalized neural networks and Kalman filter algorithm. In: Proceedings of the ninth national conference on artificial intelligence (AAAI-91), pp 762-767
11. Jang JSR (1993) ANFIS: Adaptive network based fuzzy inference systems. IEEE Trans Syst Man Cybern 23:665-685. doi: 10.1109/21.256541
12. Jang JSR, Mizutani E (1996) Levenberg-Marquardt method for ANFIS learning. In: Proceedings of the international joint conference of the north American fuzzy information processing society biannual conference, Berkeley, pp 87-91
13. Jang JSR, Sun CT, Mizutani E (1997) Neuro-fuzzy and soft computing. Prentice Hall, Upper Saddle River
14. Kashiyama K, Tamai T, Inomata W, Yamaguchi S (2000) A parallel finite element method for incompressible Navier-Stokes flows based on unstructured grids. Comput Methods Appl Mech Eng 190(3-4):333-344
15. Keles A, Hasiloglu AS, Keles A, Aksoy Y (2007) Neuro-fuzzy classification of prostate cancer using NEFCLASS-J. Comput Biol Med 37:1617-1628. doi: 10.1016/j.compbiomed.2007.03.006
16. Le Cun Y, Galland C, Hinton GE (1989) GEMINI: Gradient estimation through matrix inversion after noise injection. In: Touretzky D (ed) Advances in neural information processing systems 1 (NIPS - 88). Morgan Kaufman, Denver
17. Le Cun Y, Kanter I, Solla SA (1991) Eigenvalues of covariance matrices: application to neural network learning. Phys Rev Lett 66(18):2396-2399. doi: 10.1103/PhysRevLett.66.2396
18. Levenberg K (1944) A method for the solution of certain problems in least squares. Q Appl Math 2:164-168
19. Marquardt DW (1963) An algorithm for least squares estimation of nonlinear parameters. J Soc Ind Appl Math 11:431-441. doi: 10.1137/ 0111030
20. Moghaddam HA, Matinfar M (2007) Fast adaptive LDA using quasi-Newton algorithm. Pattern Recognit Lett 28:613-621. doi: 10.1016/ j.patrec.2006.10.011
21. Møller M (1993) A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw 6(4):525-533. doi: 10.1016/ S0893-6080(05)80056-5
22. Møller M (1997) Efficient training of feed-forward neural networks. Ph.D. Thesis, Aarhus University, Denmark
23. Mukkamala S, Sung AH, Abraham A (2005) Intrusion detection using an ensemble of intelligent paradigms. J Netw Comput Appl 28(2):167-182. doi: 10.1016/j.jnca.2004.01.003
24. Ribeiro MV, Duque CA, Romano JMT (2006) An interconnected type-1 fuzzy algorithm for impulsive noise cancellation in multicarrier-based power line communication systems. IEEE J Sel Areas Communitications 24(7):1364-1376. doi: 10.1109/JSAC.2006.874417
25. Schraudolph NN (2002) Fast curvature matrix-vector products for second-order gradient descent. Neural Comput 14:1723-1738. doi: 10.1162/ 08997660260028683
26. Shanno DF (1970) Conditioning of quasi-Newton methods for function minimization. Math Comput 24:647-656. doi: 10.2307/2004840
27. Sinha SK, Fieguth PW (2006) Neuro-fuzzy network for the classification of buried pipe defects. Autom Constr 15:73-83. doi: 10.1016/ j.autcon.2005.02.005
28. Sözen A, Arcakliog7caron;lu E, Özalp M, Kanit EG (2005) Solar-energy potential in Turkey. Appl Energy 8(4):367-381. doi: 10.1016/ j.apenergy.2004.06.001
29. Steil JJ (2006) Online stability of backpropagation-decorrelation recurrent learning. Neurocomputing 69:642-650. doi: 10.1016/ j.neucom.2005.12.012
30. Sun CT, Jang JSR (1993) A neuro-fuzzy classifier and its applications. In: Proceedings of IEEE international conference on fuzzy systems, San Francisco, vol 1, pp 94-98
31. Theoridis S, Koutroumbas K (2003) Pattern recognition, 2nd edn. Academic Press, London
32. Thomas GB, Finney RL (1995) Calculus and analytic geometry, 9th edn. Addison-Wesley, Reading
33. Toosi NA, Kahani M (2007) A new approach to intrusion detection based on an evolutionary soft computing model using neuro-fuzzy classifiers. Comput Commun 30:2201-2212. doi: 10.1016/j.comcom.2007.05.002
34. Tran C, Abraham A, Jain L (2004) Decision support systems using hybrid neurocomputing. Neurocomputing 61:85-97. doi: 10.1016/ j.neucom.2004.03.006
35. Wang C, Principe J (1999) Training neural networks with additive noise in the desired signal. IEEE Trans Neural Netw 10(6):1511-1517. doi: 10.1109/72.809097
36. Zhang P, Bui TD, Suen CY (2007) A novel cascade ensemble classifier system with a high recognition performance on handwritten digits. Pattern Recognit 40:3415-3429. doi: 10.1016/j.patcog.2007.03.022