Density-driven generalized regression neural networks (DD-GRNN) for function approximation

IEEE Transactions on Neural Networks

Volumn 18, Issue 6, 2007, Pages 1683-1696

  Goulermas, John Y ^a   Liatsis, Panos ^b   Zeng, Xiao Jun ^c   Cook, Phil ^d  

^a UNIVERSITY OF LIVERPOOL   (United Kingdom)

^b CITY UNIVERSITY   (United Kingdom)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d Knowledge Support Systems Ltd   (United Kingdom)

Author keywords

Density based; Function approximation; Generalized regression neural network (GRNN); Regularization

Indexed keywords

BAYESIAN NETWORKS; DATA PROCESSING; FEATURE EXTRACTION; REGRESSION ANALYSIS;

FUNCTION APPROXIMATION; GENERALIZED REGRESSION NEURAL NETWORKS;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; COMPUTER ANALYSIS; COMPUTER PROGRAM; COMPUTER SIMULATION; IMAGE PROCESSING; INFORMATION PROCESSING; INFORMATION RETRIEVAL; MATHEMATICAL COMPUTING; NONLINEAR SYSTEM; REGRESSION ANALYSIS; SIGNAL PROCESSING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; BAYES THEOREM; COMPUTER SIMULATION; COMPUTING METHODOLOGIES; IMAGE PROCESSING, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; REGRESSION ANALYSIS; SIGNAL PROCESSING, COMPUTER-ASSISTED; SOFTWARE;

EID: 36349022748     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.902730     Document Type: Article

References (56)

1. C. M. Bishop, Neural Networks For Pattern Recognition. Oxford, U.K.: Oxford Univ. Press, 1996.
2. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning. New York: Springer-Verlag, 2001.
3. S. Haykin, Neural Networks: A Comprehensive Foundation. Upper Saddle River, NJ: Prentice-Hall, 1999.
4. A. Webb, Statistical Pattern Recognition. New York: Wiley, 2002.
5. F. Flentge, "Locally weighted interpolating growing neural gas," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1382-1393, Nov. 2006.
6. N.-Y. Liang, G.-B. Huang, P. Saratchandran, and N. Sundararajan, "A fast and accurate online sequential learning algorithm for feedforward networks," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1411-1423, Nov. 2006.
7. D. Wedge, D. Ingram, D. McLean, C. Mingham, and Z. Bandar, "On global-local artificial neural networks for function approximation," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 942-952, Jul. 2006.
8. J.-M. Wu, Z.-H. Lin, and P.-H. Hsu, "Function approximation using generalized adalines," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 541-558, May 2006.
9. P. Singla, K. Subbarao, and J. L. Junkins, "Direction-dependent learning approach for radial basis function networks," IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 203-222, Jan. 2007.
10. D. F. Specht, "A general regression neural network," IEEE Trans. Neural Netw., vol. 2, no. 6, pp. 568-576, Nov. 1991.
11. H. Schiøler and U. Hartmann, "Mapping neural network derived from the Parzen windowestimator," Neural Netw., vol. 5, pp. 903-909, 1993.
12. J. C. Fu, S. K. Lee, S. T. C.Wong, J. Y. Yeh, A. H.Wang, and H. K.Wu, "Image segmentation, feature selection and pattern classification for mammographic micro classifications," Comput. Med. Imaging Graph., vol. 29, pp. 419-429, 2005.
13. J. Y. Goulermas, D. Howard, C. Nester, R. Jones, and L. Ren, "Regression techniques for the prediction of lower limb kinematics," J. Biomech. Eng., vol. 127, no. 6, pp. 1020-1024, 2005.
14. B. Kim, D. W. Lee, and K. H. Kwon, "Prediction of etch microtrenching using a neural network," J. Appl. Phys., vol. 96, no. 7, pp. 3612-3616, 2004.
15. I. Anagnostopoulos, C. Anagnostopoulos, G. Kouzas, and D. D. Vergados, "A generalized regression algorithm for web page categorization," Neural Comput. Appl., vol. 13, no. 3, pp. 229-236, Sep. 2004.
16. L. Rutkowski, "Generalized regression neural networks in timevarying environment," IEEE Trans. Neural Netw., vol. 15, no. 3, pp. 576-596, May 2004.
17. T. L. Seng, M. Khalid, and R. Yusof, "Adaptive general regression neural network for modeling of dynamic plants," Proc. Inst. Mech. Eng. I, J. Syst. Control Eng., vol. 213, no. I4, pp. 275-287, 1999.
18. T. Hoya, "Notions of intuition and attention modeled by a hierarchically arranged generalized regression neural network," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 1, pp. 200-209, Feb. 2004.
19. D. F. Specht, "Probabilistic neural networks," Neural Netw., vol. 3, pp. 109-118, 1990.
20. G. A. Babich and O. I. Camps, "Weighted Parzen windows for pattern classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 5, pp. 567-570, May 1996.
21. B. Tian, M. R. Azimi-Sadjaji, T. H. Vonder Haar, and D. Reinke, "Temporal updating scheme for probabilistic neural network with application to satellite cloud classification," IEEE Trans. Neural Netw., vol. 11, no. 4, pp. 903-920, Jul. 2000.
22. D. Tomandl and A. Schober, "A modified general regression neural network (MGRNN) with new, efficient training algorithms as a robust 'black box' tool for data analysis," Neural Netw., vol. 14, pp. 1023-1034, 2001.
23. E. W. M. Lee, C. P. Lim, R. K. K. Yuen, and S. M. Lo, "A hybrid neural network model for noisy data regression," IEEE Trans. Syst. Man. Cybern., B, Cybern., vol. 34, no. 2, pp. 951-960, Mar. 2004.
24. J. V. Hansen and R. D. Meservy, "Learning experiments with genetic optimization of a generalized regression neural network," Decision Support Syst., vol. 18, no. 3-4, pp. 317-325, 1996.
25. L. Breiman, W. Meisel, and E. Purcell, "Variable kernel estimates of multivariate densities," Technometrics, vol. 19, no. 2, pp. 135-144, May 1977.
26. E. Parzen, "On the estimation of a probability density function and the mode," Ann. Math. Statist., vol. 33, no. 3, pp. 1065-1076, 1962.
27. T. Cacoullos, "Estimation of a multivariate density," Ann. Inst. Stat. Math., vol. 18, no. 2, pp. 179-189, 1966.
28. L. Xu, A. Krzyzak, and A. Yuille, "On radial basis function nets and kernel regression: Statistical consistency, convergence rates, and receptive field size," Neural Netw., vol. 7, no. 4, pp. 609-628, 1994.
29. J. E. Moody and C. J. Darken, "Fast learning in networks of locally-tuned processing units," Neural Comput., vol. 1, no. 2, pp. 281-294, 1989.
30. V. Tresp, J. Hollatz, and S. Ahmad, "Network structuring and training using rule-based knowledge," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufman, 1993, vol. 5, pp. 871-878.
31. H. M. Kim and J. M. Mendel, "Fuzzy basis functions: Comparisons with other basis functions," IEEE Trans. Fuzzy Syst., vol. 3, no. 2, pp. 158-168, May 1995.
32. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
33. S. Chen, X. Hong, C. J. Harris, and P. M. Sharkey, "Sparse modeling using orthogonal forward regression with PRESS statistic and regularization," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 2, pp. 898-911, Apr. 2004.
34. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes For Machine Learning. Cambridge, MA: MIT Press, 2006.
35. H. R. Madala and A. G. Ivakhnenko, Inductive Learning Algorithms For Complex Systems Modeling. Boca Raton, FL: CRC, 1994.
36. S. S. Rao, Engineering Optimization: Theory and Practice, 3rd ed. New York: Wiley, 1996.
37. X. Hong, P. M. Sharkey, and K.Warwick, "A robust nonlinear identification algorithm using PRESS statistic and forward regression," IEEE Trans. Neural Netw., vol. 14, no. 2, pp. 454-458, Mar. 2003.
38. D. Denison, C. Holmes, B. Malick, and A. Smith, Bayesian Methods For Nonlinear Classification and Regression. New York: Wiley, 2002.
39. R. M. Neal, "Bayesian Learning For Neural Networks," in Lecture Notes in Statistics 118. New York: Springer-Verlag, 1996,.
40. D. J. C. MacKay, "Bayesian interpolation," Neural Comput., vol. 4, no. 3, pp. 415-447, 1992.
41. D. J. C. MacKay, "A practical Bayesian framework for backpropagation network," Neural Comput., vol. 4, no. 3, pp. 448-472, 1992.
42. H. H. Thodberg, "A review of Bayesian neural networks with an application to near infrared spectroscopy," IEEE Trans. Neural Netw. vol. 7, no. 1, pp. 56-72, Jan. 1996.
43. F. D. Foresee and M. T. Hagan, "Gauss-Newton approximation to Bayesian learning," in Proc. IEEE Int. Conf. Neural Netw., 1997, vol. 3, pp. 1930-1935.
44. C. P. Lim and R. F. Harrison, "An incremental adaptive network for on-line supervized learning and probability estimation," Neural Netw., vol. 10, no. 5, pp. 925-939, 1997.
45. P. Mitra, C. A. Murthy, and S. K. Pal, "Density-based multiscale data condensation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 6, pp. 734-747, Jun. 2002.
46. J. Carney and P. Cunningham, "Tuning diversity in bagged ensembles," Int. J. Neural Syst., vol. 10, no. 4, pp. 267-279, 2000.
47. M. Hagan and M. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 989-993, Nov. 1994.
48. J.-S. R. Jang, "ANFIS: Adaptive-network-based fuzzy inference system," IEEE Trans. Syst., Man, Cybern., vol. 23, no. 3, pp. 665-685, May. 1993.
49. I. T. Nabney, Netlab-Algorithms For Pattern Recognition. New York: Springer-Verlag, 2002.
50. L. Yingwei, N. Sundararajan, and P. Saratchandran, "Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm," IEEE Trans. Neural Netw., vol. 9, no. 2, pp. 308-318, Mar. 1998.
51. J. Y. Goulermas, P. Liatsis, and T. Fernando, "A constrained nonlinear energy minimization framework for the regularization of the stereo correspondence problem," IEEE Trans. Circuits Syst. Video Technol. vol. 15, no. 4, pp. 550-565, Apr. 2005.
52. D. Scharstein and R. Szeliski, "High-accuracy stereo depth maps using structured light," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Madison, WI, Jun. 2003, vol. 1, pp. 195-202.
53. R. Shorten and R. Murray-Smith, "Side effects of normalizing radial basis function networks," Int. J. Neural Syst., vol. 7, no. 2, pp. 167-179, 1996.
54. R. M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey, and D. E. Knuth, "On the Lambert w function," Adv. Comput. Math., vol. 5, pp. 329-359, 1996.
55. P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization. London, U.K.: Academic, 1981.
56. J. Nocedal and S. J. Wright, Numerical Optimization. New York: Springer-Verlag, 1999.