A cost-function approach to rival penalized competitive learning (RPCL)

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 36, Issue 4, 2006, Pages 722-737

  Ma, Jinwen ^a   Wang, Taijun ^b  

^a PEKING UNIVERSITY   (China)

^b SOUTHEAST UNIVERSITY   (China)

Author keywords

Clustering analysis; Competitive learning (CL); Convergence; Cost function; Gradient descent

Indexed keywords

COMPUTER SIMULATION; CONVERGENCE OF NUMERICAL METHODS; GRADIENT METHODS; LEARNING ALGORITHMS; OPTIMIZATION;

CLUSTERING ANALYSIS; COST-FUNCTION APPROACH; GRADIENT DESCENT; RIVAL PENALIZED COMPETITIVE LEARNING;

LEARNING SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPETITIVE BEHAVIOR; COMPUTER SIMULATION; GAME; METHODOLOGY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPETITIVE BEHAVIOR; COMPUTER SIMULATION; GAME THEORY; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED;

EID: 33746803160     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2006.870633     Document Type: Article

References (51)

1. P. A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach. London, U.K.: Prentice-Hall, 1982.
2. J. Makhoul, S. Rpucos, and H. Gish, "Vector quantization in speech coding," Proc. IEEE, vol. 73, no. 11, pp. 1551-1558, Nov. 1985.
3. N. Nasrabadi and R. A. King, "Image coding using vector quantization: A review," IEEE Trans. Commun., vol. 36, no. 8, pp. 957-971, Aug. 1988.
4. J. Moody and C. Darken, "Fast learning in networks of locally tuned processing units," Neural Comput., vol. 1, no. 2, pp. 281-294, 1989.
5. T. Poggio and F. Girosi, "Regularization algorithms for learning that are equivalent to multilayer networks," Science, vol. 247, no. 4945, pp. 978-982, 1990.
6. S. J. Nowlan and G. E. Hinton, "Evaluation of adaptive mixtures of competing experts," in Advances in Neural Information Processing System, vol. 3, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 774-780.
7. R. Hecht-Nielsen, Neurocomputing. Reading, MA: Addison-Wesley, 1990.
8. D. E. Rumelhart and D. Zipser, "Feature discovery by competitive learning," Cogn. Sci., vol. 9, no. 1, pp. 75-112, 1985.
9. S. Grossberg, "Competitive learning: From iterative activation to adaptive resonance," Cogn. Sci., vol. 11, no. 1, pp. 23-83, 1987.
10. R. Hecht-Nielsen, "Counterpropagation networks," Appl. Opt., vol. 26, no. 23, pp. 4979-4984, Dec. 1987.
11. E. L. Bienenstock, L. N. Cooper, and P. W. Munro, "Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex," J. Neurosci., vol. 2, no. 1, pp. 32-48, Jan. 1982.
12. D. DeSieno, "Adding a conscience to competitive learning," in Proc. IEEE Int. Conf. Neural Networks, San Diego, CA, 1988, vol. 1, pp. 117-124.
13. S. C. Ahalt, A. K. Krishnamurty, P. Chen, and D. E. Melton, "Competitive learning algorithms for vector quantization," Neural Netw., vol. 3, no. 3, pp. 277-291, 1990.
14. L. Xu, A. Krzyzak, and E. Oja, "Unsupervised and supervised classification by rival penalized competitive learning," in Proc. 11th Int. Conf. Pattern Recognition, The Hague, The Netherlands, Aug. 30-Sep. 3 1992, vol. 1, pp. 672-675.
15. L. Xu, A. Krzyzak, and E. Oja, "Rival penalized competitive learning for clustering analysis, RBF net, and curve detection, IEEE Trans. Neural Netw., vol. 4, no. 4, pp. 636-648, Jul. 1993.
16. J. Mao and A. K. Jain, "A self-organizing network for hyperellipsoidal clustering (HEC)," IEEE Trans. Neural Netw., vol. 7, no. 1, pp. 16-29, Jan. 1996.
17. A. Likas, "A reinforcement learning approach to online clustering," Neural Comput., vol. 11, no. 8, pp. 1915-1932, Nov. 1999.
18. R. Li, E. Sherrod, J. Kim, and G. Pan, "Fast image vector quantization using a modified competitive learning neural network approach," Int. J. Imaging Syst. Technol., vol. 8, no. 4, pp. 413-418, 1997.
19. S. A. Billings and G. L. Zheng, "Radial basis function network configuration using genetic algorithms," Neural Netw., vol. 8, no. 6, pp. 877-890, 1995.
20. A. G. Bors and I. Pitas, "Median radial basis function neural network," IEEE Trans. Neural Netw., vol. 7, no. 6, pp. 1351-1364, Nov. 1996.
21. A. Krzyzak, T. Linder, and G. Lugosi, "Nonparametric estimation and classification using radial basis function nets and empirical risk minimization," IEEE Trans. Neural Netw., vol. 7, no. 2, pp. 475-487, Mar. 1996.
22. P. R. Chang and W. H. Yang, "Environment-adaptation mobile radio propagation prediction using radial basis function neural networks," IEEE Trans. Veh. Technol., vol. 46, no. 1, pp. 155-160, Feb. 1997.
23. A. Roy, S. Govil, and R. Miranda, "A neural-network learning theory and a polynomial time RBF algorithm," IEEE Trans. Neural Netw., vol. 8, no. 6, pp. 1301-1313, Nov. 1997.
24. J. Lee, C. Beach, and N. Tepedelenlioglu, "A practical radial basis function equalizer," IEEE Trans. Neural Netw., vol. 10, no. 2, pp. 450-455, Mar. 1999.
25. A. G. Bors and I. Pitas, "Optical flow estimation and moving object segmentation based on median radial basis function network," IEEE Trans. Image Process., vol. 7, no. 5, pp. 693-702, May 1998.
26. H. Furukawa, T. Ueda, and M. Kitamura, "A systematic method for rational definition of plant diagnostic symptoms by self-organizing neural networks," Neurocomputing, vol. 13, no. 2-4, pp. 171-183, Oct. 1996.
27. Y M. Cheung, W. M. Leung, and L. Xu, "Adaptive rival penalized competitive learning and combined linear predictor model for financial forecast and investment," Int. J. Neural Syst., vol. 18, no. 5/6, pp. 517-534, 1997.
28. Y M. Cheung and L. Xu, "A RPCL-based approach for Markov model identification with unknown state number," IEEE Signal Process. Lett., vol. 7, no. 10, pp. 284-287, Oct. 2000.
29. Y M. Cheung, "Rival penalization controlled competitive learning for data clustering with unknown cluster number," in Proc. 9th ICONIP, Singapore, Nov. 18-22, 2002, vol. 2, pp. 467-471.
30. Z. Y Liu, K. C. Chiu, and L. Xu, "Strip line detection and thinning by RPCL-based local PCA," Pattern Recognit. Lett., vol. 24, no. 14, pp. 2335-2344, Oct. 2003.
31. L. Xu, "YING-YANG machine: A Bayesian-Kullback scheme for unified learning and new results on vector quantization," in Proc. ICONIP, Beijing, China, Oct. 30-Nov. 3, 1995, vol. 2, pp. 977-988. Keynote talk.
32. L. Xu, "Bayesian-Kullback Ying-Yang machine: Reviews and new results," in Proc. ICONIP, Hong Kong, Sep. 24-27, 1996, vol. 1, pp. 59-67.
33. L. Xu, "Rival penalized competitive learning, finite mixture, and multisets clustering," in Proc. IEEE Int. Joint Conf. Neural Networks, Anchorage, AK, May 4-9, 1998, vol. 3, pp. 251-2530.
34. L. Xu, "An overview on unsupervised learning from data mining perspective," in Advances in Self-Organizing Maps. N. Allison et al., Eds. New York: Springer-Verlag, Jun. 2001, pp. 181-210.
35. L. Xu, "Best harmony, unified RPCL and automated model selection for unsupervised and supervised learning on Gaussian mixtures, three-layer nets and ME-RBF-SVM models," Int. J. Neural Syst., vol. 11, no. 1, pp. 3-69, 2001.
36. L. Xu, "BYY harmony learning, structural RPCL, and topological self-organizing on mixture models," Neural Netw., vol. 15, no. 8/9, pp. 1125-1151, Oct./Nov. 2002.
37. D. M. Clark and K. Ravishankar, "A convergence theorem for Grossberg learning," Neural Netw., vol. 3, no. 1, pp. 87-92, 1990.
38. C. M. Kuan and K. Hornik, "Convergence of learning algorithms with constant learning rates," IEEE Trans. Neural Netw., vol. 2, no. 5, pp. 484-489, Sep. 1991.
39. B. Kosko, "Stochastic competitive learning," IEEE Trans. Neural Netw., vol. 2, no. 5, pp. 522-529, Sep. 1991.
40. H. Yang and T. S. Dillon, "Convergence of self-organizing algorithms," Neural Netw., vol. 5, no. 3, pp. 485-493, 1992.
41. A. S. Galanopoulos, R. L. Moses, and S. C. Ahalt, "Diffusion approximation of frequency sensitive competitive learning," IEEE Trans. Neural Netw., vol. 8, no. 5, pp. 1026-1030, Sep. 1997.
42. Y M. Cheung, "Maximum weighted likelihood via rival penalized EM for density mixture clustering with automated model selection," IEEE Trans. Knowl. Data Eng., vol. 17, no. 6, pp. 750-761, Jun. 2005.
43. H. Robbins and S. Monro, "A stochastic approximation method," Ann. Math. Stat., vol. 22, no. 3, pp. 400-407, Sep. 1951.
44. S. Y Kung, Digital Neural Networks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
45. V. A. Zorich, Mathematical Analysis I. Berlin, Germany: Springer-Verlag, 2004.
46. L. Ljung, "Strong convergence of a stochastic approximation algorithm," Ann. Stat., vol. 6, no. 3, pp. 680-696, 1978.
47. D. E. Goldberg, Genetic Algorithms in Search, Optimization & Machine Learning. Reading, MA: Addison-Wesley, 1989.
48. R. Storn and K. Price, "Differential evolution - A simple and efficient heuristic for global optimization over continuous spaces," J. Glob. Optim., vol. 11, no. 4, pp. 341-359, Dec. 1997.
49. N. Boujemaa, "Generalized competitive clustering for image segmentation," in Proc. 19th Int. Conf. North American Fuzzy Information Processing Society, Atlanta, GA, 2000, pp. 133-137.
50. C. L. Blake and C. J. Merz, (1998), UCI Repository of Machine Learning Databases, Irvine: Dept. Inf. Comput. Sci., Univ. California. [Online]. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
51. S. J. Roberts, R. Everson, and I. Rezek, "Maximum certainty data partitioning," Pattern Recognit., vol. 33, no. 5, pp. 833-839, May 2000.