Divergence based online learning in vector quantization

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6113 LNAI, Issue PART 1, 2010, Pages 479-486

  Villmann, Thomas ^a   Haase, Sven ^a   Schleif, Frank Michael ^b   Hammer, Barbara ^b  

^a UNIVERSITY OF APPLIED SCIENCES MITTWEIDA   (Germany)

^b CLAUSTHAL UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

divergence based learning; information theory; vector quantization

Indexed keywords

DIVERGENCE BASED LEARNING; GRADIENT DESCENT; LEARNING RULES; ONLINE LEARNING; SQUARED EUCLIDEAN DISTANCE;

ARTIFICIAL INTELLIGENCE; INFORMATION THEORY; SOFT COMPUTING; VECTORS;

VECTOR QUANTIZATION;

EID: 77955381437     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-13208-7_60     Document Type: Conference Paper

References (30)

1. Amari, S.-I.: Differential-Geometrical Methods in Statistics. Springer, Heidelberg (1985)
2. Banerjee, A., Merugu, S., Dhillon, I., Ghosh, J.: Clustering with bregman divergences. Journal of Machine Learning Research 6, 1705-1749 (2005)
3. Bauer, H.-U., Pawelzik, K.R.: Quantifying the neighborhood preservation of Self-Organizing Feature Maps. IEEE Trans. on Neural Networks 3(4), 570-579 (1992)
4. Campbell, J.: Introduction to Remote Sensing. The Guilford Press, U.S.A. (1996)
5. Cichocki, A., Zdunek, R., Phan, A., Amari, S.-I.: Nonnegative Matrix and Tensor Factorizations. Wiley, Chichester (2009)
6. Clark, R.N.: Spectroscopy of rocks and minerals, and principles of spectroscopy. In: Rencz, A. (ed.) Manual of Remote Sensing. John Wiley and Sons, Inc., New York (1999)
7. Cottrell, M., Hammer, B., Hasenfu, A., Villmann, T.: Batch and median neural gas. Neural Networks 19, 762-771 (2006)
8. Csiszr, I.: Information-type measures of differences of probability distributions and indirect observations. Studia Sci. Math. Hungaria 2, 299-318 (1967)
9. Frigyik, B.A., Srivastava, S., Gupta, M.: An introduction to functional derivatives. Technical Report UWEETR-2008-0001, Dept of Electrical Engineering, University of Washington (2008)
10. Fujisawa, H., Eguchi, S.: Robust parameter estimation with a small bias against heavy contamination. Journal of Multivariate Analysis 99, 2053-2081 (2008)
11. Heskes, T.: Energy functions for self-organizing maps. In: Oja, E., Kaski, S. (eds.) Kohonen Maps, pp. 303-316. Elsevier, Amsterdam (1999)
12. Hulle, M.M.V.: Kernel-based topographic map formation achieved with an information theoretic approach. Neural Networks 15, 1029-1039 (2002)
13. Jang, E., Fyfe, C., Ko, H.: Bregman divergences and the self organising map. In: Fyfe, C., Kim, D., Lee, S.-Y., Yin, H. (eds.) IDEAL 2008. LNCS, vol. 5326, pp. 452-458. Springer, Heidelberg (2008)
14. Kantorowitsch, I., Akilow, G.: Funktionalanalysis in normierten Rumen, 2nd edn. Akademie-Verlag, Berlin (1978) (revised edition)
15. Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, vol. 30. Springer, Heidelberg (1995) (Second Extended Edition 1997)
16. Kullback, S., Leibler, R.: On information and sufficiency. Annals of Mathematical Statistics 22, 79-86 (1951)
17. p norm for time series and its application to self-organizing maps. In: Cottrell, M. (ed.) Proc. of Workshop on Self-Organizing Maps, WSOM 2005, Paris, Sorbonne, pp. 733-740 (2005)
18. Lehn-Schiler, T., Hegde, A., Erdogmus, D., Principe, J.: Vector quantization using information theoretic concepts. Natural Computing 4(1), 39-51 (2005) (Pubitemid 40347930)
19. Linde, Y., Buzo, A., Gray, R.: An algorithm for vector quantizer design. IEEE Transactions on Communications 28, 84-95 (1980)
20. Martinetz, T.M., Berkovich, S.G., Schulten, K.J.: 'Neural-gas' network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4(4), 558-569 (1993)
21. Principe, J.C., FisherIII, J.W., Xu, D.: Information theoretic learning. In: Haykin, S. (ed.) Unsupervised Adaptive Filtering, Wiley, New York (2000)
22. Qin, A., Suganthan, P.: A novel kernel prototype-based learning algorithm. In: Proc. of the 17th Internat. Conf. on Pattern Recognition, ICPR 2004, vol. 4, pp. 621-624 (2004)
23. Renyi, A.: On measures of entropy and information. In: Proc. of the 4th Berkeley Symp. on Mathematical Statistics and Probability. Univ. of California Press, Berkeley (1961)
24. Renyi, A.: Probability Theory. North-Holland Publish. Company, Amsterdam (1970)
25. Sato, A., Yamada, K.: Generalized learning vector quantization. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Proc. of the 1995 Conf. on Advances in Neural Information Processing Systems, vol. 8, pp. 423-429. MIT Press, Cambridge (1996)
26. Villmann, T., Claussen, J.-C.: Magnification control in self-organizing maps and neural gas. Neural Computation 18(2), 446-469 (2006) (Pubitemid 43543845)
27. Villmann, T., Haase, S.: Mathematical aspects of divergence based vector quantization using frchet-derivatives - extended and revised version. Machine Learning Reports 4(MLR-01-2010), 1-35 (2010), ISSN:1865-3960, http://www.uni-leipzig.de/~compint/mlr/mlr-01-2010.pdf
28. Villmann, T., Merényi, E., Hammer, B.: Neural maps in remote sensing image analysis. Neural Networks 16(3-4), 389-403 (2003) (Pubitemid 36379090)
29. Villmann, T., Schleif, F.-M.: Functional vector quantization by neural maps. In: Chanussot, J. (ed.) Proceedings of First Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS 2009), pp. 1-4. IEEE Press, Los Alamitos (2009)
30. Zador, P.L.: Asymptotic quantization error of continuous signals and the quantization dimension. IEEE Transaction on Information Theory (28), 149-159 (1982)