SOMKE: Kernel density estimation over data streams by sequences of self-organizing maps

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 8, 2012, Pages 1254-1268

  Cao, Yuan ^a   He, Haibo ^b   Man, Hong ^c  

^a MATHWORKS INC   (United States)

^b University of Rhode Island   (United States)

^c Stevens Institute of Technology   (United States)

Author keywords

Kernel density estimation; Kullback Leibler divergence; probability density functions; self organized maps; stream data mining

Indexed keywords

CLUSTERING INFORMATION; EFFECTIVENESS AND EFFICIENCIES; KERNEL DENSITY ESTIMATION; KULLBACK LEIBLER DIVERGENCE; SELF-ORGANIZED MAPS; SELF-ORGANIZING MAP (SOM); STREAM DATA MINING; TIME AND SPACE COMPLEXITY;

CONFORMAL MAPPING; DATA COMMUNICATION SYSTEMS; PROBABILITY DENSITY FUNCTION; PROBABILITY DISTRIBUTIONS; STATISTICS;

DATA MINING;

EID: 84876922286     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2201167     Document Type: Article

References (66)

1. P. Domingos and G. Hulten, "A general framework for mining massive data stream," J. Comput. Graphical Statist., vol. 12, no. 4, pp. 945-949, 2003.
2. C. C. Aggarwal, J. Han, J. Wang, and P. S. Yu, "A framework for ondemand classification of evolving data streams," IEEE Trans. Knowl. Data Eng., vol. 18, no. 5, pp. 577-589, May 2006.
3. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom, "Models and issues in data stream systems," in Proc. 21st ACM Symp. Principles Database Syst., 2002, pp. 1-16.
4. P. Domingos and G. Hulten, "Mining high-speed data streams," in Proc. 6th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2000, pp. 71-80.
5. R. Elwell and R. Polikar, "Incremental learning of concept drift in nonstationary environments," IEEE Trans. Neural Netw., vol. 22, no. 10, pp. 1517-1531, Oct. 2011.
6. S. Guha, N. Mishra, R. Motwani, and L. O'Callahan, "Clustering data streams," in Proc. Foundations Comput. Sci. 41st Annu. Symp., 2000, pp. 359-366.
7. A. A. Lazar and E. A. Pnevatikakis, "Video time encoding machines," IEEE Trans. Neural Netw., vol. 22, no. 3, pp. 461-473, Mar. 2011.
8. P. Rogister, R. Benosman, S.-H. Ieng, P. Lichtsteiner, and T. Delbruck, "Asynchronous event-based binocular stereo matching," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 2, pp. 347-353, Feb. 2012.
9. B. W. Silverman, Density Estimation for Statistics and Data Analysis. London, U.K.: Chapman & Hall, 1986.
10. J. DiNardo and J. L. Tobias, "Nonparametric density and regression estimation," J. Economic Perspectives, vol. 15, no. 4, pp. 11-28, 2001.
11. Y. Cao, H. He, H. Man, and X. Shen, "Integration of self-organizing map (som) and kernel density estimation (kde) for network intrusion detection," Proc. SPIE, vol. 7480, pp. 74800N-1-74800N-12, Sep. 2009.
12. T. Brox, B. Rosenhahn, D. Cremers, and H. P. Seidel, "Nonparametric density estimation with adaptive, anisotropic kernels for human motion tracking," in Proc. 2nd Conf. Human Motion Underst. Model. Capture Animation, 2007, pp. 152-165.
13. T. Bouezmarni and J. V. K. Rombouts, "Nonparametric density estimation for multivariate bounded data," J. Statist. Plann. Inference, vol. 140, no. 1, pp. 139-152, 2010.
14. Ö. Egecioglu and A. Srinivasan, "Efficient nonparametric density estimation on the sphere with applications in fluid mechanics," SIAM J. Scientific Comput., vol. 22, no. 1, pp. 152-176, 2000.
15. W. L. Martinez and A. R. Martinez, Computational Statistics Handbook with MATLAB, 2nd ed. London, U.K.: Chapman & Hall, 2008.
16. C. Heinz and B. Seeger, "Cluster kernels: Resource-aware kernel density estimators over streaming data," IEEE Trans. Knowl. Data Eng., vol. 20, no. 7, pp. 880-893, Jul. 2008.
17. A. Hämäläinen, "Self-organizing map and reduced kernel density estimation," Ph.D. thesis, Rolf Nevanlinna Inst., Univ. Jyväskylä, Jyväskylä, Finland, 1995.
18. H. Yin and N. M. Allinson, "Self-organizing mixture networks for probability density estimation," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 405-411, Mar. 2001.
19. A. Elgammal, R. Duraiswami, and L. S. Davis, "Efficient kernel density estimation using the fast gauss transform with applications to color modeling and tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 11, pp. 1499-1504, Nov. 2003.
20. C. Lambert, S. Harrington, C. Harvey, and A. Glodjo, "Efficient on-line nonparametric kernel density estimation," Algorithmica, vol. 25, no. 1, pp. 37-57, 1999.
21. C. M. Procopiuc and O. Procopiuc, "Density estimation for spatial data streams," in Proc. 9th Int. Conf. Adv. Spatial Temporal Databases, 2005, pp. 109-126.
22. K. A. Caudle and E. Wegman, "Nonparametric density estimation of streaming data using orthogonal series," Comput. Statist. Data Anal., vol. 53, no. 12, pp. 3980-3986, 2009.
23. Z. Cai, W. Qian, L. Wei, and A. Zhou, "M-kernel merging: Toward density estimation over data streams," in Proc. Database Syst. Adv. Appl. 18th Int. Conf., 2003, pp. 285-292.
24. C. Heinz and B. Seeger, "Toward kernel density estimation over streaming Data," in Proc. Int. Conf. Manage. Data, 2006, pp. 1-12.
25. T. Kohonen, "The self-organizing map," Proc. IEEE, vol. 78, no. 9, pp. 1464-1480, Sep. 1990.
26. T. Kohonen, "The self-organizing map," Neurocomputing. New York: Elsevier, 1998.
27. T. Kohonen, Self-Organizing Map. New York: Springer-Verlag, 2001.
28. G. G. Yen and Z. Wu, "Ranked centroid projection: A data visualization approach with self-organizing maps," IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 245-259, Feb. 2008.
29. E. López-Rubio, J. M. Ortiz-de-Lazcano-Lobato, and D. López-Rodríguez, "Probabilistic PCA self-organizing maps," IEEE Trans. Neural Netw., vol. 20, no. 9, pp. 1474-1489, Sep. 2009.
30. A. Hirose and T. Nagashima, "Predictive self-organizing map for vector quantization of migratory signals and its application to mobile communications," IEEE Trans. Neural Netw., vol. 14, no. 6, pp. 1532-1540, Nov. 2003.
31. S. Cheng, H. Fu, and H. Wang, "Model-based clustering by probabilistic self-organizing maps," IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 805-826, Mar. 2009.
32. G. J. Chappell and J. G. Taylor, "The temporal kohonen map," Neural Netw., vol. 6, no. 3, pp. 441-445, 1993.
33. T. Koskela, M. Varsta, J. Heikkonen, and K. Kaski, "Temporal sequence processing using recurrent SOM," in Proc. Knowl.-Based Intell. Electron. Syst. Int. Conf., 1998, pp. 290-297.
34. T. Voegtlin, "Recursive self-organizing maps," Neural Netw., vol. 15, nos. 8-9, pp. 979-991, 2002.
35. A. Andreakis, N. V. Hoyningen-Huene, and M. Beetz, "Incremental unsupervised time series analysis using merge growing neural gas," in Advances in Self-Organizing Maps (Lecture Notes on Computer Sciences), vol. 5629, J. C. Principe and R. Miikkulainen, Eds. Heidelberg, Germany: Springer-Verlag, 2009, pp. 10-18.
36. H. Ni and H. Yin, "Self-organising mixture autoregressive model for non-stationary time series modelling," Int. J. Neural Syst., vol. 18, no. 6, pp. 469-480, 2008.
37. H. Yin and H. Ni, "Generalized self-organizing mixture autoregressive model," in Proc. Lecture Notes Comput. Sci. Adv. Self-Organ. Maps, 2009, pp. 353-361.
38. N. Alex, A. Hasenfuss, and B. Hammer, "Patch clustering for massive data sets," Neurocomputing, vol. 72, nos. 7-9l, pp. 1455-1469, 2009.
39. A. Georgakis, H. Li, and M. Gordan, "An ensemble of som networks for document organization and retrieval," in Proc. Int. Conf. Adaptive Knowl. Represent. Reasoning, 2005, pp. 6-12.
40. C. Saavedra, R. Salas, S. Moreno, and H. Allende, "Fusion of self organizing maps," in Proc. 9th Int. Work-Conf. Artificial Neural Netw. Lecture Notes Comput. Sci., 2007, pp. 227-234.
41. B. Baruque and E. Corchado, "A weighted voting summarization of SOM ensembles," Data Mining Knowl. Discovery, vol. 21, no. 3, pp. 398-426, 2010.
42. A. Rauber, D. Merkl, and M. Dittenbach, "The growing hierarchical selforganizing map: Exploratory analysis of high-dimensional data," IEEE Trans. Neural Netw., vol. 13, no. 6, pp. 1331-1341, Nov. 2002.
43. H. He, S. Chen, K. Li, and X. Xu, "Incremental learning from stream data," IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 1901-1914, Dec. 2011.
44. M. C. Jones, J. S. Marron, and S. J. Sheather, "A brief survey of bandwidth selection for density estimation," J. Amer. Statist. Associat., vol. 91, no. 433, pp. 401-407, 1996.
45. V. C. Raykar and R. Duraiswami, "Fast optimal bandwidth selection for kernel density estimation," in Proc. 6th SIAM Int. Conf. Data Mining, 2006, pp. 524-528.
46. D. W. Scott and S. R. Rain, "Multi-dimensional density estimation," in Handbook of Statistics Data Mining and Computational Statistics, C. R. Rao and E. J. Wegman, Eds. New York: Elsevier, 2004.
47. A. de Decker, J. A. Lee, D. François, and M. Verleysen, "Mode estimation in high-dimensional spaces with flat-top kernels: Application to image denoising," Adv. Artificial Neural Netw. Mach. Learn. Comput. Intell., vol. 74, no. 9, pp. 1402-1410, 2010.
48. V. Laparra, G. Camps-Valls, and J. Malo, "Iterative Gaussianization: From ICA to random rotations," IEEE Trans. Neural Netw., vol. 22, no. 4, pp. 537-549, Feb. 2011.
49. A. Peñalver and F. Escolano, "Entropy-based incremental variational bayes learning of Gaussian mixtures," IEEE Trans. Neural Netw., vol. 23, no. 3, pp. 534-540, Mar. 2012.
50. P. L. Zador, "Asymptotic quantization error of continuous signals and the quantization dimension," IEEE Trans. Inform. Theory, vol. 28, no. 2, pp. 149-159, Mar. 1982.
51. D. Dersch and P. Tavan, "Asymptotic level density in topological feature maps," IEEE Trans. Neural Netw., vol. 6, no. 1, pp. 230-236, Jan. 1995.
52. T. M. Martinetz, S. G. Berkovich, and K. J. Schulten, "Neural-gas network for vector quantization and its application to time-series prediction," IEEE Trans. Neural Netw., vol. 4, no. 4, pp. 558-569, Jul. 1993.
53. P. Grassberger and I. Procaccia, "Characterization of strange attractors," Phys. Rev. Lett., vol. 50, no. 5, pp. 346-349, 1983.
54. J. Bruske and G. Sommer, "Intrinsic dimensionality estimation with optimally topology preserving maps," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 5, pp. 572-575, May 1998.
55. H. U. Bauer, R. Der, and M. Herrmann, "Controlling the magnification factor of self-organizing feature maps," Neural Comput., vol. 8, no. 4, pp. 757-771, 1996.
56. T. Villmann and J. C. Claussen, "Magnification control in self-organizing maps and neural gas," Neural Comput., vol. 18, no. 2, pp. 446-469, 2006.
57. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, no. 1, pp. 79-86, 1951.
58. Q. Wang, S. R. Kulkarni, and S. Verdú, "Divergence estimation of continuous distributions based on data-dependent partitions," IEEE Trans. Inform. Theory, vol. 51, no. 9, pp. 3064-3074, Sep. 2005.
59. Q. Wang, S. R. Kulkarni, and S. Verdú, "Divergence estimation for multidimensional densities via k-nearest-neighbor distances," IEEE Trans. Inform. Theory, vol. 55, no. 5, pp. 2392-2405, May 2009.
60. E. Keogh, Q. Zhu, B. Hu, Y. Hao, X. Xi, L. Wei, and C. A. Ratanamahatana. The UCR Time Series Classification/Clustering Homepage [Online]. Available: http://www.cs.ucr.edu/∼eamonn/time-series-data/
61. Y. Aït-Sahalia and A. Lo, "Nonparametric estimation of state-price densities implicit in financial asset prices," to be published.
62. C. R. Harvey, "The Specification of conditional expectation," to be published.
63. A. R. Pagan and G. W. Schwert, "Alternative models for conditional stock volatility," J. Econometrics, vol. 45, nos. 1-2, pp. 267-290, 1990.
64. R. Balupari, "Real-time network-based anomaly intrusion detection," M.S. thesis, Dept. Elect. Eng. Comput. Sci., Ohio Univ., Athens, OH, 2002.
65. H. He and E. A. Garcia, "Learning from imbalanced data," IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263-1284, May 2009.
66. S. Chen, H. He, and E. A. Garcia, "RAMOBoost: Ranked minority oversampling in boosting," IEEE Trans. Neural Netw., vol. 21, no. 10, pp. 1624-1642, Aug. 2010.