Simplifying mixture models through function approximation

IEEE Transactions on Neural Networks

Volumn 21, Issue 4, 2010, Pages 644-658

  Zhang, Kai ^a   Kwok, James T ^b  

^a LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

^b HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Clustering; Mixture models; Support vector machine (SVM) testing

Indexed keywords

APPROXIMATION ERRORS; BAYESIAN FILTERING; BELIEF PROPAGATION; DECISION FUNCTIONS; DENSITY ESTIMATION; DISTANCE MEASURE; FINITE MIXTURE MODELS; FUNCTION APPROXIMATION; KERNEL DENSITY ESTIMATION; MIXTURE MODEL; MIXTURE MODELS; NUMBER OF COMPONENTS; ORIGINAL MODEL; SIMPLIFIED MODELS; SPEED-UPS; STATISTICAL LEARNING; UPPER BOUND;

IMAGE SEGMENTATION; MIXTURES; PROBABILITY DENSITY FUNCTION; STATISTICAL TESTS;

SUPPORT VECTOR MACHINES;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; CLUSTER ANALYSIS; COMPUTER SIMULATION; HUMAN; IMAGE PROCESSING; LEARNING; PROBABILITY; STATISTICAL MODEL; TIME;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; COMPUTER SIMULATION; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LEARNING; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; PROBABILITY; TIME FACTORS;

EID: 77950857605     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2040835     Document Type: Article

References (45)

1. G. A. Babich and O. I. Camps, "Weighted Parzen windows for pattern classification," IEEE Trans. Pattern Anal. Mach. Intell., vol.18, no.5, pp. 567-570, May 1996.
2. A. Banerjee, S. Merugu, I. S. Dhillon, and J. Ghosh, "Clustering with Bregman divergences," J. Mach. Learn. Res., vol.6, pp. 1705-1749, 2005.
3. C. J. C. Burges and B. Schölkopf, "Improving the accuracy and speed of support vector machines," in Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1996, pp. 375-381.
4. D. Comaniciu, "An algorithm for data-driven bandwidth selection," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.2, pp. 281-288, Feb. 2003.
5. D. Comaniciu and P. Meer, "Mean shift analysis and applications," in Proc. Int. Conf. Comput. Vis., 1999, vol.2, pp. 1197-1204.
6. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol.24, no.5, pp. 603-619, May 2002.
7. D. Comaniciu, V. Ramesh, and P. Meer, "Kernel-based object tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.5, pp. 564-577, May 2003.
8. D. Comaniciu, V. Ramesh, and P. Meer, "The variable bandwidth mean shift and data-driven scale selection," in Proc. IEEE Int. Conf. Comput. Vis., 2001, vol.1, pp. 438-445.
9. J. V. Davis and I. Dhillon, "Differential entropic clustering of multivariate Gaussians," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007.
10. D. DeMenthon and D. Doermann, "Video retrieval using spatio-temporal descriptors," in Proc. 11th ACM Int. Conf. Multimedia, Berkeley, CA, 2003, pp. 508-517.
11. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Statist. Soc. B, vol.39, no.1, pp. 1-38, 1977.
12. J. Fan and J. S. Marron, "Fast implementations of nonparametric curve estimators," J. Comput. Graph. Statist., vol.3, no.1, pp. 35-56, Mar. 1994.
13. K. Fukunaga and L. D. Hostetler, "The estimation of the gradient of a density function, with applications in pattern recognition," IEEE Trans. Inf. Theory, vol.21, no.1, pp. 32-40, Jan. 1975.
14. B. Georgescu, I. Shimshoni, and P. Meer, "Mean shift based clustering in high dimensions, a texture classification example," in Proc. 9th IEEE Int. Conf. Comput. Vis., 2003, pp. 456-463.
15. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1992.
16. A. Gionis, P. Indyk, and R. Motwani, "Similarity search in high dimensions via hashing," in Proc. 25th Int. Conf. Very Large Data Bases, 1999, pp. 518-529.
17. M. Girolami and C. He, "Probability density estimation from optimally condensed data samples," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.10, pp. 1253-1264, Oct. 2003.
18. J. Goldberger, H. Greenspan, and J. Dreyfuss, "Simplifying mixture models using the unscented transform," IEEE Trans. Pattern Anal. Mach. Intell., vol.30, no.8, pp. 1496-1502, Aug. 2008.
19. J. Goldberger and S. Roweis, "Hierarchical clustering of a mixture model," in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp. 505-512.
20. B. Han, D. Comaniciu, Y. Zhu, and L. Davis, "Incremental density approximation and kernel-based Bayesian filtering for object tracking," in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2004, vol.1, pp. 638-644.
21. W. Härdle and D. W. Scott, "Smoothing by weighted averaging of rounded points," Comput. Stat., vol.7, pp. 97-128, 1992.
22. A. J. Izenman, "Recent developments in nonparametric density estimation," J. Amer. Stat. Assoc., vol.86, no.413, pp. 205-224, 1991.
23. B. Jeon and D. A. Landgrebe, "Fast Parzen density estimation using clustering based branch and bound," IEEE Trans. Pattern Anal. Mach. Intell., vol.16, no.9, pp. 950-954, Sep. 1994.
24. S. Julier, J. Uhlmann, and H. F. Durrant-Whyte, "A new method for the nonlinear transformation of means and covariances in filters and estimators," IEEE Trans. Autom. Control, vol.45, no.3, pp. 477-482, Aug. 2002.
25. T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu, "An efficient κ-means clustering algorithm: Analysis and implementation," IEEE Trans. Pattern Anal. Mach. Intell., vol.24, no.7, pp. 881-892, Jul. 2002.
26. K. I. Kim, K. Jung, and J. H. Kim, "Texture-based approach for text detection in images using support vector machines and continuously adaptive mean shift algorithm," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.12, pp. 1631-1639, Dec. 2003.
27. H. Lütkepohl, Handbook of Matrices. New York: Wiley, 1996.
28. J. S. Marron and M. P. Wand, "Exact mean integrated squared error," Ann. Stat., vol.20, no.2, pp. 712-736, 1992.
29. G. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. New York: Wiley, 1992.
30. G. McLachlan and D. Peel, Finite Mixture Models. New York:Wiley, 2000.
31. A. W. Moore, "Very fast EM-based mixture model clustering using multiresolution kd-trees," in Advances in Neural Information Processing Systems 11, M. S. Kearns, S. A. Solla, and D. A. Cohn, Eds. San Mateo, CA: Morgan Kaufmann, 1998, pp. 543-549.
32. E. Parzen, "On estimation of a probability density function and mode," Ann. Math. Stat., vol.33, pp. 1065-1075, 1962.
33. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol.77, no.2, pp. 257-286, Feb. 1989.
34. K. Roeder, "Density estimation with confidence sets exemplified by superclusters and voids in the galaxies," J. Amer. Stat. Assoc., vol.85, no.411, pp. 617-624, 1990.
35. H. Samet, Applications of Spatial Data Structures: Computer Graphics, Image Processing, and GIS. Boston, MA: Addison- Wesley, 1990.
36. B. Schölkopf and A. J. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2002.
37. D. W. Scott and S. J. Sheather, "Kernel density estimation with binned data," Commun. Stat. A-Theory Methods, vol.14, pp. 1353-1359, 1985.
38. E. B. Sudderth, A. T. Ihler, W. T. Freeman, and A. S. Willsky, "Distributed occlusion reasoning for tracking with nonparametric belief propagation," in Advances in Neural Information Processing Systems 17. Cambridge, MA: MIT Press, 2005, pp. 1369-1376.
39. E. B. Sudderth, A. T. Ihler, W. T. Freeman, and A. S. Willsky, "Nonparametric belief propagation," in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2003, pp. 605-612.
40. P. Vincent and Y. Bengio, "Manifold Parzen windows," in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2003, pp. 825-832.
41. J. Zhang, K. Zhang, X. Xu, C. K. Tse, and M. Small, "Seeding the kernels in graphs: Toward multi-resolution community analysis," New J. Phys., vol.11, 2009, 113003.
42. K. Zhang and J. T. Kwok, "Density-weighted Nyström method for computing large kernel eigensystems," Neural Comput., vol.21, no.1, pp. 121-146, 2009.
43. K. Zhang and J. T. Kwok, "Simplifying mixture models through function approximation," in Advances in Neural Information Processing Systems 17, B. Schölkopf, J. Platt, and T. Hoffman, Eds. Cambridge, MA: MIT Press, 2007, pp. 1577-1584.
44. K. Zhang, M. Tang, and J. T. Kwok, "Applying neighborhood consistency for fast clustering and kernel density estimation," in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2005, vol.2, pp. 1001-1007.
45. X. S. Zhou, D. Comaniciu, and A. Gupta, "An information fusion framework for robust shape tracking," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.1, pp. 115-129, Jan. 2005.