Classification via minimum incremental coding length

SIAM Journal on Imaging Sciences

Volumn 2, Issue 2, 2009, Pages 367-395

  Wright, John ^a   Ma, Yi ^a   Tao, Yangyu ^b   Lin, Zhouchen ^b   Shum, Heung Yeung ^b  

^a University of Illinois at Urbana Champaign   (United States)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

Classification; Lossy data coding; Maximum a posteriori; Regularization; Regularized discriminant analysis

Indexed keywords

CODES (SYMBOLS); DISCRIMINANT ANALYSIS; MATLAB; NEAREST NEIGHBOR SEARCH; SUPPORT VECTOR MACHINES;

ASYMPTOTIC OPTIMALITY; CLASSIFICATION RESULTS; DATA CODING; DOMAIN-SPECIFIC INFORMATION; LOSSY DATA COMPRESSION; MAXIMUM A POSTERIORI; REGULARIZATION; REGULARIZED DISCRIMINANT ANALYSIS;

CLASSIFICATION (OF INFORMATION);

EID: 84981228964     PISSN: None     EISSN: 19364954     Source Type: Journal    
DOI: 10.1137/070707312     Document Type: Article

References (34)

1. A. Barron, J. Rissanen, and B. Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Theory, 44 (1998), pp. 2743-2760.
2. R. Basri and D. Jacobs, Lambertian reflection and linear subspaces, IEEE Trans. Pattern Anal. Mach. Intell., 25 (2003), pp. 218-233.
3. P. Belhumeur, J. Hespanda, and D. Kriegman, Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell., 19 (1997), pp. 711-720.
4. P. Bickel and E. Levina, Regularized estimation of large covariance matrices, Ann. Statist., 36 (2008), pp. 199-227.
5. P. Bickel and B. Li, Regularization in statistics, Test, 15 (2006), pp. 271-344.
6. C. Bishop, Pattern Recognition and Machine Learning, Springer, New York, 2006.
7. C. Chang and C. Lin, LIBSVM: A Library for Support Vector Machines, 2001; software available online at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
8. T. Cover, Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition, IEEE Trans. Electron. Comput., 14 (1965), pp. 326-334.
9. T. Cover and J. Thomas, Elements of Information Theory, Wiley Series in Telecommunications, John Wiley & Sons, New York, 1991.
10. B. Dasarathy, Nearest Neighbor Norms: NN Pattern Classification Techniques, IEEE Computer Society Press, Los Alamitos, CA, 1991.
11. J. Friedman, Regularized discriminant analysis, J. Amer. Statist. Assoc., 84 (1989), pp. 165-175.
12. A. Georghiades, P. Belhumeur, and D. Kriegman, From few to many: Illumination cone models for face recognition under variable lighting and pose, IEEE Trans. Pattern Anal. Mach. Intell., 23 (2001), pp. 643-660.
13. P. Grunwald and J. Langford, Suboptimal behavior of Bayes and MDL in classification under misspecification, Machine Learning, 66 (2007), pp. 119-149.
14. J. Hamkins and K. Zeger, Gaussian source coding with spherical codes, IEEE Trans. Inform. Theory, 48 (2002), pp. 2980-2989.
15. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, Springer, New York, 2001.
16. J. Ho, M. Yang, J. Lim, K. Lee, and D. Kriegman, Clustering appearances of objects under varying illumination conditions, in Proceedings of IEEE Computer Science Society Conference on Computer Vision and Pattern Recognition, 2003.
17. A. Holub, M. Welling, and P. Perona, Combining generative models and Fisher kernels for object recognition, in Proceedings of the Tenth IEEE International Conference on Computer Vision, 2005, pp. 136-143.
18. I. Johnstone and A. Lu, Sparse Principal Components Analysis, preprint, 2004, http://www-stat. stanford.edu/~imj/WEBLIST/AsYetUnpub/sparse.pdf.
19. D. Keysers, T. Deselaers, C. Golian, and H. Ney, Deformation models for image recognition, IEEE Trans. Pattern Anal. Mach. Intell., 29 (2007), pp. 1422-1435.
20. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, 86 (1998), pp. 2278-2324.
21. K. Lee, J. Ho, and D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting, IEEE Trans. Pattern Anal. Mach. Intell., 27 (2005), pp. 684-698.
22. J. Li, A source coding approach to classification by vector quantization and the principle of minimum description length, in Proceedings of the IEEE Data Compression Conference, 2002, pp. 382-391.
23. Y. Ma, H. Derksen, W. Hong, and J. Wright, Segmentation of multivariate mixed data via lossy data coding and compression, IEEE Trans. Pattern Anal. Mach. Intell., 29 (2007), pp. 1546-1562.
24. D. MacKay, Developments in probabilistic modelling with neural networks-ensemble learning, in Proceedings of the 3rd Annual Symposium on Neural Networks, Springer, Berlin, 1995, pp. 191-198.
25. M. Madiman, M. Harrison, and I. Kontoyiannis, Minimum description length vs. maximum likelihood in lossy data compression, in Proceedings of the IEEE International Symposium on Information Theory, 2004.
26. T. Minka, Inferring a Gaussian Distribution, MIT Media Lab note, MIT, Cambridge, MA, 1998.
27. S.A. Nene and S.K. Nayar, A simple algorithm for nearest neighbor search in high dimensions, IEEE Trans. Pattern Anal. Mach. Intell., 19 (1997), pp. 989-1003.
28. J.J. Rissanen, Modeling by shortest data description, Automatica J. IFAC, 14 (1978), pp. 465-471.
29. P. Simard, Y. LeCun, and J. Denker, Efficient pattern recognition using a new transformation distance, in Advances in Neural Information Processing Systems, Vol. 5, Morgan Kaufmann, San Francisco, 1993, pp. 50-58.
30. P. Simard, D. Steinkraus, and J. Platt, Best practice for convolutional neural networks applied to visual document analysis, in Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), IEEE Computer Society, Los Alamitos, CA, 2003, pp. 958-962.
31. M. Turk and A. Pentland, Eigenfaces for recognition, J. Cognitive Neurosci., 3 (1991), pp. 71-86.
32. V. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, 2000.
33. X. Wang and X. Tang, A unified framework for subspace face recognition, IEEE Trans. Pattern Anal. Mach. Intell., 26 (2004), pp. 1222-1228.
34. J. Wright, A. Ganesh, A. Yang, S. Sastry, and Y. Ma, Robust face recognition via sparse representation, IEEE Trans. Pattern Anal. Mach. Intell., 31 (2009), pp. 210-227.