Theoretical analysis of density ratio estimation

IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences

Volumn E93-A, Issue 4, 2010, Pages 787-798

  Kanamori, Takafumi ^a   Suzuki, Taiji ^b   Sugiyama, Masashi ^c,d  

^a NAGOYA UNIVERSITY   (Japan)

^b UNIVERSITY OF TOKYO   (Japan)

^c TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^d JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

Asymptotic analysis; Density estimation; Density ratio estimation; Gaussian assumption; Logistic regression

Indexed keywords

ASYMPTOTIC ANALYSIS; PROBABILITY; REGRESSION ANALYSIS;

DENSITY ESTIMATION; DENSITY-RATIO ESTIMATIONS; EXPONENTIAL FAMILY; GAUSSIAN ASSUMPTION; KULLBACK LEIBLER DIVERGENCE; LOGISTIC REGRESSION CLASSIFIER; LOGISTIC REGRESSIONS; POSTERIOR PROBABILITY;

DATA HANDLING;

EID: 77950856653     PISSN: 09168508     EISSN: 17451337     Source Type: Journal    
DOI: 10.1587/transfun.E93.A.787     Document Type: Article

References (35)

1. S. Bickel, M. Brückner, and T. Scheffer, " Discriminative learning for differing training and test distributions, " Proc. 24th International Conference on Machine Learning, pp. 81-88, 2007. (Pubitemid 47275052)
2. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games, Cambridge University Press, Cambridge, NY, 2006.
3. K. F. Cheng and C. K. Chu, " Semiparametric density estimation under a two-sample density ratio model, " Bernoulli, vol. 10, no. 4, pp. 583-604, 2004. (Pubitemid 44242758)
4. C. Cortes and V. Vapnik, " Support-vector networks, " Mach. Learn., vol. 20, pp. 273-297, 1995.
5. B. Efron, " The efficiency of logistic regression compared to normal discriminant analysis, " J. American Statistical Association, vol. 70, no. 352, pp. 892-898, 1975.
6. V. V. Fedorov, Theory of Optimal Experiments, Academic Press, New York, 1972.
7. S. Hido, Y. Tsuboi, H. Kashima, M. Sugiyama, and T. Kanamori, " Inlier-based outlier detection via direct density ratio estimation, " Proc. IEEE International Conference on Data Mining (ICDM2008), ed. F. Giannotti, D. Gunopulos, F. Turini, C. Zaniolo, N. Ramakrishnan, and X. Wu, pp. 223-232, Pisa, Italy, Dec. 2008.
8. J. Huang, A. Smola, A. Gretton, K. M. Borgwardt, and B. Schölkopf, " Correcting sample selection bias by unlabeled data, " in Advances in Neural Information Processing Systems 19, ed. B. Schölkopf, J. Platt, and T. Hoffman, pp. 601-608, MIT Press, Cambridge, MA, 2007.
9. T. Kanamori, " Pool-based active learning with optimal sampling distribution and its information geometrical interpretation, " Neurocomputing, vol. 71, no. 1-3, pp. 353-362, 2007. (Pubitemid 350028654)
10. T. Kanamori, S. Hido, and M. Sugiyama, " A least-squares approach to direct importance estimation, " J. Machine Learning Research, vol. 10, pp. 1391-1445, July 2009.
11. T. Kanamori and H. Shimodaira, " Active learning algorithm using the maximum weighted log-likelihood estimator, " J. Statistical Planning and Inference, vol. 116, no. 1, pp. 149-162, 2003. (Pubitemid 36742567)
12. S. Kullback and R. A. Leibler, " On information and sufficiency, " Annals of Mathematical Statistics, vol. 22, pp. 79-86, 1951.
13. E. L. Lehmann and G. Casella, Theory of Point Estimation, second ed., Springer, New York, 1998.
14. P. Liang and M. Jordan, An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators, pp. 584-591, 2008.
15. X. Nguyen, M. Wainwright, and M. Jordan, " Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization, " in Advances in Neural Information Processing Systems 20, ed. J. C. Platt, D. Koller, Y. Singer, and S. Roweis, pp. 1089-1096, MIT Press, Cambridge, MA, 2008.
16. J. Qin, " Inferences for case-control and semiparametric two-sample density ratio models, " Biometrika, vol. 85, no. 3, pp. 619-639, 1998.
17. J. Quiñonero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence, eds., Dataset Shift in Machine Learning, MIT Press, Cambridge, MA, 2009.
18. H. Shimodaira, " Improving predictive inference under covariate shift by weighting the log-likelihood function, " J. Statistical Planning and Inference, vol. 90, no. 2, pp. 227-244, 2000.
19. A. Smola, L. Song, and C. H. Teo, " Relative novelty detection, " JMLR Workshop and Conference Proceedings, ed. D. van Dyk and M. Welling, Twelfth International Conference on Artificial Intelligence and Statistics, vol. 5, pp. 536-543, 2009.
20. M. Sugiyama, " Active learning in approximately linear regression based on conditional expectation of generalization error, " J. Machine Learning Research, vol. 7, pp. 141-166, Jan. 2006. (Pubitemid 43100915)
21. M. Sugiyama, T. Kanamori, T. Suzuki, S. Hido, J. Sese, I. Takeuchi, and L. Wang, " A density-ratio framework for statistical data processing, " IPSJ Trans. Computer Vision and Applications, vol. 1, pp. 183-208, 2009.
22. M. Sugiyama, M. Krauledat, and K. R. Müller, " Covariate shift adaptation by importance weighted cross validation, " J. Machine Learning Research, vol. 8, pp. 985-1005, May 2007. (Pubitemid 46798411)
23. M. Sugiyama and K. R. Müller, " Input-dependent estimation of generalization error under covariate shift, " Statistics & Decisions, vol. 23, no. 4, pp. 249-279, 2005.
24. M. Sugiyama and S. Nakajima, " Pool-based active learning in approximate linear regression, " Mach. Learn., vol. 75, no. 3, pp. 249-274, 2009.
25. M. Sugiyama, T. Suzuki, S. Nakajima, H. Kashima, P. von Bünau, and M. Kawanabe, " Direct importance estimation for covariate shift adaptation, " Annals of the Institute of Statistical Mathematics, vol. 60, no. 4, pp. 699-746, 2008.
26. M. Sugiyama, I. Takeuchi, T. Suzuki, T. Kanamori, and H. Hachiya, " Least-squares conditional density estimation, " Tech. Rep. TR09-0004, Department of Computer Science, Tokyo Institute of Technology, Feb. 2009.
27. M. Sugiyama, P. von Bünau, M. Kawanabe, and K. R. Müller, Covariate Shift Adaptation: Towards Machine Learning in Non-Stationary Environment, MIT Press, Cambridge, MA, 2010. to appear.
28. T. Suzuki and M. Sugiyama, " Estimating squared-loss mutual information for independent component analysis, " Independent Component Analysis and Signal Separation, ed. T. Adali, C. Jutten, J. M. T. Romano, and A. K. Barros, Lect. Notes Comput. Sci., vol. 5441, pp. 130-137, Springer, Berlin, 2009.
29. T. Suzuki and M. Sugiyama, " Sufficient dimension reduction via squared-loss mutual information estimation, " Tech. Rep. TR09-0005, Department of Computer Science, Tokyo Institute of Technology, Feb. 2009.
30. T. Suzuki, M. Sugiyama, T. Kanamori, and J. Sese, " Mutual information estimation reveals global associations between stimuli and biological processes, " BMC Bioinformatics, vol. 10, no. 1, p. S52, 2009.
31. T. Suzuki, M. Sugiyama, J. Sese, and T. Kanamori, " Approximating mutual information by maximum likelihood density ratio estimation, " JMLR Workshop and Conference Proceedings, ed. Y. Saeys, H. Liu, I. Inza, L. Wehenkel, and Y. V. de Peer, New Challenges for Feature Selection in Data Mining and Knowledge Discovery, vol. 4, pp. 5-20, 2008.
32. A. W. van der Vaart, Asymptotic Statistics, Cambridge University Press, 1998.
33. V. N. Vapnik, Statistical Learning Theory, Wiley, New York, 1998.
34. D. P. Wiens, " Robust weights and designs for biased regression models: Least squares and generalized M-estimation, " J. Statistical Planning and Inference, vol. 83, no. 2, pp. 395-412, 2000.
35. B. Zadrozny, " Learning and evaluating classifiers under sample selection bias, " Proc. Twenty-First International Conference on Machine Learning, pp. 903-910, ACM Press, New York, NY, 2004. (Pubitemid 40290896)