Density-difference estimation

Neural Computation

Volumn 25, Issue 10, 2013, Pages 2734-2775

  Sugiyama, Masashi ^a   Kanamori, Takafumi ^b   Suzuki, Taiji ^c   Plessis, Marthinus Christoffel Du ^a   Liu, Song ^a   Takeuchi, Ichiro ^b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b NAGOYA UNIVERSITY   (Japan)

^c UNIVERSITY OF TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUSTRALIA; COMPUTER PROGRAM; DATA MINING; DIABETES MELLITUS; FACTUAL DATABASE; GERMANY; HUMAN; STATISTICAL MODEL; STATISTICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUSTRALIA; DATA MINING; DATABASES, FACTUAL; DIABETES MELLITUS; GERMANY; HUMANS; LOGISTIC MODELS; SOFTWARE;

EID: 84877702876     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00492     Document Type: Article

References (70)

1. Anderson, N., Hall, P., & Titterington, D. (1994). Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates. Journal of Multivariate Analysis, 50, 41-54.
2. Arlot, S., Celisse, A., & Harchaoui, Z. (2012). Kernel change-point detection (Tech. Rep. 1202.3878). arXiv.
3. Aronszajn, N. (1950). Theory of reproducing kernels. Transactions of the American Mathematical Society, 68, 337-404.
4. Atif, J., Ripoche, X., & Osorio, A. (2003). Non-rigid medical image registration by maximisation of quadratic mutual information. In Proceedings of the IEEE 29th Annual Northeast Bioengineering Conference (pp. 32-40). Piscataway, NJ: IEEE.
5. Baranchik, A. J. (1964). Multiple regression and estimation of the mean of a multivariate normal distribution (Tech. Rep. 51). Stanford, CA: Department of Statistics, Stanford University.
6. Basu, A., Harris, I. R., Hjort, N. L., & Jones, M. C. (1998). Robust and efficient estimation by minimising a density power divergence. Biometrika, 85, 549-559.
7. Besbeas, P., & Morgan, B. J. T. (2004). Integrated squared error estimation of normal mixtures. Computational Statistics and Data Analysis, 44, 517-526.
8. Bickel, S., Brückner, M., & Scheffer, T. (2007). Discriminative learning for differing training and test distributions. In Proceedings of the 24th International Conference on Machine Learning (pp. 81-88). New York: ACM.
9. Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory (pp. 144-152). New York: ACM Press.
10. Chapelle, O., Schölkopf, B., & Zien, A. (Eds.). (2006). Semi-supervised learning. Cambridge, MA: MIT Press.
11. Chow, C. K. (1970). On optimum recognition error and reject tradeoff. IEEE Transactions on Information Theory, 16, 41-46.
12. Cortes, C., Mansour, Y., & Mohri, M. (2010). Learning bounds for importance weighting. In J. Lafferty, C. K. I.Williams, R. Zemel, J. Shawe-Taylor, & A. Culotta (Eds.), Advances in neural information processing systems, 23 (pp. 442-450). Cambridge, MA: MIT Press.
13. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273-297.
14. Desobry, F., Davy, M., & Doncarli, C. (2005). An online kernel change detection algorithm. IEEE Transactions on Signal Processing, 53, 2961-2974.
15. du Plessis, M. C. (2013). Class-prior change labeling. Unpublished research memo.
16. du Plessis, M. C., & Sugiyama, M. (2012). Semi-supervised learning of class balance under class-prior change by distribution matching. In Proceedings of 29th International Conference on Machine Learning (pp. 823-830). Madison, WI: Omnipress.
17. Duda, R. O., Hart, P. E., & Stork, D. G. (2001). Pattern classification (2nd ed.). New York:Wiley.
18. Duong, T., Koch, I., & Wand, M. P. (2009). Highest density difference region estimation with application to flow cytometric data. Biometrical Journal, 51, 504-521.
19. Eberts, M., & Steinwart, I. (2011). Optimal learning rates for least squares SVMs using gaussian kernels. In J. Shawe-Taylor, R. S. Zemel, P. Bartlett, F. C. N. Pereira, & K. Q. Weinberger (Eds.), Advances in neural information processing systems, 24 (pp. 1539-1547). Red Hook, NY: Curran.
20. Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. Boca Raton, FL: Chapman & Hall/CRC.
21. Farrell, R. H. (1972). On the best obtainable asymptotic rates of convergence in estimation of a density function at a point. Annals of Mathematical Statistics, 43, 170-180.
22. Gray, D.M., & Principe, J. C. (2010). Quadratic mutual information for dimensionality reduction and classification. Proceedings of SPIE (p. 76960D). Bellingham, WA: SPIE.
23. Gretton, A., Smola, A., Huang, J., Schmittfull, M., Borgwardt, K., & Schölkopf, B. (2009). Covariate shift by kernel mean matching. In J. Quiñonero, M. Sugiyama, A. Schwaighofer, & N. Lawrence (Eds.), Dataset shift in machine learning (pp. 131-160). Cambridge, MA: MIT Press.
24. Hall, P., & Wand, M. P. (1988). On nonparametric discrimination using density differences. Biometrika, 75, 541-547.
25. Harchaoui, Z., Bach, F., & Moulines, E. (2009). Kernel change-point analysis. In D. Koller, D. Schuurmans, Y. Bengio, & L. Bettou (Eds.), Advances in neural information processing systems, 21 (pp. 609-616). Cambridge, MA: MIT Press.
26. Härdle, W., Müller, M., Sperlich, S., & Werwatz, A. (2004). Nonparametric and semiparametric models. Berlin: Springer.
27. Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: Data mining, inference, and prediction. New York: Springer.
28. Henkel, R. E. (1976). Tests of significance. Beverly Hills, CA: Sage.
29. Huang, J., Smola, A., Gretton, A., Borgwardt, K. M., & Schölkopf, B. (2007). Correcting sample selection bias by unlabeled data. In B. Schölkopf, J. C. Platt, & T. Hoffmann (Eds.), Advances in neural information processing systems, 19 (pp. 601-608). Cambridge, MA: MIT Press.
30. Kanamori, T., Hido, S., & Sugiyama, M. (2009). A least-squares approach to direct importance estimation. Journal of Machine Learning Research, 10, 1391-1445.
31. Kanamori, T., Suzuki, T., & Sugiyama, M. (2012). Statistical analysis of kernel-based least-squares density-ratio estimation. Machine Learning, 86, 335-367.
32. Kawahara, Y., & Sugiyama, M. (2012). Sequential change-point detection based on direct density-ratio estimation. Statistical Analysis and Data Mining, 5, 114-127.
33. Kawahara, Y., Yairi, T., & Machida, K. (2007). Change-point detection in time-series data based on subspace identification. In Proceedings of the 7th IEEE International Conference on Data Mining (pp. 559-564). Piscataway, NJ: IEEE.
34. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics, 22, 79-86.
35. Liu, B., Cheng, H. D., Huang, J., Tian, J., Tang, X., & Liu, J. (2010). Probability density difference-based active contour for ultrasound image segmentation. Pattern Recognition, 43, 2028-2042.
36. Liu, S., Yamada, M., Collier, N., & Sugiyama, M. (2013). Change-point detection in time-series data by relative density-ratio estimation. Neural Networks, 43, 72-83.
37. Moskvina, V., & Zhigljavsky, A. A. (2003). An algorithm based on singular spectrum analysis for change-point detection. Communication in Statistics: Simulation and Computation, 32, 319-352.
38. Nguyen, X., Wainwright, M. J., & Jordan, M. I. (2010). Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56, 5847-5861.
39. Parzen, E. (1962). On the estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, 1065-1076.
40. Pérez-Cruz, F. (2008). Kullback-Leibler divergence estimation of continuous distributions. In Proceedings of IEEE International Symposium on Information Theory (pp. 1666-1670). Piscataway, NJ: IEEE.
41. Qin, J. (1998). Inferences for case-control and semiparametric two-sample density ratio models. Biometrika, 85, 619-630.
42. Rao, C. R. (1965). Linear statistical inference and its applications. NewYork: Wiley.
43. Rifkin, R., Yeo, G., & Poggio, T. (2003). Regularized least-squares classification. In J.A.K. Suykens, G.Horvath, S. Basu, C. Micchelli, & J.Vandewalle (Eds.), Advances in learning theory: Methods, models and applications (pp. 131-154). Amsterdam: IOS Press.
44. Rockafellar, R. T. (1970). Convex analysis. Princeton, NJ: Princeton University Press.
45. Saerens, M., Latinne, P., & Decaestecker, C. (2002). Adjusting the outputs of a classifier to new a priori probabilities: A simple procedure. Neural Computation, 14, 21-41.
46. Scott, D. W. (2001). Parametric statistical modeling by minimum integrated square error. Technometrics, 43, 274-285.
47. Silva, J., & Narayanan, S. S. (2010). Information divergence estimation based on data-dependent partitions. Journal of Statistical Planning and Inference, 140, 3180-3198.
48. Silverman, B. W. (1986). Density estimation for statistics and data analysis. London: Chapman and Hall.
49. Steinwart, I., & Christmann, A. (2008). Support vector machines. NewYork: Springer.
50. Steinwart, I., & Scovel, C. (2007). Fast rates for support vector machines using gaussian kernels. Annals of Statistics, 35, 575-607.
51. Sugiyama, M. (2010). Superfast-trainable multi-class probabilistic classifier by leastsquares posterior fitting. IEICE Transactions on Information and Systems, E93-D, 2690-2701.
52. Sugiyama, M., & Kawanabe, M. (2012). Machine learning in non-stationary environments: Introduction to covariate shift adaptation. Cambridge, MA: MIT Press.
53. Sugiyama, M., Kawanabe, M., & Chui, P. L. (2010). Dimensionality reduction for density ratio estimation in high-dimensional spaces. Neural Networks, 23, 44-59.
54. Sugiyama, M., Krauledat, M., & Müller, K.-R. (2007). Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8, 985-1005.
55. Sugiyama, M., Suzuki, T., & Kanamori, T. (2012a). Density ratio estimation in machine learning. Cambridge: Cambridge University Press.
56. Sugiyama, M., Suzuki, T., & Kanamori, T. (2012b). Density ratio matching under the Bregman divergence: A unified framework of density ratio estimation. Annals of the Institute of Statistical Mathematics, 64, 1009-1044.
57. Sugiyama, M., Suzuki, T., Kanamori, T., du Plessis, M. C., Liu, S., & Takeuchi, I. (2012). Density-difference estimation. In P. Bartlett, F.C.N. Pereira, C.J.C. Burgess, L. Bettou, & K. Q. Weinberger (Eds.), Advances in neural information processing systems, 25. Red Hook, NY: Curran.
58. Sugiyama, M., Suzuki, T., Nakajima, S., Kashima, H., von Bünau, P., & Kawanabe, M. (2008). Direct importance estimation for covariate shift adaptation. Annals of the Institute of Statistical Mathematics, 60, 699-746.
59. Sugiyama, M., Takeuchi, I., Suzuki, T., Kanamori, T., Hachiya, H., & Okanohara, D. (2010). Least-squares conditional density estimation. IEICE Transactions on Information and Systems, E93-D, 583-594.
60. Sugiyama, M., Yamada, M., von Bünau, P., Suzuki, T., Kanamori, T., & Kawanabe, M. (2011). Direct density-ratio estimation with dimensionality reduction via leastsquares hetero-distributional subspace search. Neural Networks, 24, 183-198.
61. Suzuki, T., & Sugiyama, M. (2013). Sufficient dimension reduction via squared-loss mutual information estimation. Neural Computation, 25, 725-758.
62. Takeuchi, Y., & Yamanishi, K. (2006). A unifying framework for detecting outliers and change points from non-stationary time series data. IEEE Transactions on Knowledge and Data Engineering, 18, 482-489.
63. Torkkola, K. (2003). Feature extraction by non-parametric mutual information maximization. Journal of Machine Learning Research, 3, 1415-1438.
64. Vapnik, V. N. (1998). Statistical learning theory. New York:Wiley.
65. von Bünau, P., Meinecke, F. C., Király, F. J., & Müller, K.-R. (2009). Finding stationary subspaces in multivariate time series. Physical Review Letters, 103, 214101.
66. Wang, Q., Kulkarmi, S. R., & Verdu, S. (2005). Divergence estimation of continuous distributions based on data-dependent partitions. IEEE Transactions on Information Theory, 51, 3064-3074.
67. Yamada, M., & Sugiyama, M. (2011). Direct density-ratio estimation with dimensionality reduction via hetero-distributional subspace analysis. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence (pp. 549-554). San Francisco: AAAI Press.
68. Yamada, M., Suzuki, T., Kanamori, T., Hachiya, H., & Sugiyama, M. (2013). Relative density-ratio estimation for robust distribution comparison. Neural Computation, 25, 1324-1370.
69. Yamanaka, M., Matsugu, M., & Sugiyama, M. (forthcoming a). Detection of activities and events without explicit categorization. IPSJ Transactions on Mathematical Modeling and Its Applications.
70. Yamanaka, M., Matsugu, M., & Sugiyama, M. (forthcoming b). Salient object detection based on direct density-ratio estimation. IPSJ Transactions on Mathematical Modeling and Its Applications.