Minimum-entropy estimation in semi-parametric models

Signal Processing

Volumn 85, Issue 5 SPEC. ISS., 2005, Pages 937-949

  Wolsztynski, Eric ^a   Thierry, Eric ^a   Pronzato, Luc ^a  

^a UNIVERSITY OF NICE SOPHIA ANTIPOLIS   (France)

Author keywords

Adaptive estimation; Efficiency; Entropy; Outliers; Parameter estimation; Robustness; Semi parametric models

Indexed keywords

EFFICIENCY; ENTROPY; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; NONLINEAR SYSTEMS; PARAMETER ESTIMATION; PROBABILITY; PROBLEM SOLVING; ROBUSTNESS (CONTROL SYSTEMS);

ADAPTIVE ESTIMATION; NONLINEAR REGRESSION; OUTLIERS; SEMI-PARAMETRIC MODELS;

REGRESSION ANALYSIS;

EID: 14844356926     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2004.11.028     Document Type: Article

References (31)

1. C. Stein, Efficient nonparametric testing and estimation, In: Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probabilities, vol. 1, University of California Press, Berkeley, 1956, pp. 187-196.
2. P. Bickel On adaptive estimation Ann. Stat. 10 1982 647 671
3. J. Begun, W. Hall, W.-M. Huang, and J. Wellner Information and asymptotic efficiency in parametric-non parametric models Ann. Stat. 11 2 1983 432 452
4. R. Beran Asymptotically efficient rank estimates in location models Ann. Stat. 2 1974 63 74
5. C. Stone Adaptive maximum likelihood estimators of a location parameter Ann. Stat. 3 2 1975 267 284
6. C. Manski Adaptive estimation of nonlinear regression models Econometric Rev. 3 2 1984 145 194
7. M.A. Peters, and P.A. Iglesias Minimum entropy control for discrete-time time-varying systems Automatica 33 4 1997 591 605
8. H.P. Wynn, and A.A. Zhigljavsky The Theory of Random Search from a Statistical Viewpoint Test 3 2 1994 1 45
9. L. Pronzato, H.P. Wynn, and A.A. Zhigljavsky Using Renyi Entropies to Measure Uncertainty in Search Problems Lect. Appl. Math. AMS 33 1997 253 268
10. D. Erdogmus, and J. Principe An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems IEEE Trans. Signal Process. 50 7 2002 1780 1786
11. N. Schraudolph Gradient-based manipulation of non-parametric entropy estimates IEEE Trans. Neural Networks 14 2 2004 1 10
12. R. Ash, Information Theory, Wiley, New York, 1965, (Republished by Dover, New York, 1990).
13. L. Pronzato, E. Thierry, Entropy minimization for parameter estimation problems with unknown distribution of the output noise, In: Proceedings of the ICASSP'2001, Salt Lake City, 2001.
14. L. Pronzato, E. Thierry, A minimum-entropy estimator for regression problems with unknown distribution of observation errors, In: A. Mohammad-Djafari (Ed.), Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Proceedings of the 20th International Workshop, Gif-sur-Yvette, France, July 2000. American Institute of Physics, New York, 2001, pp. 169-180.
15. R. Beran An efficient and robust adaptive estimator of location Ann. Stat. 6 2 1978 292 313
16. R. Beran Minimum Hellinger distance estimates for parametric models Ann. Stat. 5 3 1977 445 463
17. R. Pak Minimum Hellinger distance estimation in simple linear regression models distribution and efficiency Stat. Probab. Lett. 26 1996 263 269
18. J. Beirlant, E. Dudewicz, L. Györfi, and E. van der Meulen Nonparametric entropy estimation an overview Internat. J. Math. Stat. Sci. 6 1 1997 17 39
19. L. Le Cam, Théorie Asymptotique de la Décision Statistique, Les Presses de l'Université de Montréal, 1969.
20. D. Andrews Asymptotics for semiparametric econometric models III testing and examples 1989 Cowles Foundation Discussion Paper No. 910 Yale University
21. L. Pronzato, E. Thierry, E. Wolsztynski, Minimum entropy estimation in semi parametric models: a candidate for adaptive estimation? In: A. Di Bucchianico, H. Läuter, H. Wynn (Eds.), mODa'7 - Advances in Model-Oriented Design and Analysis, Proceedings of the Seventh International Workshop, Heeze (Netherlands), Physica Springer Verlag, Heidelberg, 2004, pp. 125-132.
22. E. Schuster Estimation of a probability density function and its derivatives Ann. Math. Stat. 40 1969 1187 1195
23. R. Jennrich Asymptotic properties of nonlinear least squares estimation Ann. Math. Stat. 40 1969 633 643
24. Y. Dmitriev, and F. Tarasenko On the estimation of functionals of the probability density and its derivatives Theory Probab. Appl. 18 3 1973 628 633
25. W. Newey Uniform convergence in probability and stochastic equicontinuity Econometrica 9 4 1991 1161 1167
26. E. Wolsztynski, E. Thierry, L. Pronzato, Consistency of a minimum-entropy estimator of location, Internal Report No I3S/RR-2004-38-FR, Nov. 2004, 30 p, http://www.i3s.unice.fr/ ∼ mh/RR/rapports.html.
27. A. Berlinet, and L. Devroye A comparison of kernel density estimates Publications de l'institut de statistique de l'Université de Paris 38 1994 3 59
28. O. Vasicek A test for normality based on sample entropy J. Royal Stat. Soc. B 38 1 1976 54 59
29. L. Kozachenko, and N. Leonenko On statistical estimation of entropy of random vector Problems Infor. Transmiss. 23 2 1987 95 101 (translated from Problemy Peredachi Informatsii, in Russian, vol. 23, No. 2, pp. 9-16, 1987)
30. M. Broniatowski Estimation through Kullback-Leibler divergence Math. Meth. Stat. 12 4 2003 391 409
31. M. Broniatowski, A. Keziou, Parametric estimation and testing through divergences, Prepublication 2004-1, L.S.T.A., Université Paris 6, 2004.