Building cost functions minimizing to some summary statistics

IEEE Transactions on Neural Networks

Volumn 11, Issue 6, 2000, Pages 1263-1271

  Saerens, Marco ^a,b,c  

^a IEEE   (Belgium)

^b UNIVERSITÉ LIBRE DE BRUXELLES   (Belgium)

^c Smals Belgium   (Belgium)

Author keywords

A posteriori probabilities estimation; Absolute deviations; Conditional expectation estimation; Cost function; L 1 approximation; Loss function; Median; Median estimation; Penalty function; Performance criterion; q quantile; Quasi likelihood

Indexed keywords

EID: 0010481518     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.883416     Document Type: Article

References (65)

1. 1-Statistical Procedures and Related Topics, ser. Inst. Math. Statist. Lecture Notes - Monograph Series, Y. Dodge, Ed., 1997, vol. 31.
2. T. Arthanari and Y. Dodge, Mathematical Programming in Statistics. New York: Wiley, 1981.
3. Y. Bard, Nonlinear Parameter Estimation. New York: Academic, 1974.
4. 1 approximation and the analysis of data," Appl. Statist., vol. 17, pp. 51-57, 1968.
5. 1 linear approximation," SIAM J. Numer. Anal., vol. 10, no. 5, pp. 839-848, 1973.
6. 1 norm," Commun. ACM, vol. 17, no. 6, pp. 319-320, 1974.
7. M. Basseville, "Distance measures for signal processing and pattern recognition," Signal Processing, vol. 18, pp. 349-369, 1989.
8. C. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon, 1995.
9. R. Blattberg and T. Sargent, "Regression with non-Gaussian stable disturbances: Some sampling results," Econometrica, vol. 39, no. 3, pp. 501-510, 1971.
10. H. Bourlard and N. Morgan, Connectionist Speech Recognition: A Hybrid Approach. Amsterdam, The Netherlands: Kluwer, 1994.
11. H. Bourlard and C. Wellekens, "Links between Markov models and multilayer perceptrons," in Advances in Neural Information Processing Systems 1, Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 502-510.
12. J. L. Brown, "Asymmetric nonmean-square error criteria," IEEE Trans. Automat. Contr., vol. AC-7, pp. 64-66, 1962.
13. M. Buchinsky, "Changes in the U.S. Wage Structure 1963-1987: An application of quantile regression," Econometrica, vol. 62, pp. 405-458, 1994.
14. P. Burrascano, "A norm selection criterion for the generalized delta rule," IEEE Trans. Neural Networks, vol. 2, pp. 125-130, Jan. 1991.
15. J. Cid-Sueiro, J. Arribas, S. Urban-Munoz, and A. Figueiras-Vidal, "Cost functions to estimate a posteriori probabilities in multiclass problems," IEEE Trans. Neural Networks, vol. 10, pp. 645-656, May 1999.
16. M. De Groot and S. Fienberg, "The comparison and evaluation of forecasters," Statistician, vol. 32, pp. 12-22, 1983.
17. R. Deutsch, Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1965.
18. W. Fisher, "A note on curve fitting with minimum deviations by linear programming," J Amer. Statist. Assoc., vol. 56, pp. 359-362, 1961.
19. I. M. Gelfand and S. V. Fomin, Calculus of Variations. Englewood Cliffs, NJ: Prentice-Hall, 1963.
20. H. Gish, "A probabilistic approach to the understanding and training of neural network classifiers," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (ICASSP), 1990, pp. 1361-1364.
21. L. Gonzales Sotelino, M. Saerens, and H. Bersini, "Classification of temporal trajectories by continuous time recurrent nets," Neural Networks, vol. 7, no. 5, pp. 767-776, 1994.
22. J. B. Hampshire and B. Pearlmutter, "Equivalence proofs for multilayer perceptron classifiers and the Bayesian discriminant function," in Proc. 1990 Connectionnist Models Summer School, D. Touretzky, J. Elman, T. Sejnowski, and G. Hinton, Eds. San Mateo, CA: Morgan Kaufmann, 1990, pp. 159-172.
23. S. Hanson and D. Burr, "Minkowski-r backpropagation: Learning in connectionist models with non-Euclidian error signals," in Neural Information Processing Systems, D. Anderson, Ed. College Park, MD: Amer. Inst. Phys., 1988, pp. 348-357.
24. D. Hosmer and S. Lemeshow, Applied Logistic Regression. New York: Wiley, 1989.
25. P. Huber, Robust Statistics. New York: Wiley, 1981.
26. F. Kanaya and S. Miyake, "Bayes statistical behavior and valid generalization of pattern classifying neural networks," IEEE Trans. Neural Networks, vol. 2, pp. 471-475, July 1991.
27. S. M. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice-Hall, 1993.
28. R. Koenker and G. Bassett, "Regression quantiles," Econometrica, vol. 46, no. 1, pp. 33-50, 1978.
29. R. Koenker and V. D'Orey, "Computing regression quantiles," Appl. Statist., vol. 36, pp. 383-393, 1987.
30. F. Lad, Operational Subjective Statistical Methods. New York: Wiley, 1996.
31. Y. Le Cun, "A theorical framework for backpropagation," in Proc. 1988 Connectionnist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds. San Mateo, CA: Morgan Kaufmann, 1989, pp. 21-28.
32. D. Lindley, "Scoring rules and the inevitability of probability (with discussions)," Int. Statist. Rev., vol. 50, pp. 1-26, 1982.
33. J. Matheson and R. Winkler, "Scoring rules for continuous probability distributions," Management Sci., vol. 22, pp. 1087-1096, 1976.
34. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. London, U.K.: Chapman and Hall, 1990.
35. J. S. Meditch, Stochastic Optimal Linear Estimation and Control. New York: McGraw-Hill, 1969.
36. J. L. Melsa and D. L. Cohn, Decision and Estimation Theory. New York: McGraw-Hill, 1978.
37. J. W. Miller, R. Goodman, and P. Smyth, "Objective functions for probability estimation," in Proc. IEEE Int. Joint Conf. Neural Networks, San Diego, CA, 1991, pp. I-881-886.
38. _, "On loss functions which minimize to conditional expected values and posterior probabilities," IEEE Trans. Inform. Theory, vol. 39, pp. 1404-1408, 1993.
39. N. E. Nahi, Estimation Theory and Applications. New York: Wiley, 1976.
40. M. R. Osborne, Finite Algorithms in Optimization and Data Analysis. New York: Wiley, 1985.
41. 1 approximation," Comput. J., vol. 14, pp. 184-188, 1971.
42. A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3th ed. New York: McGraw-Hill, 1991.
43. R. Pfaffenberger and J. Dinkel, "Absolute deviations curve fitting: An alternative to least squares," in Contributions to Survey Sampling and Applied Statistics, H. A. David, Ed. New York: Academic, 1978, pp. 279-294.
44. 1 Approximation. Cambridge, U.K.: Cambridge Univ. Press, 1989.
45. M. D. Richard and R. P. Lippmann, "Neural-network classifiers estimate Bayesian a posteriori probabilities," Neural Comput., vol. 3, pp. 461-483, 1991.
46. R. Rojas, "A short proof of the posterior probability property of classifier neural networks," Neural Comput., vol. 8, pp. 41-43, 1996.
47. P. Rousseuw and A. Leroy, Robust Regression and Outlier Detection. New York: Wiley, 1988.
48. D. W. Ruck, S. K. Rogers, M. Kabrisky, M. E. Oxley, and B. W. Suter, "The multilayer perceptron as an approximation to a Bayes optimal discriminant function," IEEE Trans. Neural Networks, vol. 1, pp. 296-298, 1990.
49. ∞ minimization via a variant of Karmakar's algorithm," IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 245-253, 1989.
50. M. Saerens, "Nonmean square error criteria for the training of learning machines," in Proc. 13th Int. Conf. Machine Learning (ICML), Bari, Italy, July 1996, pp. 427-434.
51. A. P. Sage and J. L. Melsa, Estimation Theory with Applications to Communications and Control. New York: McGraw-Hill, 1971.
52. S. Santini and A. Del Bimbo, "Recurrent neural networks can be trained to be maximum a posteriori probability classifiers," Neural Networks, vol. 8, no. 1, pp. 25-29, 1995.
53. L. Savage, "Elicitation of personal probabilities and expectations," J. Amer. Statist. Assoc., vol. 66, pp. 783-801, 1971.
54. E. Schlossmacher, "An iterative technique for absolute deviations curve fitting," J. Amer. Statist. Assoc., vol. 68, pp. 857-865, 1973.
55. S. Sherman, "Nonmean-square error criteria," IEEE Trans. Inform. Theory, vol. IT-4, pp. 125-126, 1958.
56. P. A. Shoemaker, "A note on least-squares learning procedures and classification by neural-network models," IEEE Trans. Neural Networks, vol. 2, pp. 158-160, 1991.
57. H. W. Sorenson, Parameter Estimation, Principles and Problems. New York: Marcel Dekker, 1980.
58. L. D. Taylor, "Estimation by minimizing the sum of absolute errors," in Frontiers in Econometrics, P. Zarembka, Ed. New York: Academic, 1974, pp. 169-190.
59. H. M. Wagner, "Linear programming techniques for regression analysis," J. Amer. Statist. Assoc., vol. 54, pp. 206-212, 1959.
60. E. A. Wan, "Neural network classification: A Bayesian interpretation," IEEE Trans. Neural Networks, vol. 1, pp. 303-305, 1990.
61. 1 linear estimation," IEEE Trans. Automat. Contr., vol. AC-29, pp. 67-71, 1984.
62. R. Wedderburn, "Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method," Biometrika, vol. 61, no. 3, pp. 439-447, 1974.
63. H. White, "Learning in artificial neural networks: A statistical perspective," Neural Comput., vol. 1, pp. 425-464, 1989.
64. H. Wilson, "Least squares versus minimum absolute deviations estimation in linear models," Decision Sci., vol. 9, pp. 322-335, 1978.
65. A. Zellner, "Bayesian estimation and prediction using asymmetric loss functions," J. Amer. Statist. Assoc., vol. 81, pp. 446-451, 1986.