Penalized maximum-likelihood estimation, the Baum-Welch algorithm, diagonal balancing of symmetric matrices and applications to training acoustic data

Journal of Computational and Applied Mathematics

Volumn 119, Issue 1-2, 2000, Pages 301-331

  Micchelli, Charles A ^a   Olsen, Peder ^b  

^a STATE UNIVERSITY OF NEW YORK   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; MARKOV PROCESSES; MATHEMATICAL MODELS; MATRIX ALGEBRA; OPTIMIZATION; PROBLEM SOLVING; SET THEORY; SPEECH RECOGNITION; STATISTICAL METHODS; THEOREM PROVING;

BAUM-WELCH ALGORITHM; HIDDEN MARKOV MODEL (HMM); NONPARAMETRIC DENSITY ESTIMATION; PENALIZED MAXIMUM LIKELIHOOD ESTIMATION (MLE);

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 0040194031     PISSN: 03770427     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0377-0427(00)00385-X     Document Type: Article

References (58)

1. Bacharach M. Biproportional Matrices and Input-Output Change, Monograph, Vol. 16. 1970;Cambridge University Press, Cambridge.
2. L.R. Bahl, P.F. Brown, P.V. de Souza, R.L. Mercer, D. Nahamoo, A fast algorithm for deleted interpolation, Proceedings Eurospeech, Vol. 3, 1991, pp. 1209-1212.
3. L.R. Bahl, P.V. Desouza, P.S. Gopalkrishnan, M.A. Picheny, Context dependent vector quantization for continuous speech recognition, Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, 1993, pp. 632-635.
4. S. Basu, C.A. Micchelli, Maximum likelihood estimation for acoustic vectors in speech recognition, in Advanced Black-Box Techniques For Nonlinear Modeling: Theory and Applications, Kluwer Publishers, Dordrecht, 1998, to appear.
5. S. Basu, C.A. Micchelli, P. Olsen, Power exponential densities for the training and classification of acoustic feature vectors in speech recognition, preprint, 1999, to appear.
6. Baum L.E., Eagon J.A. An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model of ecology. Bull. Amer. Math. Soc. 73:1967;360-363.
7. Baum L.E., Petrie T., Soules G., Weiss N. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist. 41(1):1970;164-171.
8. C. Berge, A. Ghouile-Houri, Programming, Games and Transportation Networks, Wiley, New York, 1965, pp. 65-66.
9. Bergman L.M. Proof of the convergence of Sheleikhovskii's method for a problem with transportation constraints. USSR Comput. Math. Math. Phys. 1(1):1967;191-204.
10. L. Breiman, Classification and Regression Trees, Wadsworth International, Belmont, CA, 1983.
11. Brualdi S., Parter S., Schneider H. The diagonal equivalence of a non-negative matrix to a stochastic matrix. J. Math. Anal. Appl. 16:1966;31-50.
12. Cacoullos T. Estimates of a multivariate density. Ann. Inst. Statist. Math. 18:1966;178-189.
13. Cavaretta A.S., Micchelli C.A. Design of curves and surfaces by subdivision algorithms. Lyche T., Schumaker L. Mathematical Methods in Computer Aided Geometric Design. 1989;115-153 Academic Press, Boston.
14. Csima J., Datta B.N. The DAD theorem for symmetric non-negative matrices. J. Combin. Theory. 12(A):1972;147-152.
15. Dempster A.P., Liard N.M., Baum D.B. Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. 39(B):1977;1-38.
16. 1 View:1985;Wiley, New York.
17. Engel G.M., Schneider H. Matrices diagonally similar to a symmetric matrix. Linear Algebra Appl. 29:1980;131-138.
18. Engel G.M., Schneider H. Algorithms for testing the diagonal similarity of matrices and related problems. SIAM J. Algorithms Discrete Math. 3(4):1982;429-438.
19. Franklin J., Lorenz J. On the scaling of multidimensional matrices. Linear Algebra Appl. 114 / 115:1989;717-735.
20. Good I.J., Gaskin R.A. Nonparametric roughness penalties for probability densities. Biometrika. 58:1971;255-277.
21. R.A. Gopinath, Constrained maximum likelihood modeling with Gaussian distributions, Broadcast News Transcription and Understanding Workshop, 1998.
22. Goulden I.P., Jackson D.M. Combinatorial Enumeration. 1983;Wiley, New York.
23. C. Gu, Multivariate function estimation: some automatic density and hazard estimates, preprint, 1998.
24. C. Gu, Penalized likelihood estimation: convergence under correct model, preprint, 1998.
25. Hershkowitz D., Rothblum U.G., Schneider H. Classification of nonnegative matrices using diagonal equivalence. SIAM J. Matrix Anal. Appl. 9(4):1988;455-460.
26. Horn R.A., Johnson C.R. Matrix Analysis. 1985;Cambridge University Press, Cambridge, UK.
27. Jelinek F. Statistical Methods for Speech Recognition. 1998;The MIT Press, Cambridge MA.
28. Karlin S., Nirenberg L. On a theorem of P. Nowosad. Math. Anal. Appl. 17:1967;61-67.
29. Liporace L.A. Maximum likelihood estimation for multivariate observation of Markov sources. IEEE Trans. Inform. Theory. 5:1982;729-734.
30. M. Marcus, M. Newman, The permanent of a symmetric matrix, Abstract 587-85. Amer. Math. Soc. Notices 8 595.
31. Marshall A.W., Olkin I. Scaling of matrices to achieve specified row and column sums. Numer. Math. 12:1968;83-90.
32. Menon M.V., Schneider H. The spectrum of a nonlinear operator associated with a matrix. Linear Algebra Appl. 2:1969;321-334.
33. C.A. Micchelli, Mathematical Aspects of Geometric Modeling, CBMSF-NSF Regional Conference Series in Applied Mathematics, Vol. 65, SIAM, Philadelphia, 1995.
34. Micchelli C.A., Pinkus A. Some remarks on nonnegative polynomials on polyhedra. Anderson T.W., Athreya K.B., Iglehart D.L. Probability, Statistics and Mathematics: Papers in Honor of Samuel Karlin. 1989;163-186 Academic Press, San Diego.
35. Micchelli C.A., Utreras F.I. Smoothing and interpolation in a convex subset of a Hilbert Space. SIAM J. Sci. Statist. Comput. 9:1988;728-746.
36. Micchelli C.A., Utreras F.I. Smoothing and interpolation in a convex subset of a semi-Hilbert Space. Math. Modelling Numer. Methods. 4:1991;425-440.
37. Novosad P. On the integral equation. Kf=1/f arising in a problem in communication J. Math. Anal. Appl. 14:1966;484-492.
38. Parzen E. On the estimation of a probability density function and the mode. Ann. Math. Statist. 33:1962;1065-1076.
39. Raghavan T.E.S. On pairs of multidimensional matrices. Linear Algebra Appl. 62:1984;263-268.
40. Rosenblatt M. Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27:1956;832-837.
41. Rothblum U.G. Generalized scalings satisfying linear equations. Linear Algebra Appl. 114 / 115:1989;765-783.
42. Rothblum U.G., Schneider H. Characterization of optimal scaling of matrices. Math. Programm. 19:1980;121-136.
43. Rothblum U.G., Schneider H. Scalings of matrices which have prescribed row sums and column sums via optimization. Linear Algebra Appl. 114 / 115:1989;737-764.
44. Rothblum U.G., Schneider H., Schneider M.H. Scaling matrices to prescribed row and column maxima. SIAM J. Matrix Anal. Appl. 15:1994;1-14.
45. S. Saitoh, Theory of Reproducing Kernels and its Applications, Pilman Research Notes in Mathematical Analysis, Vol. 189, Longman Scientific and Technical, Essex, UK, 1988.
46. Saunders B.D., Schneider H. Flows on graphs applied to diagonal similarity and diagonal equivalence for matrices. Discrete Math. 24:1978;205-220.
47. Saunders B.D., Schneider H. Cones, graphs and optimal scaling of matrices. Linear Multilinear Algebra. 8:1979;121-135.
48. Schneider M.H. Matrix scaling, entropy minimization and conjugate duality. I Existence conditions. Linear Algebra Appl. 114 / 115:1989;785-813.
49. Scott D. Multivariate Density Estimation: Theory, Practice and Visualization. 1992;Wiley, New York.
50. Silverman B.W. Density Estimation for Statistics and Data Analysis. 1986;Chapman & Hall, / CRC New York.
51. Silverman B.W. On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Statist. 10(4):1992;795-810.
52. Sinkhorn R. A relationship between arbitrary positive matrices and doubly stochastic matrices. Ann. Math. Statist. 38:1964;439-455.
53. Sinkhorn R., Knopp P. Concerning nonnegative matrices and doubly stochastic matrices. Pacific J. Math. 212:1967;343-348.
54. Thompson J.R., Tapia R. Nonparametric Probability Density Estimation. 1978;The John Hopkins University Press, Baltimore.
55. Thompson J.R., Tapia R. Nonparametric Function Estimation, Modeling and Simulation. 1970;SIAM, Philadelphia.
56. K. Tok, Primal-dual path-following algorithms for determinant maximization problems with linear matrix inequalities, Computational Optimization and Applications, Kluwer Academic Publishers, Boston, to appear.
57. Wahba G. Spline Models for Observational Data, CBMS-NSF Regional Conference series in Applied Mathematics, Vol. 59. 1990;SIAM, Philadelphia.
58. Whittle S. Optimization under Constraints. 1971;Wiley Interscience, New York.