Nonnegativity constraints in numerical analysis

The Birth of Numerical Analysis

Volumn , Issue , 2009, Pages 109-140

  Chen, Donghui ^a,b   Plemmons, Robert J ^a,c  

^a WAKE FOREST UNIVERSITY   (United States)

^b TUFTS UNIVERSITY   (United States)

^c WAKE FOREST SCHOOL OF MEDICINE   (United States)

Author keywords

Image processing; Matrix and tensor factorizations; Nonnegative least squares; Nonnegativity constraints; Optimization

Indexed keywords

FACTORIZATION; IMAGE PROCESSING; NONLINEAR PROGRAMMING; NUMERICAL ANALYSIS; OPTIMIZATION; SURVEYS; TENSORS;

MODERN APPLICATIONS; NON-LINEAR OPTIMIZATION; NON-NEGATIVITY CONSTRAINTS; NONNEGATIVE LEAST SQUARES; NUMERICAL LINEAR ALGEBRA; SCIENCE AND ENGINEERING; SCIENTIFIC COMPUTATION; TENSOR FACTORIZATION;

MATRIX ALGEBRA;

EID: 84969715286     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1142/9789812836267_0008     Document Type: Chapter

References (86)

1. R.B. Bapat, T.E.S. Raghavan, Nonnegative Matrices and Applications, Cambridge University Press, UK, 1997.
2. M.S. Bartlett, J.R. Movellan, and T.J. Sejnowski, Face recognition by independent component analysis, IEEE Trans. Neural Networks, 13, No. 6, (2002) 1450-1464.
3. P. Belhumeur, J. Hespanha, and D. Kriegman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection, IEEE PAMI, 19, No. 7, (1997) 711-720.
4. S. Bellavia, M. Macconi, and B. Morini, An interior point newton-like method for nonnegative least squares problems with degenerate solution, Numerical Linear-Algebra with Applications, 13, (2006) 825-846.
5. J.R. Bellegarda, A multispan language modelling framework for large vocabulary speech recognition, IEEE Transactions on Speech and Audio Processing, 6 No. 5, (1998) 456-467.
6. A. Berman and R. Plemmons, Rank factorizations of nonnegative matrices, Problems and Solutions, 73-14 (Problem), SIAM Rev., 15 (1973) 655.
7. A. Berman, R. Plemmons, Nonnegative Matrices in the Mathematical Sciences, Academic Press, NY, 1979. Revised version in SIAM Classics in Applied Mathematics, Philadelphia, 1994.
8. M. Berry, M. Browne, A. Langville, P. Pauca, and R. Plemmons, Algorithms and applications for approximate nonnegative matrix factorization, Computational Statistics and Data Analysis, 52, (2007) 155-173. Preprint available at http://www.wfu.edu/~plemmons
9. M.W. Berry, Computational Information Retrieval, SIAM, Philadelphia, 2000.
10. D. Bertsekas, Nonlinear Programming, Athena Scientific, Belmont, MA., 1999.
11. B. Boser, I. Guyon, and V. Vapnik, A training algorithm for optimal margin classifiers, Fifth Annual Workshop on Computational Learning Theory, ACM Press, 1992.
12. R. Bro, S. de Jong, A fast non-negativity-constrained least squares algorithm, Journal of Chemometrics, 11, No. 5, (1997) 393-401.
13. J. Cantarella, M. Piatek, Tsnnls: A solver for large sparse least squares problems with non-negative variables, ArXiv Computer Science e-pri, nts, 2004.
14. X. Chen, L. Gu, S.Z. Li, and H.J. Zhang, Learning representative local features for face detection, in IEEE Conference on Computer Vision and Pattern Recognition, 1, (2001) 1126-1131.
15. M.T. Chu, F. Diele, R. Plemmons, S. Ragni, Optimality, computation, and interpretation of nonnegative matrix factorizations, preprint. Available at: http://www.wfu.edu/~plemmons
16. M.T. Chu and R.J. Plemmons, Nonnegative matrix factorization and applications, Appeared in IMAGE, Bulletin of the International Linear Algebra Society, 34, (2005) 2-7. Available at: http://www.wfu.edu/~plemmons
17. A. Cichocki, R. Zdunek, and S. Amari, Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization, In: Independent Component Analysis, ICA07, London, UK, September 9-12, 2007, Lecture Notes in Computer Science, Vol. LNCS 4666, Springer, (2007) 169-176.
18. C. Cortes, V. Vapnik, Support Vector networks, Machine Learning, 20, (1995) 273-297.
19. A. Dax, On computational aspects of bounded linear least squares problems, ACM Trans. Math. Softw., 17, (1991) 64-73.
20. I.S. Dhillon, D.M. Modha, Concept decompositions for large sparse text data using clustering, Machine Learning J., 42, (2001) 143-175.
21. C. Ding, X. He, and H. Simon, On the equivalence of nonnegative matrix factorization and spectral clustering, Proceedings of the Fifth SIAM International Conference on Data Mining, Newport Beach, CA, 2005.
22. N.K.M. Faber, R. Bro, and P.K. Hopke, Recent developments in CANDE-COMP/PARAFAC algorithms: a critical review, Ch, em, om, etr. Intell. Lab., 65, No. 1, (2003) 119-137.
23. V. Franc, V. Hlavac, and M. Navara, Sequential coordinate-wise algorithm for nonnegative least squares problem, Research report CTU-CMP-2005-06, Center for Machine Perception, Czech Technical University, Prague, Czech Republic, February 2005.
24. P.E. Gill, W. Murray and M.H. Wright, Practical Optimization, Academic, London, 1981.
25. A.A. Giordano, F.M. Hsu, Least Square Estimation With Applications To Digital Signal Processing, John Wiley & Sons, 1985.
26. L. Grippo, M. Sciandrone, On the convergence of the block nonlinear Gauss-Seidel method under convex constraints, Oper. Res. Lett., 26, No. 3, (2000) 127-136.
27. D. Guillamet, J. Vitria, Classifying faces with non-negative matrix factorization, Accepted CCIA 2002, Castello de la Plana, Spain.
28. D. Guillamet, J. Vitria, Non-negative matrix factorization for face recognition, Lecture Notes in Computer Science, 254, (2002) 336-344.
29. M. Hanke, J.G. Nagy and C.R. Vogel, Quasi-newton approach to nonnegative image restorations, Linear Algebra Appl., 316, (2000) 223-236.
30. R.A. Harshman, Foundations of the PARAFAC procedure: models and conditions for an “explantory” multi-modal factor analysis, UCLA working papers in phonetics, Vol. 16, pp. 1-84, 1970.
31. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer-Verlag, New York, 2001.
32. N.-D. Ho, Nonnegative Matrix Factorization Algorithms and Applications, PhD thesis, Univ. Catholique de Louvain, June 2008. (Available from edoc.bib.ucl.ac.be:81/ETD-db/collection/available/BelnUcetd-06052008-235205/).
33. P.K. Hopke, Receptor Modeling in Environmental Chemistry, Wiley and Sons, New York, 1985.
34. P.K. Hopke, Receptor Modeling for Air Quality Management, Elsevier, Amsterdam, Netherlands, 1991.
35. P.O. Hoyer, Nonnegative sparse coding, neural networks for signal processing XII, Proc. IEEE Workshop on Neural Networks for Signal Processing, Martigny, 2002.
36. P. Hoyer, Nonnegative matrix factorization with sparseness constraints, J. of Mach. Leanring Res., 5, (2004) 1457-1469.
37. C.G. Khatri, C.R. Rao, Solutions to some functional equations and their applications to. characterization of probability distributions, Sankhya, 30, (1968) 167-180.
38. B. Kim, Numerical optimization methods for image restoration, Ph.D. thesis, Stanford University, 2002.
39. E. Kim, P.K. Hopke, E.S. Edgerton, Source identification of Atlanta aerosol by positive matrix factorization, J. Air Waste Manage. Assoc., 53, (2003) 731-739.
40. H. Kim, H. Park, Sparse non-negative matrix factorizations via alternating nonnegativity-constrained least squares for microarray data analysis, Bioinformatics, 23, No. 12, (2007) 1495-1502.
41. D. Kim, S. Sra, and I.S. Dhillon, A new projected quasi-newton approach for the nonnegative least squares problem, Dept. of Computer Sciences, The Univ. of Texas at Austin, Technical Report # TR-06-54, Dec. 2006.
42. D. Kim, S. Sra, and I.S. Dhillon, Fast newton-type methods for the least squares nonnegative matrix approximation problem, Statistical Analysis and Data Mining, 1, No. 1, (2008) 38-51.
43. S. Kullback, and R. Leibler, On information and sufficiency, Annals of Mathematical Statistics, 22 (1951) 79-86.
44. A. Langville, C. Meyer, R. Albright, J. Cox, D. Duling, Algorithms, initializations, and convergence for the nonnegative matrix factorization, NCSU Technical Report Math 81706, 2006.
45. C.L. Lawson and R.J. Hanson, Solving Least Squares Problems, Prentice-Hall, 1987.
46. D. Lee and H. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401, (1999) 788-791.
47. D. Lee and H. Seung, Algorithms for nonnegative matrix factorization, Advances in Neural Information Processing Systems, 13, (2001) 556-562.
48. S.Z. Li, X.W. Hou and H.J. Zhang, Learning spatially localized, parts-based representation, IEEE Conference on Computer Vision and Pattern Recognition, (2001) 1-6.
49. C.J. Lin, Projected gradient methods for non-negative matrix factorization, Neural Computation, 19, No. 10, (2007) 2756-2779.
50. W. Liu, J. Yi, Existing and new algorithms for nonnegative matrix factorization, University of Texas at Austin, 2003, report.
51. P. Matstoms, snnls: a matlab toolbox for Solve sparse linear least squarest problem with nonnegativity constraints by an active set method, 2004, available at http://www.math.liu.se/~milun/sls/.
52. A. Mazer, M. Martin, M. Lee and J. Solomon, Image processing software for imaging spectrometry data analysis, Remote Sensing of Environment, 24, (1988) 201-220.
53. J.J. More, G. Toraldo, On the solution of large quadratic programming problems with bound constraints, SIAM Journal on Optimization, 1, No. 1, (1991) 93-113.
54. J.G. Nagy, Z. Strakos, Enforcing nonnegativity in image reconstruction algorithms, in Mathematical Modeling, Estimation, and Imaging, 4121, David C. Wilson, et al, (eds.), (2000) 182-190.
55. J. Nocedal, S. Wright, Numerical Optimization, Springer, Berlin, 2006.
56. M. Novak, R. Mammone, Use of non-negative matrix factorization for language model adaptation in a lecture transcription task, IEEE Workshop on ASRU 2001, pp. 190-193, 2001.
57. P. Paatero and U. Tapper, Positive matrix factorization-a nonnegative factor model with optimal utilization of error-estimates of data value, Environmetrics, 5, (1994) 111-126.
58. P. Paatero, The multilinear engine-a table driven least squares program for solving mutilinear problems, including the n-way parallel factor analysis model, J. Comput. Graphical Statist., 8, No. 4, (1999) 854-888.
59. H. Park, M. Jeon, J.B. Rosen, Lower dimensional representation of text data in vector space based information retrieval, in Computational Information Retrieval, ed. M. Berry, Proc. Comput. Inform. Retrieval Conf., SIAM, (2001) 3-23.
60. V.P. Pauca, F. Shahnaz, M.W. Berry, R.J. Plemmons, Text mining using nonnegative matrix factorizations, In Proc. SIAM Inter. Conf. on Data Mining, Orlando, FL, April 2004.
61. P. Pauca, J. Piper, and R. Plemmons, Nonnegative matrix factorization for spectral data analysis, Lin. Alg. Applic., 416, No. 1, (2006) 29-47.
62. P. Pauca, J. Piper, R. Plemmons, M. Giffin, Object characterization from spectral data using nonnegative factorization and information theory, Proc. AMOS Technical Conf., Maui HI, September 2004. Available at http://www.wfu.edu/~plemmons
63. P. Pauca, R. Plemmons, M. Giffin and K. Hamada, Unmixing spectral data using nonnegative matrix factorization, Proc. AMOS Technical Conference, Maui, HI, September 2004. Available at http://www.wfu.edu/~plemmons
64. E. Polak, Computational Methos in Optimization: A Unified Approach, Academic Press, New York, 1971.
65. L.F. Portugal, J.J. Judice, and L.N. Vicente, A comparison of block pivoting and interior-point algorithms for linear least squares problems with nonnegative variables, Mathematics of Computation, 63, No. 208, (2004) 625-643.
66. M. Powell, An efficient method for finding the minimum of a function of several variables without calculating derivatives, Comput. J. 7, (1964) 155-162.
67. M. Powell, On Search Directions For Minimization, Math. Programming 4, (1973) 193-201.
68. Z. Ramadan, B. Eickhout, X. Song, L.M.C. Buydens, P.K. Hopke, Comparison of positive matrix factorization and multilinear engine for the source apportionment of particulate pollutants, Chemometrics and Intelligent Laboratory Systems. 66, (2003) 15-28.
69. R. Ramath, W. Snyder, and H. Qi, Eigenviews for object recognition in multispectral imaging systems, 32nd Applied Imagery Pattern Recognition Workshop, Washington D.C., pp. 33-38, 2003.
70. M. Rojas, T. Steihaug, Large-Scale optimization techniques for nonnegative image restorations, Proceedings of SPIE, 4791: 233-242, 2002.
71. K. Schittkowski, The numerical solution of constrained linear least-squares problems, IMA Journal of Numerical Analysis, 3, (1983) 11-36.
72. F. Sha, L.K. Saul, D.D. Lee, Multiplicative updates for large margin classifiers, Technical Report MS-CIS-03-12, Department of Computer and Information Science, University of Pennsylvania, 2003.
73. ndInternational Conference on Machine Learning, Bonn, Germany, pp. 792-799, 2005.
74. A. Shashua and A. Levin, Linear image coding for regression and classification using the tensor-rank principal, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2001.
75. L. Shure, Brief History of Nonnegative Least Squares in MATLAB, Blog available at: http://blogs.mathworks.com/loren/2006/.
76. N. Smith, M. Gales, Speech recognition using SVMs, in Advances in Neural and Information Processing Systems, 14, Cambridge, MA, 2002, MIT Press.
77. G. Tomasi, R. Bro, PARAFAC and missing values, Chemometr. Intell. Lab., 75, No. 2, (2005) 163-180.
78. M.A. Turk, A.P. Pentland, Eigenfaces for recognition, Cognitive Neuroscience, 3, No. 1, (1991) 71-86.
79. M.A. Turk, A.P. Pentland, Face recognition using eigenfaces, Proc. IEEE Conference on Computer Vision and Pattern Recognition, Maui, Hawaii, 1991.
80. M.H. van Benthem, M.R. Keenan, Fast algorithm for the solution of large-scale non-negativity constrained least squares problems, Journal of Chemometrics, 18, (2004) 441-450.
81. V. Vapnik, Statistical Learning Theory, Wiley, 1998.
82. V. Vapnik, The Nature of Statistical Learning Theory, Springer-Verlag, 1999.
83. R.S. Varga, Matrix Iterative Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1962.
84. W. Zangwill, Minimizing a function without calculating derivatives. Comput. J., 10, (1967) 293-296.
85. P. Zhang, H. Wang, R. Plemmons, and P. Pauca, Spectral unmixing using nonnegative tensor factorization, Proc. ACM, Conference, Winston-Salem, NC, March 2007.
86. P. Zhang, H. Wang, R. Plemmons, and P. Pauca, Hyperspectral Data Analysis: A Space Object Material Identification Study, Journal of the Optical Soc. Amer., Series, 25, (2008) 3001-3012.