Non-convex optimization for machine learning

Foundations and Trends in Machine Learning

Volumn 10, Issue 3-4, 2017, Pages 142-336

  Jain, Prateek ^a   Kar, Purushottam ^b  

^a MICROSOFT RESEARCH   (United States)

^b INDIAN INSTITUTE OF TECHNOLOGY KANPUR   (India)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONVEX OPTIMIZATION; FUNCTIONS; HEURISTIC METHODS; INFERENCE ENGINES; LEARNING ALGORITHMS; LEARNING SYSTEMS; PROBLEM SOLVING;

ALTERNATING MINIMIZATION; HEURISTIC TECHNIQUES; HIGH DIMENSIONAL SPACES; LARGE-SCALE OPTIMIZATION; NONCONVEX OPTIMIZATION; OPTIMIZATION PROBLEMS; STRUCTURAL CONSTRAINTS; TOOLS AND TECHNIQUES;

OPTIMIZATION;

EID: 85037538890     PISSN: 19358237     EISSN: 19358245     Source Type: Journal    
DOI: 10.1561/2200000058     Document Type: Article

References (132)

1. Alekh Agarwal, Sahand N. Negahban, and Martin J. Wainwright. Fast global convergence of gradient methods for high-dimensional statistical recovery. The Annals of Statistics, 40(5):2452-2482, 2012.
2. Alekh Agarwal, Animashree Anandkumar, Prateek Jain, and Praneeth Netrapalli. Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization. SIAM Journal of Optimization, 26(4):2775-2799, 2016.
3. Naman Agarwal, Zeyuan Allen-Zhu, Brian Bullins, Elad Hazan, and Tengyu Ma. Finding Approximate Local Minima Faster than Gradient Descent. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing (STOC), 2017.
4. Animashree Anandkumar and Rong Ge. Efficient approaches for escaping higher order saddle points in non-convex optimization. In Proceedings of the 29th Conference on Learning Theory (COLT), pages 81-102, 2016.
5. Animashree Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. Tensor Decompositions for Learning Latent Variable Models. Journal of Machine Learning Research, 15:2773-2832, 2014.
6. Andreas Andresen and Vladimir Spokoiny. Convergence of an Alternating Maximization Procedure. Journal of Machine Learning Research, 17:1-53, 2016.
7. Sanjeev Arora, Rong Ge, and Ankur Moitra. New Algorithms for Learning Incoherent and Overcomplete Dictionaries. In Proceedings of The 27th Conference on Learning Theory (COLT), 2014.
8. Kamyar Azizzadenesheli, Alessandro Lazaric, and Anima Anandkumar. Reinforcement Learning of POMDPs using Spectral Methods. In Proceedings of the 29th Conference on Learning Theory (COLT), 2016.
9. Sivaraman Balakrishnan, Martin J. Wainwright, and Bin Yu. Statistical Guarantees for the EM Algorithm: From Population to Sample-based Analysis. Annals of Statistics, 45(1):77-120, 2017.
10. Richard Baraniuk, Mark Davenport, Ronald DeVore, and Michael Wakin. A Simple Proof of the Restricted Isometry Property for Random Matrices. Constructive Approximation, 28(3):253-263, 2008.
11. Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, 3rd edition, 2016.
12. Kush Bhatia, Prateek Jain, and Purushottam Kar. Robust Regression via Hard Thresholding. In Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS), 2015.
13. Kush Bhatia, Prateek Jain, Parameswaran Kamalaruban, and Purushottam Kar. Consistent Robust Regression. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), 2017.
14. Srinadh Bhojanapalli and Prateek Jain. Universal Matrix Completion. In Proceedings of the 31st International Conference on Machine Learning (ICML), 2014.
15. Thomas Blumensath. Sampling and Reconstructing Signals From a Union of Linear Subspaces. IEEE Transactions on Information Theory, 57(7): 4660-4671, 2011.
16. Jean Bourgain, Stephen Dilworth, Kevin Ford, Sergei Konyagin, and Denka Kutzarova. Explicit constructions of RIP matrices and related problems. Duke Mathematical Journal, 159(1):145-185, 2011.
17. Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
18. Alon Brutzkus and Amir Globerson. Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017.
19. Sebastien Bubeck. Convex Optimization: Algorithms and Complexity. Foundations and Trends® in Machine Learning, 8(34):231-357, 2015.
20. Jian-Feng Cai, Emmanuel J. Candès, and Zuowei Shen. A Singular Value Thresholding Algorithm for Matrix Completion. SIAM Journal of Optimization, 20(4):1956-1982, 2010.
21. Emmanuel Candès and Terence Tao. Decoding by Linear Programming. IEEE Transactions on Information Theory, 51(12):4203-4215, 2005.
22. Emmanuel J. Candès. The Restricted Isometry Property and Its Implications for Compressed Sensing. Comptes Rendus Mathematique, 346(9-10):589-592, 2008.
23. Emmanuel J. Candès and Xiaodong Li. Solving Quadratic Equations via PhaseLift When There Are About as Many Equations as Unknowns. Foundations of Computational Mathematics, 14(5):1017-1026, 2014.
24. Emmanuel J. Candès and Benjamin Recht. Exact Matrix Completion via Convex Optimization. Foundations of Computational Mathematics, 9(6): 717-772, 2009.
25. Emmanuel J. Candès and Terence Tao. The power of convex relaxation: Near-optimal matrix completion. IEEE Transactions on Information Theory, 56 (5):2053-2080, 2009.
26. Emmanuel J. Candès, Justin K. Romberg, and Terence Tao. Stable Signal Recovery from Incomplete and Inaccurate Measurements. Communications on Pure and Applied Mathematics, 59(8):1207-1223, 2006.
27. Emmanuel J. Candès, Xiaodong Li, and Mahdi Soltanolkotabi. Phase Retrieval via Wirtinger Flow: Theory and Algorithms. IEEE Transactions on Information Theory, 61(4):1985-2007, 2015.
28. Yair Carmon, John C. Duchi, Oliver Hinder, and Aaron Sidford. "Convex Until Proven Guilty": Dimension-Free Acceleration of Gradient Descent on Non-Convex Functions. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017.
29. Rick Chartrand. Exact Reconstruction of Sparse Signals via Nonconvex Minimization. IEEE Information Processing Letters, 14(10):707-710, 2007.
30. Caihua Chen and Bingsheng He. Matrix Completion via an Alternating Direction Method. IMA Journal of Numerical Analysis, 32(1):227-245, 2012.
31. Laming Chen and Yuantao Gu. Local and global optimality of LP minimization for sparse recovery. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015.
32. Yin Chen and Arnak S. Dalalyan. Fused sparsity and robust estimation for linear models with unknown variance. In Proceedings of the 26th Annual Conference on Neural Information Processing Systems (NIPS), 2012.
33. Yudong Chen, Constantine Caramanis, and Shie Mannor. Robust Sparse Regression under Adversarial Corruption. In Proceedings of the 30th International Conference on Machine Learning (ICML), 2013.
34. Yudong Chen, Huan Xu, Constantine Caramanis, and Sujay Sanghavi. Matrix Completion with Column Manipulation: Near-Optimal Sample-Robustness-Rank Tradeoffs. IEEE Transactions on Information Theory, 62(1):503-526, 2016.
35. Yeshwanth Cherapanamjeri, Kartik Gupta, and Prateek Jain. Nearly-optimal Robust Matrix Completion. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017.
36. Anna Choromanska, Mikael Hena, Michael Mathieu, Gérard Ben Arous, and Yann LeCun. The Loss Surfaces of Multilayer Networks. In Proceedings of the 18th International Conference on Arti cial Intelligence and Statistics (AISTATS), 2015.
37. Albert Cohen, Wolfgang Dahmen, and Ronald DeVore. Compressed Sensing and Best k-term Approximation. Journal of the American Mathematical Society, 22(1):211-231, 2009.
38. Christophe Croux and Kristel Joossens. Robust Estimation of the Vector Autoregressive Model by a Least Trimmed Squares Procedure. In Proceedings in Computational Statistics (COMPSTAT), 2008.
39. Yann N. Dauphin, Razvan Pascanu, Çaglar Gülçehre, Kyunghyun Cho, Surya Ganguli, and Yoshua Bengio. Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In Proceedings of the 28th Annual Conference on Neural Information Processing Systems (NIPS), pages 2933-2941, 2014.
40. Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1-38, 1977.
41. David L. Donoho. Compressed Sensing. IEEE Transactions on Information Theory, 52(4):1289-1306, 2006.
42. David L. Donoho, Arian Maleki, and Andrea Montanari. Message Passing Algorithms for Compressed Sensing: I. Motivation and Construction. Proceedings of the National Academy of Sciences USA, 106(45):18914-18919, 2009.
43. 1-Ball for Learning in High Dimensions. In Proceedings of the 25th International Conference on Machine Learning (ICML), 2008.
44. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research, 9:1871-1874, 2008.
45. Maryam Fazel, Ting Kei Pong, Defeng Sun, and Paul Tseng. Hankel matrix rank minimization with applications in system identification and realization. SIAM Journal on Matrix Analysis and Applications, 34(3):946-977, 2013.
46. Martin A. Fischler and Robert C. Bolles. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Communications of the ACM, 24(6):381-395, 1981.
47. 1-minimization. Applied and Computational Harmonic Analysis, 29(1):97-103, 2010.
48. Simon Foucart. Hard Thresholding Pursuit: an Algorithm for Compressive Sensing. SIAM Journal on Numerical Analysis, 49(6):2543-2563, 2011.
49. q-minimization for 0 < q ≤ 1. Applied and Computational Harmonic Analysis, 26(3):395-407, 2009.
50. Rosalind Franklin and Raymond G. Gosling. Evidence for 2-Chain Helix in Crystalline Structure of Sodium Deoxyribonucleate. Nature, 172:156-157, 1953a.
51. Rosalind Franklin and Raymond G. Gosling. Molecular Configuration in Sodium Thymonucleate. Nature, 171:740-741, 1953b.
52. Rahul Garg and Rohit Khandekar. Gradient Descent with Sparsification: An iterative algorithm for sparse recovery with restricted isometry property. In Proceedings of the 26th International Conference on Machine Learning (ICML), 2009.
53. Rong Ge, Furong Huang, Chi Jin, and Yang Yuan. Escaping From Saddle Points - Online Stochastic Gradient for Tensor Decomposition. In Proceedings of The 28th Conference on Learning Theory (COLT), pages 797-842, 2015.
54. Rong Ge, Jason D. Lee, and Tengyu Ma. Matrix Completion has No Spurious Local Minimum. In Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NIPS), 2016.
55. R. W. Gerchberg and W. Owen Saxton. A Practical Algorithm for the Determination of Phase from Image and Diffraction Plane Pictures. Optik, 35 (2):237-246, 1972.
56. Surbhi Goel and Adam Klivans. Learning Depth-Three Neural Networks in Polynomial Time. arXiv:1709.06010v1 [cs.DS], 2017.
57. Donald Goldfarb and Shiqian Ma. Convergence of Fixed-Point Continuation Algorithms for Matrix Rank Minimization. Foundations of Computational Mathematics, 11(2):183-210, 2011.
58. Gene H. Golub and Charles F. Van Loan. Matrix Computations. Johns Hopkins Studies in Mathematical Sciences. The John Hopkins University Press, 3rd edition, 1996.
59. Rémi Gribonval, Rodolphe Jenatton, and Francis Bach. Sparse and Spurious: Dictionary Learning With Noise and Outliers. IEEE Transaction on Information Theory, 61(11):6298-6319, 2015.
60. Moritz Hardt. Understanding Alternating Minimization for Matrix Completion. In Proceedings of the 55th IEEE Annual Symposium on Foundations of Computer Science (FOCS), 2014.
61. Moritz Hardt and Mary Wootters. Fast Matrix Completion Without the Condition Number. In Proceedings of The 27th Conference on Learning Theory (COLT), 2014.
62. Moritz Hardt, Raghu Meka, Prasad Raghavendra, and Benjamin Weitz. Computational limits for matrix completion. In Proceedings of The 27th Conference on Learning Theory (COLT), 2014.
63. Trevor Hastie, Robert Tibshirani, and Martin Wainwright. Statistical Learning with Sparsity: The Lasso and Generalizations. Number 143 in Monographs on Statistics and Applied Probability. The CRC Press, 2016.
64. Ishay Haviv and Oded Regev. The Restricted Isometry Property of Subsampled Fourier Matrices. In Bo'az Klartag and Emanuel Milman, editors, Geometric Aspects of Functional Analysis, Volume 2169 of Lecture Notes in Mathematics, pages 163-179. Springer, Cham, 2017.
65. Peter J. Huber and Elvezio M. Ronchetti. Robust Statistics. Wiley Series in Probability and Statistics. John Wiley & Sons, 2nd edition, 2009.
66. Kishore Jaganathan, Samet Oymak, and Babak Hassibi. Sparse Phase Retrieval: Convex Algorithms and Limitations. In Proceedings of the IEEE International Symposium on Information Theory (ISIT), 2013.
67. Prateek Jain and Praneeth Netrapalli. Fast Exact Matrix Completion with Finite Samples. In Proceedings of The 28th Conference on Learning Theory (COLT), 2015.
68. Prateek Jain and Ambuj Tewari. Alternating Minimization for Regression Problems with Vector-valued Outputs. In Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS), 2015.
69. Prateek Jain, Raghu Meka, and Inderjit Dhillon. Guaranteed Rank Minimization via Singular Value Projections. In Proceedings of the 24th Annual Conference on Neural Information Processing Systems (NIPS), 2010.
70. Prateek Jain, Ambuj Tewari, and Inderjit S. Dhillon. Orthogonal Matching Pursuit with Replacement. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS), 2011.
71. Prateek Jain, Praneeth Netrapalli, and Sujay Sanghavi. Low-rank Matrix Completion using Alternating Minimization. In Proceedings of the 45th annual ACM Symposium on Theory of Computing (STOC), pages 665-674, 2013.
72. Prateek Jain, Ambuj Tewari, and Purushottam Kar. On Iterative Hard Thresholding Methods for High-dimensional M-Estimation. In Proceedings of the 28th Annual Conference on Neural Information Processing Systems (NIPS), 2014.
73. Ali Jalali, Christopher C. Johnson, and Pradeep D. Ravikumar. On Learning Discrete Graphical Models using Greedy Methods. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS), pages 1935-1943, 2011.
74. Chi Jin, Rong Ge, Praneeth Netrapalli, Sham M. Kakade, and Michael I. Jordan. How to Escape Saddle Points Efficiently. In Proceedings of the 34th International Conference on Machine Learning (ICML), pages 1724-1732, 2017.
75. Boris Sergeevich Kashin. The diameters of octahedra. Uspekhi Matematicheskikh Nauk, 30(4(184)):251-252, 1975.
76. Raghunandan H. Keshavan. Efficient Algorithms for Collaborative Filtering. Ph.D. Thesis, Stanford University, 2012.
77. Raghunandan H. Keshavan, Andrea Montanari, and Sewoong Oh. Matrix Completion from a Few Entries. IEEE Transactions on Information Theory, 56(6):2980-2998, 2010.
78. Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix Factorization Techniques for. Recommender Systems. IEEE Computer, 42(8):30-37, 2009.
79. Jason D. Lee, Max Simchowitz, Michael I. Jordan, and Benjamin Recht. Gradient Descent Only Converges to Minimizers. In Proceedings of the 29th Conference on Learning Theory (COLT), pages 1246-1257, 2016.
80. Kiryung Lee and Yoram Bresler. ADMiRA: Atomic Decomposition for Minimum Rank Approximation. IEEE Transactions on Information Theory, 56 (9):4402-4416, 2010.
81. Yuanzhi Li and Yang Yuan. Convergence Analysis of Two-layer Neural Networks with ReLU Activation. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NIPS), 2017.
82. Stuart P. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2):129-137, 1982.
83. Amand A. Lucas. A-DNA and B-DNA: Comparing Their Historical X-ray Fiber Diffraction Images. Journal of Chemical Education, 85(5):737-743, 2008.
84. Zhi-Quan Luo and Paul Tseng. On the Convergence of the Coordinate Descent Method for Convex Differentiable Minimization. Journal of Optimization Theory and Applications, 72(1):7-35, 1992.
85. Zhi-Quan Luo and Paul Tseng. Error bounds and convergence analysis of feasible descent methods: A general approach. Annals of Operations Research, 46(1):157-178, 1993.
86. Ricardo A. Maronna, R. Douglas Martin, and Victor J. Yoha. Robust Statistics: Theory and Methods. John Wiley, 2006.
87. R. Douglas Martin and Judy Zeh. Robust Generalized M-estimates for Autoregressive Parameters: Small-sample Behavior and Applications. Technical Report 214, University of Washington, 1978.
88. Raghu Meka, Prateek Jain, Constantine Caramanis, and Inderjit Dhillon. Rank Minimization via Online Learning. In Proceedings of the 25th International Conference on Machine Learning (ICML), 2008.
89. Balas Kausik Natarajan. Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24(2):227-234, 1995.
90. Deanna Needell and Joel A. Tropp. CoSaMP: Iterative Signal Recovery from Incomplete and Inaccurate Samples. Applied and Computational Harmonic Analysis, 26:301-321, 2008.
91. Sahand N. Negahban, Pradeep Ravikumar, Martin J. Wainwright, and Bin Yu. A Unified Framework for High-Dimensional Analysis of M-Estimators with Decomposable Regularizers. Statistical Science, 27(4):538-557, 2012.
92. Jelani Nelson, Eric Price, and Mary Wootters. New constructions of RIP matrices with fast multiplication and fewer rows. In Proceedings of the 25th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2014.
93. Yurii Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Kluwer-Academic, 2003.
94. Yurii Nesterov. Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems. SIAM Journal of Optimization, 22(2):341-362, 2012.
95. Yurii Nesterov and B.T. Polyak. Cubic regularization of Newton method and its global performance. Mathematical Programming, 108(1):177-205, 2006.
96. Praneeth Netrapalli, Prateek Jain, and Sujay Sanghavi. Phase Retrieval using Alternating Minimization. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems (NIPS), 2013.
97. Nam H. Nguyen and Trac D. Tran. Exact recoverability from dense corrupted observations via L1 minimization. IEEE Transactions on Information Theory, 59(4):2036-2058, 2013a.
98. Nam H. Nguyen and Trac D. Tran. Robust Lasso With Missing and Grossly Corrupted Observations. IEEE Transaction on Information Theory, 59(4): 2036-2058, 2013b.
99. Samet Oymak, Benjamin Recht, and Mahdi Soltanolkotabi. Sharp Time-Data Tradeoffs for Linear Inverse Problems. arXiv:1507.04793 [cs.IT], 2015.
100. Garvesh Raskutti, Martin J. Wainwright, and Bin Yu. Restricted Eigenvalue Properties for Correlated Gaussian Designs. Journal of Machine Learning Research, 11:2241-2259, 2010.
101. Benjamin Recht. A Simpler Approach to Matrix Completion. Journal of Machine Learning Research, 12:3413-3430, 2011.
102. Benjamin Recht, Maryam Fazel, and Pablo A. Parrilo. Guaranteed Minimum Rank Solutions to Linear Matrix Equations via Nuclear Norm Minimization. SIAM Review, 52(3):471-501, 2010.
103. Sashank Reddi, Ahmed Hefny, Suvrit Sra, Barnabas Poczos, and Alexander J. Smola. Fast Stochastic Methods for Nonsmooth Nonconvex Optimization. In Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016.
104. Peter J. Rousseeuw. Least Median of Squares Regression. Journal of the American Statistical Association, 79(388):871-880, 1984.
105. Peter J. Rousseeuw and Annick M. Leroy. Robust Regression and Outlier Detection. John Wiley and Sons, 1987.
106. Ankan Saha and Ambuj Tewari. On the Non-asymptotic Convergence of Cyclic Coordinate Descent Methods. SIAM Journal on Optimization, 23 (1):576-601, 2013.
107. Hanie Sedghi and Anima Anandkumar. Training Input-Output Recurrent Neural Networks through Spectral Methods. arXiv:1603.00954 [CS.LG], 2016.
108. Shai Shalev-Shwartz and Tong Zhang. Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization. Journal of Machine Learning Research, 14:567-599, 2013.
109. Yiyuan She and Art B. Owen. Outlier Detection Using Nonconvex Penalized Regression. Journal of the American Statistical Association, 106(494):626-639, 2011.
110. Daniel A. Spielman, Huan Wang, and John Wright. Exact Recovery of Sparsely-Used Dictionaries. In Proceedings of the 25th Annual Conference on Learning Theory (COLT), 2012.
111. Suvrit Sra, Sebastian Nowozin, and Stephen J. Wright, editors. Optimization for Machine Learning. The MIT Press, 2011.
112. Norbert Stockinger and Rudolf Dutter. Robust time series analysis: A survey. Kybernetika, 23(7):1-3, 1987.
113. Ju Sun, Qing Qu, and John Wright. When Are Nonconvex Problems Not Scary? arXiv:1510.06096 [math.OC], 2015.
114. Ruoyu Sun and Zhi-Quan Lu. Guaranteed Matrix Completion via Non-convex Factorization. In Proceedings of the 56th IEEE Annual Symposium on Foundations of Computer Science (FOCS), 2015.
115. Ambuj Tewari, Pradeep Ravikumar, and Inderjit S. Dhillon. Greedy Algorithms for Structurally Constrained High Dimensional Problems. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS), 2011.
116. Joel A. Tropp. User-Friendly Tail Bounds for Sums of Random Matrices. Foundations of Computational Mathematics, 12(4):389-434, 2012.
117. Joel A. Tropp and Anna C. Gilbert. Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit. IEEE Transactions on Information Theory, 53(12):4655-4666, Dec. 2007. ISSN 0018-9448.
118. pMinimization (0 < p ≤ 1). IEEE Transactions on Information Theory, 57(11):7255-7278, 2011.
119. Zhaoran Wang, Quanquan Gu, Yang Ning, and Han Liu. High Dimensional EM Algorithm: Statistical Optimization and Asymptotic Normality. In Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS), 2015.
120. Karen H.S. Wilson, Sarah E. Eckenrode, Quan-Zhen Li, Qing-Guo Ruan, Ping Yang, Jing-Da Shi, Abdoreza Davoodi-Semiromi, Richard A. McIndoe, Byron P. Croker, and Jin-Xiong She. Microarray Analysis of Gene Expression in the Kidneys of New- and Post-Onset Diabetic NOD Mice. Diabetes, 52 (8):2151-2159, 2003.
121. John Wright, Alan Y. Yang, Arvind Ganesh, S. Shankar Sastry, and Yi Ma. Robust Face Recognition via Sparse Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(2):210-227, 2009.
122. Stephen J. Wright and Jorge Nocedal. Numerical Optimization, Volume 2. Springer New York, 1999.
123. C.-F. Jeff Wu. On the Convergence Properties of the EM Algorithm. The Annals of Statistics, 11(1):95-103, 1983.
124. 1-Minimization Algorithms for Robust Face Recognition. IEEE Transactions on Image Processing, 22(8):3234-3246, 2013.
125. Fanny Yang, Sivaraman Balakrishnan, and Martin J. Wainwright. Statistical and computational guarantees for the Baum-Welch algorithm. In Proceedings of the 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), 2015.
126. Xinyang Yi and Constantine Caramanis. Regularized EM Algorithms: A Unified Framework and Statistical Guarantees. In Proceedings of the 29th Annual Conference on Neural Information Processing Systems (NIPS), 2015.
127. Xinyang Yi, Constantine Caramanis, and Sujay Sanghavi. Alternating Minimization for Mixed Linear Regression. In Proceedings of the 31st International Conference on Machine Learning (ICML), 2014.
128. Ya-Xiang Yuan. Recent advances in trust region algorithms. Mathematical Programming, 151(1):249-281, 2015.
129. Tong Zhang. Adaptive Forward-Backward Greedy Algorithm for Learning Sparse Representations. IEEE Transactions on Information Theory, 57: 4689-4708, 2011.
130. Yuchen Zhang, Percy Liang, and Moses Charikar. A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics. In Proceedings of the 30th Conference on Learning Theory, 2017.
131. Kai Zhong, Zhao Song, Prateek Jain, Peter L. Bartlett, and Inderjit S. Dhillon. Recovery Guarantees for One-hidden-layer Neural Networks. In Proceedings of the 34th International Conference on Machine Learning (ICML), 2017.
132. Yunhong Zhou, Dennis Wilkinson, Robert Schreiber, and Rong Pan. Large-scale Parallel Collaborative Filtering for the Netflix Prize. In Proceedings of the 4th International Conference on Algorithmic Aspects in Information and Management (AAIM), 2008.