A theoretical understanding of self-paced learning

Information Sciences

Volumn 414, Issue , 2017, Pages 319-328

  Meng, Deyu ^a   Zhao, Qian ^a   Jiang, Lu ^b  

^a XI'AN JIAOTONG UNIVERSITY   (China)

^b Carnegie Mellon University   (United States)

Author keywords

Curriculum learning; Multimedia event detection; Non convex regularized penalty; Self paced learning

Indexed keywords

DATA HANDLING; LEARNING SYSTEMS;

LARGE-SCALE DATA PROCESSING; MINIMIZATION ALGORITHMS; MULTIMEDIA EVENT DETECTIONS; NON-CONVEX REGULARIZED PENALTY; SELF-PACED LEARNING; SMOOTHLY CLIPPED ABSOLUTE DEVIATION; STATE-OF-THE-ART PERFORMANCE; THEORETICAL ARGUMENTS;

PATTERN RECOGNITION;

EID: 85020810893     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2017.05.043     Document Type: Article

References (40)

1. Basu, S., Christensen, J., Teaching classification boundaries to humans. Proceeding of American Association for Artificial Intelligence, 2013, 109–115.
2. Bengio, Y., Louradour, J., Collobert, R., Weston, J., Curriculum learning. International Confenence on Machine Learning, 2009, 41–48.
3. Bradley, P.S., Mangasarian, O.L., Feature selection via concave minimization and support vector machines. International Confenence on Machine Learning, 1998, 82–90.
4. Candès, E.J., Wakin, M.B., Boyd, S.P., Enhancing sparsity by reweighted l1 minimization. J. Fourier Anal. Appl. 14:5–6 (2008), 877–905.
5. Fan, J., Li, R., Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96:456 (2001), 1348–1360.
6. Friedman, J.H., Stochastic gradient boosting. Comput. Stat. Data Anal. 38:4 (2002), 367–378.
7. Gong, P., Zhang, C., Lu, Z., Huang, J., Ye, J., A general iterative shrinkage and thresholding algorithm for non-convex regularized optimization problems. International Confenence on Machine Learning, 2013, 37–45.
8. Jiang, L., Meng, D., Mitamura, T., Hauptmann, A., Easy samples first: self-paced reranking for zeroexample multimedia search. ACM Multimedia, 2014, 547–556.
9. Jiang, L., Meng, D.Y., Yu, S., Lan, Z.Z., Shan, S.G., Hauptman, A., Self-paced learning with diversity. Conference on Neural Information Processing Systems, 2014, 2078–2086.
10. Jiang, L., Meng, D.Y., Zhao, Q., Shan, S.G., Hauptman, A., Self-paced curriculum learning. Proceeding of American Association for Artificial Intelligence, 2015, 2694–2700.
11. Y.G. Jiang, Z. Wu, J. Wang, X. Xue, S.F. Chang, Exploiting feature and class relationships in video categorization with regularized deep neural networks, 2015b, ArXiv preprint: 1502.07209.
12. Y. Kang, Z. Zhang, W. Li, On the global convergence of majorization minimization algorithms for nonconvex optimization problems, 2015, ArXiv preprint: 1504.07791v2.
13. Khan, F., Zhu, X., Mutlu, B., How do humans teach: on curriculum learning and teaching dimension. Conference on Neural Information Processing Systems, 2011, 1449–1457.
14. Krizhevsky, A., Sutskever, I., Hinton, G.E., Imagenet classification with deep convolutional neural networks. Conference on Neural Information Processing Systems, 2012, 1097–1105.
15. Kumar, M., Packer, B., Koller, D., Self-paced learning for latent variable models. Conference on Neural Information Processing Systems, 2010, 1189–1197.
16. Kumar, M., Turki, H., Preston, D., Koller, D., Learning specific-class segmentation from diverse data. International Conference on Computer Vision, 2011, 1800–1807.
17. Lange, K., Hunter, D., Yang, I., Optimization transfer using surrogate objective functions. J. Comput. Graph. Stat. 9:1 (2000), 1–20.
18. A. Lapedriza, H. Pirsiavash, Z. Bylinskii, A. Torralba, Are all training examples equally valuable?, 2013, In arXiv preprint: 1311.6510.
19. Lee, Y., Grauman, K., Learning the easy things first: self-paced visual category discovery. Conference on Computer Vision and Pattern Recognition, 2011, 1721–1728.
20. Liang, J.W., Jiang, L., Meng, D.Y., Hauptman, A., Learning to detect concepts from webly-labeled video data. International Joint Conference on Artificial Intelligence, 2016, 1746–1752.
21. Liang, X., Liu, S., Wei, Y., Liu, L., Lin, S., Yan, L., Towards computational baby learning: a weakly-supervised approach for object detection. International Conference on Computer Vision, 2015, 999–1007.
22. Ni, E., Ling, C., Supervised learning with minimal effort. Advances in Knowledge Discovery and Data Mining, 2010, Springer, 476–487.
23. Pan, Z., Zhang, C., Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression. Pattern Recognit. 48:1 (2015), 231–243.
24. Spitkovsky, V.I., Alshawi, H., Jurafsky, D., Baby steps: how “less is more” in unsupervised dependency parsing. NIPS Workshop: Grammar Induction, Representation of Language and Language Learning, 2009, 1–10.
25. Supančič, J. III, Ramanan, D., Self-paced learning for long-term tracking. Conference on Computer Vision and Pattern Recognition, 2013, 2379–2386.
26. Suzumura, S., Ogawa, K., Sugiyama, M., Takeuchi, I., Outlier path: a homotopy algorithm for robust SVM. International Conference on Machine Learning, 2014, 1098–1106.
27. Tang, K., Ramanathan, V., Li, F., Koller, D., Shifting weights: adapting object detectors from image to video. Conference on Neural Information Processing Systems, 2012, 638–646.
28. Vaida, F., Parameter convergence for EM and MM algorithms. Stat. Sin. 15:3 (2005), 831–840.
29. B. Varadarajan, G. Toderici, S. Vijayanarasimhan, A. Natsev, Efficient large scale video classification, 2015, ArXiv preprint: 1505.06250.
30. Wang, S., Liu, D., Zhang, Z., Nonconvex relaxation approaches to robust matrix recovery. International Joint Conference on Artificial Intelligence, 2013, 1764–1770.
31. Wang, Z., Liu, H., Zhang, T., Optimal computational and statistical rates of convergence for sparse nonconvex learning problems. Ann. Stat. 42 (2014), 2164–2201.
32. Weston, J., Elisseeff, A., Schölkopf, B., Tipping, M., Use of the zero-norm with linear models and kernel methods. J. Mach. Learn. Res. 3 (2003), 1439–1461.
33. Yu, S., Jiang, L., Mao, Z., al, e., CMU-informedia@TRECVID 2014 multimedia eventdetection (MED). TRECVID Video Retrieval Evaluation Workshop, 2014.
34. Zhang, C., Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38:2 (2010), 894–942.
35. Zhang, C., Zhang, T., A general theory of concave regularization for high-dimensional sparse estimation problems. Stat. Sci. 27:4 (2012), 576–593.
36. Zhang, D., Meng, D., Han, J., Co-saliency detection via a self-paced multiple-instance learning framework. International Conference on Computer Vision, 2015, 594–602.
37. Zhang, D., Meng, D., Han, J., Co-saliency detection via a self-paced multiple-instance learning framework. IEEE Trans. Pattern Anal. Mach. Intell., 2017, 10.1109/TPAMI.2016.2567393.
38. Zhang, T., Analysis of multi-stage convex relaxation for sparse regularization. J. Mach. Learn. Res. 11 (2010), 1081–1107.
39. Zhang, Z., Tu, B., Nonconvex penalization using laplace exponents and concave conjugates. Conference on Neural Information Processing Systems, 2012, 611–619.
40. Zhao, Q., Meng, D.Y., Jiang, L., Xie, Q., Xu, Z.B., Hauptman, A., Self-paced learning for matrix factorization. Proceeding of American Association for Artificial Intelligence, 2015, 3196–3202.