Error analysis of stochastic gradient descent ranking

IEEE Transactions on Cybernetics

Volumn 43, Issue 3, 2013, Pages 898-909

  Chen, Hong ^a,b   Tang, Yi ^c   Li, Luoqing ^d   Yuan, Yuan ^c   Li, Xuelong ^c   Tang, Yuanyan ^b  

^a HUAZHONG AGRICULTURAL UNIVERSITY   (China)

^b UNIVERSITY OF MACAU   (Macao)

^c XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

^d HUBEI UNIVERSITY   (China)

Author keywords

Error analysis; Integral operator; Ranking; Reproducing kernel Hilbert space; Sampling operator; Stochastic gradient descent

Indexed keywords

ANALYSIS TECHNIQUES; INTEGRAL OPERATORS; RANKING; RANKING FUNCTIONS; REGULARIZATION PARAMETERS; REPRODUCING KERNEL HILBERT SPACES; STOCHASTIC GRADIENT DESCENT; STOCHASTIC GRADIENT DESCENT ALGORITHM;

LEARNING ALGORITHMS; MATHEMATICAL OPERATORS; TIME VARYING NETWORKS;

ERROR ANALYSIS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; INFORMATION RETRIEVAL; METHODOLOGY; STATISTICAL MODEL; STATISTICS; MARKOV CHAIN; PROCEDURES;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; STOCHASTIC PROCESSES;

EID: 84890428197     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2012.2217957     Document Type: Article

References (31)

1. M. Varewyck and J.-P. Martens, "A practical approach to model selection for support vector machines with a Gaussian kernel," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 2, pp. 330-340, Apr. 2011.
2. L. Chen, C. L. Philip Chen, and W. Pedrycz, "A gradient-descent- based approach for transparent linguistic interface generation in fuzzy models," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 40, no. 5, pp. 1219-1230, Oct. 2010.
3. F. Nie, D. Xu, X. Li, and S. Xiang, "Semisupervised dimensionality reduction and classification through virtual label regression," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 3, pp. 675-685, Jun. 2011.
4. Y. Tang, L. Q. Li, and X. Li, "Learning similarity with multikernel method," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 1, pp. 131-138, Feb. 2011.
5. S. Agarwal, T. Graepel, R. Herbrich, S. Har-Peled, and D. Roth, "Generalization bounds for the area under the ROC curve," J. Mach. Learn. Res., vol. 6, no. 4, pp. 393-425, Apr. 2005.
6. S. Agarwal and P. Niyogi, "Generalization bounds for ranking algorithms via algorithmic stability," J. Mach. Learn. Res., vol. 10, no. 2, pp. 441-474, Feb. 2009.
7. S. Agarwal, "Learning to rank on graphs," Mach. Learn., vol. 81, no. 3, pp. 333-357, Dec. 2010.
8. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, no. 3, pp. 337-404, May 1950.
9. S. Boyd and L. Vandenberghe, Convex Optimization. New York: Cambridge Univ. Press, 2004.
10. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender, "Learning to rank using gradient descent," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 89-96.
11. C. Burges, R. Ragno, and Q. Le, "Learning to rank with nonsmooth cost functions," in Advances in Neural Information Processing Systems 19, B. Schölkopf, J. Platt, and T. Hoffman, Eds. Cambridge, MA: MIT Press, 2007.
12. J. Cai, H. Wang, and D. X. Zhou, "Gradient learning in a classification setting by gradient descent," J. Approx. Theory, vol. 161, no. 2, pp. 674-692, Dec. 2009.
13. O. Chapelle, Y. Chang, and T. Y. Liu, "Future directions in learning to rank," in Proc. JMLR: Workshop Conf., 2011, vol. 14, pp. 91-100.
14. H. Chen and L. Q. Li, "Semi-supervised multi-category classification with imperfect model," IEEE Trans. Neural Netw., vol. 20, no. 10, pp. 1594-1603, Oct. 2009.
15. H. Chen, L. Q. Li, and J. T. Peng, "Error bounds of multi-graph regularized semi-supervised classification," Inf. Sci., vol. 179, no. 12, pp. 1960-1969, May 2009.
16. W. Chen, T. Y. Liu, and Z. Ma, "Two-layer generalization analysis for ranking using rademacher average," in Proc. Adv. Neural Inf. Process. Syst. 23, 2010, pp. 1-9.
17. S. Clemencon, G. Luogosi, and N. Vayatis, "Ranking and empirical minimization of U-satistics," Ann. Statist., vol. 36, no. 2, pp. 844-874, Apr. 2008.
18. D. Cossock and T. Zhang, "Statistical analysis of bayes optimal subset ranking," IEEE Trans. Inf. Theory, vol. 54, no. 11, pp. 5140-5154, Nov. 2008.
19. K. Crammer and Y. Singer, "Pranking with ranking," in Proc. Adv. Neural Inf. Process. Syst., 2002, vol. 14, pp. 641-647.
20. C. Cortes, M. Mohri, and A. Rastogi, "Magnitude-preserving ranking algorithms," in Proc. 24th Int. Conf. Mach. Learn., 2007, pp. 169-176.
21. C. Cortes, M. Mohri, and A. Rastogi, "An alternative ranking problem for search engines," in Proc. 6th WEA, Heidelberg, Germany, 2007, pp. 1-22.
22. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bull. Amer. Math. Soc., vol. 39, no. 1, pp. 1-49, Jan. 2002.
23. F. Cucker and D. X. Zhou, Learning Theory: An Approximation Theory Viewpoint . Cambridge, U.K.: Cambridge Univ. Press, 2007.
24. X. Dong and D. R. Chen, "Learning rates of gradient descent algorithm for classification," J. Comput. Appl. Math., vol. 224, no. 1, pp. 182-192, Feb. 2009.
25. X. Dong and D. X. Zhou, "Learning gradient by a gradient descent algorithm," J. Math. Anal. Appl., vol. 341, no. 2, pp. 1018-1027, May 2008.
26. Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer, "An efficient boosting algorithm for combining preferences," J. Mach. Learn. Res., vol. 4, no. 6, pp. 933-969, Nov. 2003.
27. R. Herbrich, T. Graepel, P. Bollmann-Sdorra, and K. Obermayer, "Learning preference relations for information retrieval," in Proc. Conf. Mach. Learn. Workshop Learn. Text Categorization, 1998, pp. 80-84.
28. R. Herbrich, T. Graepel, and K. Obermayer, "Large margin rank boundaries for ordinal regression," in Advances in Large Margin Classifiers. Cambridge, MA: MIT Press, 2000, pp. 115-132.
29. X. Li, S. Lin, S. Yan, and D. Xu, "Discriminant locally linear embedding with high-order tensor data," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 38, no. 2, pp. 342-352, Apr. 2008.
30. S. Mukherjee and D. X. Zhou, "Learning coordinate covariances via gradients," J. Mach. Learn. Res., vol. 7, no. 3, pp. 519-549, Mar. 2006.
31. V. C. Raykar, R. Duraiswami, and B. Krishnapuram, "A fast algorithm for learning a ranking function from large-scale data sets," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 7, pp. 1158-1170, Jul. 2008.