Reduction from cost-sensitive ordinal ranking to weighted binary classification

Neural Computation

Volumn 24, Issue 5, 2012, Pages 1329-1367

  Lin, Hsuan Tien ^a   Li, ng ^b  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; LEARNING; PHYSIOLOGY; STATISTICAL ANALYSIS;

ALGORITHMS; DATA INTERPRETATION, STATISTICAL; LEARNING; MODELS, BIOLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 84861176005     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00265     Document Type: Article

References (57)

1. Abe, N., Zadrozny, B.,&Langford, J. (2004). An iterativemethod formulti-class costsensitive learning. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 3-11). New York: ACM.
2. Agarwal, S. (2008). Generalization bounds for some ordinal regression algorithms. In Proceedings of the 19th Conference on Algorithmic Learning Theory (pp. 7-21). Berlin: Springer.
3. Agresti, A. (2002). Categorical data analysis (2nd ed.). Hoboken, NJ: Wiley.
4. Ailon, N., & Mohri, M. (2008). An efficient reduction of ranking to classification. In Learning Theory: 21st Annual Conference on Learning Theory (pp. 87-98). Berlin: Springer.
5. Anderson, J. A. (1984). Regression and ordered categorical variables. Journal of the Royal Statistical Society, Series B, 46, 1-30.
6. Balcan, M.-F., Bansal, N., Beygelzimer, A., Coppersmith, D., Langford, J., & Sorkin, G. B. (2007). Robust reductions from ranking to classification. In Learning Theory: 20th Annual Conference on Learning Theory (pp. 604-619). Berlin: Springer.
7. Bartlett, P. L., & Shawe-Taylor, J. (1998). Generalization performance of support vector machines and other pattern classifiers. In B. Schölkopf, C.J.C. Burges, & A. J. Smola (Eds.), Advances in kernel methods: Support vector learning (pp. 43-54). Cambridge, MA: MIT Press.
8. Beygelzimer, A., Daniand, V., Hayes, T., Langford, J., & Zadrozny, B. (2005). Error limiting reductions between classification tasks. In Machine Learning: Proceedings of the 22nd International Conference (pp. 49-56). Madison, WI: Omnipress.
9. Beygelzimer, A., Langford, J., & Ravikumar, P. (2007). Multiclass classification with filter trees. Available online at http://hunch.net/~jl.
10. Cao, Z., Qin, T., Liu, T.-Y., Tsai, M.-F., & Li, H. (2007). Learning to rank: From pairwise approach to listwise approach. In Machine Learning: Proceedings of the 24th International Conference (pp. 129-136). Madison, WI: Omnipress.
11. Cardoso, J. S., & da Costa, J. F. P. (2007). Learning to classify ordinal data: The data replication method. Journal of Machine Learning Research, 8, 1393-1429.
12. Chang, C.-C., & Lin, C.-J. (2001). LIBSVM: A library for support vector machines. National Taiwan University. Available online at http://www.csie.ntu.edu .tw/~cjlin/libsvm.
13. Chang, K.-Y., Chen, C.-S., & Hung, Y.-P. (2011). Ordinal hyperplanes ranker with cost sensitivities for age estimation. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition (pp. 585-592). Piscataway, NJ: IEEE.
14. Chu, W., & Ghahramani, Z. (2005). Gaussian processes for ordinal regression. Journal of Machine Learning Research, 6, 1019-1041.
15. Chu, W., & Keerthi, S. S. (2007). Support vector ordinal regression. Neural Computation, 19, 792-815.
16. Cossock, D., & Zhang, T. (2008). Statistical analysis of Bayes optimal subset ranking. IEEE Transactions on Information Theory, 54, 4140-5154.
17. Crammer, K., & Singer, Y. (2005). Online ranking by projecting. Neural Computation, 17, 145-175.
18. Dembczyński, K., Kotłowski, W., & Słowiński, R. (2008). Ordinal classification with decision rules. In Proceedings of the 3rd International Workshop on Mining Complex Data (pp. 169-181). Berlin: Springer.
19. Figueira, J., Greco, S., & Ehrgott, M. (Eds.). (2005). Multiple criteria decision analysis: State of the art surveys. Berlin: Springer-Verlag.
20. Frank, E., & Hall, M. (2001). A simple approach to ordinal classification. In Machine Learning: Proceedings of the 12th European Conference onMachine Learning (pp. 145-156).
21. Freund, Y., & Schapire, R. E. (1999). Large margin classification using the perceptron algorithm. Machine Learning, 37(3), 277-296.
22. Greco, S., Słowiński, R., &Matarazzo, B. (2000). Extension of the rough set approach to multicriteria decision support. European Journal of Operational Research, 38, 161-195.
23. Har-Peled, S., Roth, D.,&Zimak, D. (2003).Constraint classification:Anew approach to multiclass classification and ranking. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems, 15 (pp. 365-379). Cambridge, MA: MIT Press.
24. Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: Data mining, inference, and prediction. Berlin: Springer-Verlag.
25. Herbrich, R., Graepel, T., & Obermayer, K. (2000). Large margin rank boundaries for ordinal regression. In A. J. Smola, P. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers (pp. 115-132).
26. Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI repository of machine learning databases. Available online at http://www.ics.uci.edu/~mlearn/MLRepository.html.
27. Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301), 13-30.
28. Holte, R. C. (1993). Very simple classification rules perform well on most commonly used datasets. Machine Learning, 11(1), 63-91.
29. Hsu, C.-W., Chang, C.-C., & Lin, C.-J. (2003). A practical guide to support vector classification (Tech. Rep.). Taipei: National Taiwan University.
30. Joachims, T. (2006). Training linear SVMs in linear time. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 217-226). New York: ACM.
31. Kearns, M. J., & Vazirani, U. V. (1994). An introduction to computational learning theory. Cambridge, MA: MIT Press.
32. Kotłowski, W., & Słowiński, R. (2009). Rule learning with monotonicity constraints. In Machine Learning: Proceedings of the 26th International Conference (pp. 537-544). Berlin: Springer.
33. Li, L.,&Lin, H.-T. (2007a).Optimizing 0/1 loss for perceptrons by random coordinate descent. In Proceedings of the 2007 International Joint Conference on Neural Networks (pp. 749-754). Piscataway, NJ: IEEE.
34. Li, L.,&Lin, H.-T. (2007b). Ordinal regression by extended binary classification. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems, 19 (pp. 865-872). Cambridge, MA: MIT Press.
35. Li, P., Burges, C., & Wu, Q. (2008). McRank: Learning to rank using multiple classification and gradient boosting. In J. C. Platt, D. Koller, Y. Singer, & S. Roweis (Eds.), Advances in neural information processing systems, 20 (pp. 897-904).
36. Lin, H.-T. (2008). From ordinal ranking to binary classification. Unpublished doctoral dissertation, California Institute of Technology.
37. Lin, H.-T., & Li, L. (2006). Large-margin thresholded ensembles for ordinal regression: Theory and practice. In Proceedings of the 17th Conference on Algorithmic Learning Theory (pp. 319-333). Berlin: Springer.
38. Lin, H.-T., & Li, L. (2008). Support vector machinery for infinite ensemble learning. Journal of Machine Learning Research, 9, 285-312.
39. Lin, H.-T., & Li, L. (2009). Combining ordinal preferences by boosting. In Preference Learning Workshop at ECML/PKDD. Berlin: Springer.
40. Liu, T.-Y. (2009). Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3), 225-331.
41. McCullagh, P. (1980). Regression models for ordinal data. Journal of theRoyal Statistical Society. Series B, 42, 109-142.
42. Meir, R., & Rätsch, G. (2003). An introduction to boosting and leveraging. In O. Bousquet, U. von Luxburg, & G. Rätsch (Eds.), Advanced Lectures on Machine Learning (pp. 119-184). Berlin: Springer.
43. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1(1), 81-106.
44. Rajaram, S., Garg, A., Zhou, X. S., & Huang, T. S. (2003). Classification approach towards ranking and sorting problems. In Machine Learning: Proceedings of the 14th European Conference on Machine Learning (pp. 301-312). Berlin: Springer.
45. Rennie, J. D. M., & Srebro, N. (2005). Loss functions for preference levels: Regression with discrete ordered labels. In Proceedings of the IJCAI MultidisciplinaryWorkshop on Advances in Preference Handling (pp. 180-186). Norwell, MA: Kluwer.
46. Rosenblatt, F. (1962). Principles of neurodynamics: Perceptrons and the theory of brain mechanisms. N.p.: Spartan Books.
47. Schapire, R. E., Freund, Y., Bartlett, P. L., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics, 26(5), 1651-1686.
48. Schapire, R. E.,&Singer, Y. (1999). Improved boosting algorithms:Using confidencerated predictions. Machine Learning, 37(3), 297-336.
49. Schölkopf, B., & Smola, A. (2002). Learning with kernels. Cambridge, MA: MIT Press.
50. Shashua, A., & Levin, A. (2003). Ranking with large margin principle: Two approaches. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems, 15 (pp. 937-944). Cambridge, MA: MIT Press.
51. Sill, J. (1998). Monotonic networks. In M. Jordan, M. Kearns, & S. Solla (Eds.), Advances in neural information processing systems, 10 (pp. 661-667). Cambridge, MA: MIT Press.
52. Słowiński, R., Greco, S., & Matarazzo, B. (2007). Dominance-based rough set approach to reasoning about ordinal data. In Proceedings of the International Conference on Rough Sets and Intelligent Systems Paradigms (pp. 5-11). Berlin: Springer.
53. Tsai, M.-F., Chen, S.-T., Chen, Y.-N., Ferng, C.-S., Wang, C.-H., Wen, T.-Y., et al. (2010). An ensemble ranking solution to the Yahoo! learning to rank challenge (Tech. Rep.). Taipei: National Taiwan University.
54. Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27(11), 1134-1142.
55. Vapnik, V. N. (1995). The nature of statistical learning theory. Berlin: Springer-Verlag.
56. Xia, F., Zhou, L., Yang, Y., & Zhang, W. (2007). Ordinal regression as multiclass classification. International Journal of Intelligent Control and Systems, 12(3), 230-236.
57. Zadrozny, B., Langford, J., & Abe, N. (2003). Cost sensitive learning by costproportionate example weighting. In Proceedings of the 3rd IEEE International Conference on Data Mining (pp. 435-442). Piscataway, NJ: IEEE.