Bagging gradient-boosted trees for high precision, low variance ranking models

SIGIR'11 - Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval

Volumn , Issue , 2011, Pages 85-94

  Ganjisaffar, Yasser ^a   Caruana, Rich ^b   Lopes, Cristina Videira ^a  

^a Ringgold Standard Institution Informatics   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Bagging; Learning to rank; Tree ensembles

Indexed keywords

BAGGING; LEARNING METHODS; LEARNING TO RANK; PARAMETER-TUNING; PREDICTIVE PERFORMANCE; TREE ENSEMBLES; VARIANCE RANKING; VARIANCE REDUCTION TECHNIQUES;

FORESTRY;

EID: 80052107008     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2009916.2009932     Document Type: Conference Paper

References (34)

1. Microsoft learning to rank datasets. http://research.microsoft.com/en-us/ projects/mslr/.
2. R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999.
3. L. Breiman. Bagging predictors. Mach. Learn., 24:123-140, August 1996. (Pubitemid 126724382)
4. L. Breiman. Random forests. Machine Learning, 45:5-32, 2001. 10.1023/A:1010933404324. (Pubitemid 32933532)
5. L. Breiman. Using iterated bagging to debias regressions. Mach. Learn., 45:261-277, December 2001. (Pubitemid 33074822)
6. C. Burges. From RankNet to LambdaRank to LambdaMART: An overview. Technical report, Microsoft Research Technical Report MSR-TR-2010-82, 2010.
7. C. J. C. Burges, R. Ragno, and Q. V. Le. Learning to rank with nonsmooth cost functions. In NIPS, pages 193-200, 2006.
8. C. J. C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. N. Hullender. Learning to rank using gradient descent. In ICML, pages 89-96, 2005. (Pubitemid 43183320)
9. Z. Cao, T. Qin, T.-Y. Liu, M.-F. Tsai, and H. Li. Learning to rank: from pairwise approach to listwise approach. In ICML '07: Proceedings of the 24th international conference on Machine learning, pages 129-136, New York, NY, USA, 2007. ACM.
10. O. Chapelle, Y. Chang, and T.-Y. Liu. The Yahoo! learning to rank challenge. http://learningtorankchallenge.yahoo.com, 2010.
11. O. Chapelle and M. Wu. Gradient descent optimization of smoothed information retrieval metrics. Inf. Retr., 13:216-235, June 2010.
12. K. Crammer and Y. Singer. Pranking with ranking. In Advances in Neural Information Processing Systems 14, pages 641-647. MIT Press, 2001.
13. Y. Freund, R. Iyer, R. E. Schapire, and Y. Singer. An efficient boosting algorithm for combining preferences. J. Mach. Learn. Res., 4:933-969, 2003.
14. J. H. Friedman. Stochastic gradient boosting. Technical report, Technical report, Dept. Statistics, Stanford Univ., 1999.
15. J. H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29:1189-1232, 2000. (Pubitemid 33405972)
16. S. Geman, E. Bienenstock, and R. Doursat. Neural networks and the bias/variance dilemma. Neural Comput., 4:1-58, January 1992.
17. R. Herbrich, T. Graepel, and K. Obermayer. Large margin rank boundaries for ordinal regression. In A. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 115-132, Cambridge, MA, 2000. MIT Press.
18. T. K. Ho, J. Hull, and S. Srihari. Decision combination in multiple classifier systems. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 16(1):66-75, Jan. 1994.
19. K. Järvelin and J. Kekäläinen. IR evaluation methods for retrieving highly relevant documents. In SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 41-48, New York, NY, USA, 2000. ACM.
20. T. Joachims. Training linear SVMs in linear time. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '06, pages 217-226, New York, NY, USA, 2006. ACM. (Pubitemid 44535518)
21. P. Li, C. J. C. Burges, and Q. Wu. Mcrank: Learning to rank using multiple classification and gradient boosting. In NIPS, 2007.
22. T.-Y. Liu. Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3):225-331, 2009.
23. D. Y. Pavlov, A. Gorodilov, and C. A. Brunk. BagBoo: a scalable hybrid bagging-the-boosting model. In Proceedings of the 19th ACM international conference on Information and knowledge management, CIKM '10, pages 1897-1900, New York, NY, USA, 2010. ACM.
24. T. Qin, T.-Y. Liu, J. Xu, and H. Li. LETOR: A benchmark collection for research on learning to rank for information retrieval. Information Retrieval, 13:346-374, 2010. 10.1007/s10791-009-9123-y.
25. D. Sculley. Combined regression and ranking. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '10, pages 979-988, New York, NY, USA, 2010. ACM.
26. D. Sorokina, R. Caruana, and M. Riedewald. Additive groves of regression trees. In Proceedings of the 18th European conference on Machine Learning, ECML '07, pages 323-334, Berlin, Heidelberg, 2007. Springer-Verlag.
27. Y. L. Suen, P. Melville, and R. J. Mooney. Combining bias and variance reduction techniques for regression trees. In In Proceedings of the European Conference on Machine Learning (ECML'05), pages 741-749, 2005.
28. M.-F. Tsai, T.-Y. Liu, T. Qin, H.-H. Chen, and W.-Y. Ma. FRank: a ranking method with fidelity loss. In SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 383-390, New York, NY, USA, 2007. ACM. (Pubitemid 350164984)
29. G. Valentini and T. G. Dietterich. Low bias bagged support vector machines. In Proceedings of the International Conference on Machine Learning, ICML'03, pages 752-759. Morgan Kaufmann, 2003.
30. M. N. Volkovs and R. S. Zemel. Boltzrank: learning to maximize expected ranking gain. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pages 1089-1096, New York, NY, USA, 2009. ACM.
31. G. I. Webb. Multiboosting: A technique for combining boosting and wagging. Mach. Learn., 40:159-196, August 2000.
32. Q. Wu, C. Burges, K. Svore, and J. Gao. Ranking, boosting and model adaptation. Technical report, Microsoft Technical Report MSR-TR-2008-109, 2008.
33. J. Xu and H. Li. AdaRank: a boosting algorithm for information retrieval. In SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 391-398, New York, NY, USA, 2007. ACM. (Pubitemid 350164985)
34. Y. Yue, T. Finley, F. Radlinski, and T. Joachims. A support vector method for optimizing average precision. In Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, SIGIR '07, pages 271-278, New York, NY, USA, 2007. ACM. (Pubitemid 350164971)