Ranking with non-random missing ratings: Influence of popularity and positivity on evaluation metrics

RecSys'12 - Proceedings of the 6th ACM Conference on Recommender Systems

Volumn , Issue , 2012, Pages 147-154

  Pradel, Bruno ^a   Usunier, Nicolas ^a   Gallinari, Patrick ^a  

^a UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

Evaluation; Ranking; Recommender systems

Indexed keywords

DATA SETS; EVALUATION; EVALUATION METRICS; MISSING DATA; PREDICTION TASKS; RANKING; SKEWED DISTRIBUTION;

STATISTICAL TESTS;

RECOMMENDER SYSTEMS;

EID: 84867391520     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2365952.2365982     Document Type: Conference Paper

References (14)

1. J. Bennett, S. Lanning, and N. Netix. The netix prize. In In KDD Cup and Workshop in conjunction with KDD, 2007.
2. P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-n recommendation tasks. In RecSys '10, pages 39-46, 2010.
3. A. Gunawardana and G. Shani. A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res., 10:2935-2962, 2009.
4. J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating Collaborative Filtering Recommender Systems. ACM Trans. on Inf. Sys., 22(1):5-53, 2004.
5. K. Järvelin and J. Kekäläinen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst., 20(4):422-446, Oct. 2002. (Pubitemid 44642296)
6. Y. Koren, R. Bell, and C. Volinsky. Matrix Factorization Techniques for Recommender Systems. Computer, 42(8):30-37, 2009.
7. B. Marlin. Collaborative Filtering: A Machine Learning Perspective. Master's thesis, University of Toronto, 2004.
8. B. Marlin, R. Zemel, S. Roweis, and M. Slaney. Collaborative filtering and the missing at random assumption. In UAI-07, pages 267-275, 2007.
9. B. M. Marlin and R. S. Zemel. Collaborative prediction and ranking with non-random missing data. In RecSys'09, pages 5-12, 2009.
10. MovieLens data. homepage: http://www.grouplens.org/node/73, 2006.
11. H. Steck. Training and testing of recommender systems on data missing not at random. In KDD '10, pages 713-722, 2010.
12. H. Steck. Item popularity and recommendation accuracy. In RecSys '11, pages 125-132, 2011.
13. M. Weimer, A. Karatzoglou, Q. V. Le, and A. J. Smola. Cofi rank - maximum margin matrix factorization for collaborative ranking. In NIPS, 2007.
14. Yahoo! Webscope Datasets. Yahoo! Music ratings for User Selected and Randomly Selected songs, v1.0, http://research.yahoo.com/AcademicRelations.