CROC: A new evaluation criterion for recommender systems

Electronic Commerce Research

Volumn 5, Issue 1, 2005, Pages 51-74

  Schein, Andrew I ^a   Popescul, Alexandrin ^a   Ungar, Lyle H ^a   Pennock, David M ^b  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b Overture Services Inc ^*  (United States)

Author keywords

Collaborative filtering; Performance evaluation; Recommender systems

Indexed keywords

COLLABORATIVE FILTERING;

AREAS UNDER THE CURVES; EMPIRICAL STUDIES; EVALUATION CRITERIA; PERFORMANCE; PERFORMANCE CHARACTERISTICS; PERFORMANCES EVALUATION; ROC CURVES; SYSTEM ALGORITHM; SYSTEM EVALUATION; TWO DOMAINS;

RECOMMENDER SYSTEMS;

EID: 27444441370     PISSN: 13895753     EISSN: 15729362     Source Type: Journal    
DOI: 10.1023/B:ELEC.0000045973.51289.8c     Document Type: Conference Paper

References (36)

1. Basu, C., H. Hirsh, and W. Cohen. (1998). "Recommendation as Classification: Using Social and Content-Based Information in Recommendation." In Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 714-720.
2. Billsus, D. and M.J. Pazzani. (1998). "Learning Collaborative Information Filters." In Proceedings of the 15th International Conference on Machine Learning, pp. 46-54.
3. Breese, J.S., D. Heckerman, and C. Kadie. (1998). "Empirical Analysis of Predictive Algorithms for Collaborative Filtering." In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 43-52.
4. Claypool, M., A. Gokhale, and T. Miranda. (1999). "Combining Content-Based and Collaborative Filters in an Online Newspaper." In Proceedings of the ACM SIGIR Workshop on Recommender Systems -Implementation and Evaluation.
5. Cohen, W.W., R.E. Schapire, and Y. Singer. (1999). "Learning to Order Things." Journal of Artificial Intelligence Research 10, 243-270.
6. Condliff, M.K., D.D. Lewis, D. Madigan, and C. Posse. (1999). "Bayesian Mixed-Effect Models for Recommender Systems." In ACM SIGIR '99 Workshop on Recommender Systems: Algorithms and Evaluation.
7. Feller, W. (1990). Introduction to Probability Theory and Its Application, Vol. 1. Wiley, 3rd edition.
8. Freund, Y., R. Iyer, R.E. Schapire, and Y. Singer. (1998). "An Efficient Boosting Algorithm for Combining Preferences." In Proceedings of the 15th International Conference on Machine Learning, pp. 170-178.
9. González-Caro, C.N., M.L. Calderón-Benavides, J. de J. Pérez-Alcázar, J.C. García-Díaz, and J. Delgado. (2002). "Towards a More Comprehensive Comparison of Collaborative Filtering Algorithms." In Proceedings of the 2002 String Processing and Information Retrieval Conference (SPIRE 2002), pp. 248-253.
10. Good, N., J.B. Schafer, J.A. Konstan, A. Borchers, B.M. Sarwar, J.L. Herlocker, and J. Riedl. (1999). "Combining Collaborative Filtering with Personal Agents for Better Recommendations." In Proceedings of the 16th National Conference on Artificial Intelligence, pp. 439-146.
11. Guo, H. (1997). "SOAP: Live Recommendations through Social Agents." In 5th DELOS Workshop on Filtering and Collaborative Filtering, Budapest.
12. Heckerman, D., D.M. Chickering, C. Meek, R. Rounthwaite, and C. Kadie. (2000). "Dependency Networks for Collaborative Filtering and Data Visualization." In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp. 264-273.
13. Herlocker, J.L., J.A. Konstan, A. Borchers, and J. Riedl. (1999). "An Algorithmic Framework for Performing Collaborative Filtering." In Proceedings of the Conference on Research and Development in Information Retrieval.
14. Hill, W., L. Stead, M. Rosenstein, and G. Furnas. (1995). "Recommending and Evaluating Choices in a Virtual Community of Use." In Proceedings of the Conference on Human Factors in Computing Systems, pp. 194-201.
15. Hofmann, T. (1999). "Probabilistic Latent Semantic Indexing." In 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 50-57.
16. Hofmann, T. and J. Puzicha. (1999). "Latent Class Models for Collaborative Filtering." In Proceedings of the 16th International Joint Conference on Artificial Intelligence, pp. 688-693.
17. Konstan, J.A., B.N. Miller, D. Maltz, J.L. Herlocker, L.R. Gordon, and J. Riedl. (1997). "GroupLens: Applying Collaborative Filtering to Usenet News." Communications of the ACM 40(3). 77-87.
18. Melville, P., R.J. Mooney, and R. Nagarajan. (2001). "Content- Boosted Collaborative Filtering." In Proceedings of the 2001 SIGIR Workshop on Recommender Systems.
19. Mooney, R.J. and L. Roy. (2000). "Content-Based Book Recommending Using Learning for Text Categorization." In Proceedings of the 5th ACM Conference on Digital Libraries, pp. 195-204.
20. Nakamura, A. and N. Abe. (1998). "Collaborative Filtering Using Weighted Majority Prediction Algorithms." In Proceedings of the 15th International Conference on Machine Learning, pp. 395-403.
21. Pennock, D.M., E. Horvitz, and C.L. Giles. (2000). "Social Choice Theory and Recommender Systems: Analysis of the Axiomatic Foundations of Collaborative Filtering." In Proceedings of the 17th National Conference on Artificial Intelligence, pp. 729-734.
22. Pennock, D.M., E. Horvitz, S. Lawrence, and C.L. Giles. (2000). "Collaborative Filtering by Personality Diagnosis: A Hybrid Memory- and Model-Based Approach." In Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, pp. 473-1-80.
23. Popescul, A., L.H. Ungar, D.M. Pennock, and S. Lawrence. (2001). "Probabilistic Models for Unified Collaborative and Content-Based Recommendation in Sparse-Data Environments." In Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence.
24. Resnick, P., N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl. (1994). "GroupLens: An Open Architecture for Collaborative Filtering of Netnews." In Proceedings of the ACM Conference on Computer Supported Cooperative Work, pp. 175-186.
25. Resnick, P. and H.R. Varian. (1997). "Reeommender Systems." Communications of the ACM 40(3). 56-58.
26. Salton, G. and M. McGill. (1983). Introduction to Modern Information Retrieval. McGraw-Hill.
27. Sarwar, B.M., G. Karypis, J.A. Konstan. and J.T. Riedl. (2000). "Analysis of Recommender Algorithms for E-Commerce." In Proceedings of the 2nd ACM Conference on Electronic Commerce, pp. 158-167.
28. Sarwar, B.M., G. Karypis, J.A. Konstan, and J.T. Riedl. (2000). "Application of Dimensionality Reduction in Recommender System - A Case Study." In ACM WebKDD Web Mining for E-Commerce Workshop.
29. Sarwar, B.M., G. Karypis, J.A. Konstan, and J.T. Riedl. (2001). "Item-Based Collaborative Filtering Recommendation Algorithms," In Proceedings of the 10th International World Wide Web Conference, pp. 285-295.
30. Schein, A.I., A. Popescul, and L.H. Ungar. (2001). "PennAspect: A Two-Way Aspect Model Implementation." Technical Report MS-CIS-01-25. Department of Computer and Information Science, The University of Pennsylvania.
31. Schein, A.I., A. Popescul, L.H. Ungar, and D.M. Pennock. (2001). "Generative Models for Cold-Start Recommendations." In Proceedings of the 2001 SIGIR Workshop on Recommender Systems.
32. Schein, A.I., A. Popescul, L.H. Ungar, and D.M. Pennock. (2002). "Methods and Metrics for Cold-Start Recommendations." In Proceedings of the 25th International ACM Conference on Research and Development in Information Retrieval.
33. Schein, A.I., L.K. Saul, and L.H. Ungar. (2003). "A Generalized Linear Model for Principal Component Analysis of Binary Data." In Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics.
34. Shardanand, U. and P. Maes. (1995.) "Social Information Filtering: Algorithms for Automating 'Word of Mouth'." In Proceedings of the Conference on Human Factors in Computing Systems, pp. 210-217.
35. Swets, J.A. (1988). "Measuring the Accuracy of Diagnostic Systems." Science 240, 1285-1293.
36. Ungar, L.H. and D.P. Foster. (1998). "Clustering Methods for Collaborative Filtering." In Workshop on Recommendation Systems at the 15th National Conference on Artificial Intelligence.