A user-centric evaluation framework for recommender systems

RecSys'11 - Proceedings of the 5th ACM Conference on Recommender Systems

Volumn , Issue , 2011, Pages 157-164

  Pu, Pearl ^a   Chen, Li ^b   Hu, Rong ^a  

^a EPFL   (Switzerland)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

e commerce recommender; post study questionnaire; quality of user experience; recommender systems

Indexed keywords

BEHAVIORAL INTENTION; DEVELOPMENT RESOURCES; EVALUATION FRAMEWORK; INTERACTION QUALITY; KEY DETERMINANTS; POST-STUDY QUESTIONNAIRE; PSYCHOMETRIC METHODS; USER EXPERIENCE; USER SURVEYS; USER-CENTRIC;

QUALITY CONTROL; SURVEYS;

RECOMMENDER SYSTEMS;

EID: 82555181738     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2043932.2043962     Document Type: Conference Paper

References (26)

1. Adomavicius, G. and Tuzhilin, A. 2005. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734-749. (Pubitemid 40860454)
2. Beenen, G., Ling, K., Wang, X., Chang, K., Frankowski, D., Resnick, P., et al. 2004. Using social psychology to motivate contributions to online communities. In Proceedings of CSCW '04. New York: ACM Press.
3. Chen, L. and Pu, P. 2006. Trust Building with Explanation Interfaces. In Proceedings of International Conference on Intelligent User Interface, 93-100.
4. Chen, L. and Pu, P. 2009. Interaction Design Guidelines on Critiquing-based Recommender Systems. User Modeling and User-Adapted Interaction Journal (UMUAI), Springer Netherlands, Volume 19, Issue3, 167-206.
5. Churchill, G.A. A paradigm for developing better measures of marketing constructs, Journal of Marketing Research 16 (1), 1979, pp. 64-73.
6. Davis, F.D. 1989. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart. 13 319-339.
7. Grabner-Kräuter, S. and Kaluscha, E.A. 2003. Empirical research in on-line trust: a review and critical assessment Int. J. Hum.-Comput. Stud. (IJMMS) 58(6), 783-812.
8. Herlocker, J.L., Konstan, J.A., and Riedl, J. 2000. Explaining collaborative filtering recommendations. CSCW'00, 241-250. (Pubitemid 32400055)
9. Herlocker, J.L., Konstan, J.A., Terveen, L.G., and Riedl, J. 2004. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5-53.
10. Jones, N., and Pu, P. 2007. User Technology Adoption Issues in Recommender Systems. In Proceedings of Networking and Electronic Commerce Research Conference, 379-394.
11. Kirakowski, J. 1993. SUMI: the Software Usability Measurement Inventory. British Journal of Educational Technology, 24 (3) 210-214.
12. Knijnenburg, B. P., Willemsen, M. C., Gantner, Z. and Soncu, H. 2011. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction Journal (UMUAI), Special Issue on User Interfaces for Recommender Systems. (upcoming)
13. McNee, S.M., Lam, S.K., Konstan, J.A., Riedal, J. 2003. Interfaces for eliciting new user preferences in recommender systems. User Modeling 2003, 178-187. (Pubitemid 36768563)
14. McNee, S.M., Riedl, J., and Konstan, J.A. 2006. Being accurate is not enough: How accuracy metrics have hurt recommender systems. CHI Extended Abstracts, 1097-1101.
15. Nunnally, J. C. 1978. Psychometric Theory.
16. Ozok, A.A, Fan, Q., Norcio, A.F. 2010. Design guidelines for effective recommender system interfaces based on a usability criteria conceptual model: results from a college student population. Behaviour & Information Technology, Volume 29, Issue 1, 57 - 83.
17. Peterson, R.A. 1994. A meta-analysis of Cronbach's coefficient alpha. Journal of Consumer Research, Vol. 21 September, pp. 381-91.
18. Pu, P., Chen, L., and Kumar, P. 2008. Evaluating Product Search and Recommender Systems for E-Commerce Environments. Electronic Commerce Research Journal, 8(1-2), 1-27.
19. Pu, P., Zhou, M., and Castagnos, S. 2009. Critiquing Recommenders for Public Taste Products. In proceedings of RecSys'09. New York, ACM Press, 249-252.
20. Pu, P. and Chen, L. 2010. A User-Centric Evaluation Framework of Recommender Systems. In the 3rd ACM Conference on Recommender Systems, Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces, Barcelona, Spain, Sept. 26-30.
21. Simonson, I. 2005. Determinants of customers' responses to customized offers: Conceptual framework and research propositions. Journal of Marketing, 69, 32-45. (Pubitemid 40195117)
22. Swearingen, K. and Sinha, R. 2002. Interaction design for recommender systems. In Interactive Systems (DIS2002).
23. Tintarev, N. and Masthoff, J. 2007. Survey of explanations in recommender systems. ICDE Workshops 2007, 801-810.
24. Venkatesh,V., Morris, M.G., Davis, G.B. and Davis, F.D. 2003. User acceptance of information technology: Toward a unified view. MIS Quarterly, 2003, 27, 3, 425-478.
25. Ziegler, C.N., McNee, S.M., Konstan, J.A., and Lausen, G. 2005. Improving Recommendation Lists through Topic Diversification. In Proc. of WWW 2005, ACM Press (2005), 22-32.
26. Xiao, B. and Benbasat, I. 2007. Ecommerce Product Recommendation Agents: Use, Characteristics, and Impact. Mis Quarterly 31(1), 137-209. (Pubitemid 46382521)