Time-aware recommender systems: A comprehensive survey and analysis of existing evaluation protocols

User Modeling and User-Adapted Interaction

Volumn 24, Issue 1-2, 2014, Pages 67-119

  Campos, Pedro G ^a,b   Díez, Fernando ^b   Cantador, Iván ^b  

^a UNIVERSIDAD DEL BÍO BÍO   (Chile)

^b UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

Author keywords

Context aware recommender systems; Evaluation methodologies; Survey; Time aware recommender systems

Indexed keywords

EID: 84894047367     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-012-9136-x     Document Type: Article

References (100)

1. Adomavicius, G.; Sankaranarayanan, R.; Sen, S.; Tuzhilin, A.: Incorporating contextual information in recommender systems using a multidimensional approach. ACM Trans. Inf. Syst. 23(1), 103-145 (2005)
2. Adomavicius, G.; Tuzhilin, A.: Multidimensional recommender systems: a data warehousing approach. Second International Workshop on Electronic Commerce, pp. 180-192, Heidelberg (2001).
3. Adomavicius, G.; Tuzhilin, A.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734-749 (2005)
4. Adomavicius, G.; Tuzhilin, A.: Context-aware recommender systems. In: Ricci, F.; Rokach, L.; Shapira, B.; Kantor, P.B. (eds.), Recommender Systems Handbook. ISBN 978-0-387-85820-3 (2011)
5. Ardissono, L.; Gena, C.; Torasso, P.; Bellifemine, F.; Difino, A.; Negro, B.: User modeling and recommendation techniques for personalized electronic program guides. Personalized Digital Television. Springer. ISBN 978-1-4020-2164-0 (2004).
6. Arlot, S.; Celisse, A.: A survey of cross-validation procedures for model selection. Stat. Surv. 4, 40-79 (2010)
7. Baeza-Yates, R.A.; Ribeiro-Neto, B.: Modern Information Retrieval. ACM Press, New York. ISBN 020139829X (1999).
8. Baltrunas, L.: Context-aware collaborative filtering recommender systems. PhD Dissertation, Free University of Bozen-Bolzano (2011)
9. Baltrunas, L.; Amatriain, X.: Towards time-dependant recommendation based on implicit feedback. RecSys 2009 Workshop on Context-Aware Recommender Systems. New York, NY (2009).
10. Baltrunas, L.; Ricci, F.: Experimental evaluation of context-dependent collaborative filtering using item splitting. User Model. User-Adapt. Interact.; Special issue on Context-Aware Recommender Systems (2013)
11. Bell, R.M.; Koren, Y.: Scalable collaborative filtering with fointly derived neighborhood interpolation weights. Seventh IEEE International Conference on Data Mining, pp. 43-52, Omaha (2007)
12. Bell, R.M.; Koren, Y.; Volinsky, C.: The Bellkor 2008 solution to the Netflix Prize. Netflix prize report (2008)
13. Bellogín, A.; Castells, P.; Cantador, I.: Precision-based evaluation of recommender systems: an algorithmic comparison. Fifth ACM Conference on Recommender Systems, pp. 333-336, Chicago (2011)
14. Bennet, J.; Lanning, S.: The Netflix Prize. KDD Cup and Workshop, San Jose (2007)
15. Brenner, A.; Pradel, B.; Usunier, N.; Gallinari, P.: Predicting most rated items in weekly recommendation with temporal regression. Workshop on Context-Aware Movie Recommendation, pp. 24-27, Barcelona (2010)
16. Burke, R.: Hybrid recommender systems: survey and experiments. User Model. User-Adapt. Interact. 12(4), 331-370 (2002)
17. Burke, R.: Hybrid web recommender systems. In: Brusilovsky, P.; Kobsa, A.; Nejdl, W. (eds.) The Adaptive Web, pp. 377-408. Springer, Berlin (2007)
18. Campos, P.G.; Bellogín, A.; Díez, F.; Chavarriaga, J.E.: Simple time-biased KNN-based recommendations. Workshop on Context-Aware Movie Recommendation, pp. 20-23, Barcelona (2010)
19. Campos, P.G.; Díez, F.; Bellogín, A.: Temporal rating habits: a valuable tool for rater differentiation. Second Workshop on Context-Aware Movie Recommendation, pp. 29-35, Chicago (2011a)
20. Campos, P.G.; Díez, F.; Sánchez-Montañés, M.: Towards a more realistic evaluation: testing the ability to predict future tastes of matrix factorization-based recommenders. Fifth ACM Conference on Recommender Systems, pp. 309-312, Chicago (2011b)
21. Cao, H.; Chen, E.; Yang, J.; Xiong, H.: Enhancing recommender systems under volatile user interest drifts. Eighteenth ACM conference on Information and knowledge management, pp. 1257-1266, Hong Kong (2009)
22. Celma, O.: Music recommendation and discovery in the long tail. PhD Dissertation, Universitat Pompeu Fabra, Spain (2010)
23. Chen, P.; Tsai, C.; Chen, Y.; Chou, K.; Li, C.; Tsai, C.; Wu, K.; Chou, Y.; Li, C.; Lin, W.; Yu, S.; Chiu, R.; Lin, C.; Wang, C.; Wang, P.; Su, W.; Wu, C.; Kuo, T.; McKenzie, T.G.; Chang, Y.; Ferng, C.; Ni, C.; Lin, H.; Lin, C.; Lin, S.: A linear ensemble of individual and blended models for music rating prediction. KDD-Cup Report, San Diego (2011a)
24. Chen, T.; Zheng, Z.; Lu, Q.; Jiang, X.; Chen, Y.; Zhang, W.; Chen, K.; Yu, Y.: Informative ensemble of multi-resolution dynamic factorization models'. KDD-Cup Report, San Diego (2011b)
25. Cosley, D.; Lam, S.K.; Albert, I.; Konstan, J.A.; Riedl, J.: Is seeing believing? How recommender interfaces affect users' opinions'. SIGCHI Conference on Human Factors in Computing Systems, pp. 585-592, Fort Lauderdale (2003)
26. Cremonesi, P.; Garzotto, F.; Negro, S.; Papadopoulos, A.V.; Turrin, R.: Looking for "Good" recommendations: a comparative evaluation of recommender systems'. Thirteenth IFIP TC 13 International conference on Human-computer interaction, pp. 152-168, Lisbon (2011)
27. Cremonesi, P.; Koren, Y.; Turrin, R.: Performance of recommender algorithms on top-n recommendation tasks. Fourth ACM conference on Recommender systems, pp. 39-46, Barcelona (2010)
28. Cremonesi, P.; Turrin, R.: Analysis of cold-start recommendations in IPTV systems. Third ACM Conference on Recommender Systems, pp. 233-236, New York (2009)
29. Cremonesi, P.; Turrin, R.: Time-evolution of IPTV recommender systems. Eighth International Interactive Conference on Interactive TV & Video, pp. 105-114, Tampere (2010)
30. Dey, A.K.: Understanding and using context. Pers. Ubiquitous Comput. 5(1), 4-7 (2001)
31. Dietterich, T.G.: Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput. 10(7), 1895-1923 (1998)
32. Ding, Y.; Li, X.: Time weight collaborative filtering. Fourteenth ACM international Conference on Information and Knowledge Management, pp. 485-492, Bremen (2005)
33. Ding, Y.; Li, X.; Orlowska, M.E.: Recency-based collaborative filtering. Seventeenth Australasian Database Conference, pp. 99-107, Hobart, Tasmania (2006)
34. Dourish, P.: What we talk about when we talk about context. Pers. Ubiquitous Comput. 8(1), 19-30 (2004)
35. Duda, R.O.; Hart, P.E.; Stork, D.G.: Pattern Classification, 2nd edn. Wiley. ISBN 0471056693 (2000)
36. Gantner, Z.; Rendle, S.; Schmidt-Thieme, L.: Factorization models for context-/time-aware movie recommendations. Workshop on Context-Aware Movie Recommendation, pp. 14-19, Barcelona (2010)
37. Gordea, S.; Zanker, M.: Time filtering for better recommendations with small and sparse rating matrices. Eighth International Conference on Web Information Systems Engineering, pp. 171-183, Nancy (2007)
38. Gorgoglione, M.; Panniello, U.: Including context in a transactional recommender system using a pre-filtering approach: two real e-commerce applications. 2009 International Conference on Advanced Information Networking and Applications Workshops, pp. 667-672, Bradford (2009)
39. Gorgoglione, M.; Paniello, U.; Tuzhilin, A.: The effect of context-aware recommendations on customer purchasing behavior and trust'. Fifth ACM Conference on Recommender Systems, pp 85-82, Chicago (2011)
40. Gunawardana, A.; Shani, G.: A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 10, 2935-2962 (2009)
41. Herlocker, J.L.; Konstan, J.A.; Borchers, A.; Riedl, J.: An algorithmic framework for performing collaborative Filtering. Twenty-Second Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 230-237, Berkeley (1999)
42. Herlocker, J.L.; Konstan, J.A.; Terveen, L.G.; Riedl, J.T.: Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst. 22(1), 5-53 (2004)
43. Hermann, C.: Time-based recommendations for lecture materials. 2010 World Conference on Educational Multimedia, Hypermedia and Telecommunications, pp. 1028-1033, Toronto (2010)
44. Hussein, T.; Linder, T.; Werner, G.; Ziegler, J.: Hybreed: A software framework for developing context-aware hybrid recommender systems. User Model. User-Adapt. Interact, Special Issue on Context-aware Recommender Systems (2014)
45. Iofciu, T.; Demartini, G.: Time based tag recommendation using direct and extended users sets'. ECML PKDD discovery Challenge 2009, pp. 99-107, Paris (2009)
46. Jahrer, M.; Töscher, A.: Collaborative filtering ensemble. KDD-Cup Report, San Diego (2011)
47. Jahrer, M.; Töscher, A.; Legenstein, R.: Combining predictions for accurate recommender systems. Sixteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 693-702, Washington (2010)
48. Karatzoglou, A.: Collaborative temporal order modeling. Fifth ACM Conference on Recommender Systems, pp. 313-316, Chicago (2011)
49. Karatzoglou, A.; Amatriain, X.; Baltrunas, L.; Oliver, N.: Multiverse recommendation: N-dimensional tensor factorization for context-aware collaborative filtering. Fourth ACM Conference on Recommender Systems, pp. 79-86, Barcelona (2010)
50. Knijnenburg, B.P.; Willemsen, M.C.; Gantner, Z.; Soncu, H.; Newell, C.: Explaining the user experience of recommender systems. User Model. User-Adapt. Interact. 22, 441-504 (2012)
51. Koenigstein, N.; Dror, G.; Koren, Y.: Yahoo! Music recommendations: modeling music ratings with temporal dynamics and item taxonomy'. Fifth ACM Conference on Recommender Systems, pp. 165-172, Chicago (2011)
52. Kohavi, R.; Longbotham, R.; Sommerfield, D.; Henne, R.: Controlled experiments on the web: survey and practical guide. Data Mining Knowl. Discov. 18(1), 140-181 (2009)
53. Konstan, J.A.; Riedl, J.: Recommender systems: from algorithms to user experience. User Model. User-Adapt. Interact. 22, 101-123 (2012)
54. Koren, Y.: The BellKor solution to the Netflix grand prize'. Netflix Prize, Report (2009a)
55. Koren, Y.: Collaborative filtering with temporal dynamics. Fifteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 447-456, Paris (2009b)
56. Lathauwer, L.D.; Moor, B.D.; Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl 21(4), 1253-1278 (2000)
57. Lathia, N.; Hailes, S.; Capra, L.: Temporal collaborative filtering with adaptive neighbourhoods. Thirty-Second International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 796-797, Boston (2009)
58. Lathia, N.; Hailes, S.; Capra, L.; Amatriain, X.: Temporal diversity in recommender systems. Thirty-Third International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 210-217, Geneva (2010)
59. Lee, D.; Park, S.; Kahng, M.; Lee, S.; Lee, S.: Exploiting contextual information from event logs for personalized recommendation'. In Lee, R. (ed.), Computer and Information Science. ISBN 978-3-642-15404-1 (2010)
60. Lee, T.Q.; Park, Y.; Park, Y.: A time-based approach to effective recommender systems using implicit feedback. Expert Syst. Appl. 34(4), 3055-3062 (2008)
61. Lee, T.Q.; Park, Y.; Park, Y.: An empirical study on effectiveness of temporal information as implicit ratings. Expert Syst. Appl. 36(2), 1315-1321 (2009)
62. Li, R.; Li, B.; Jin, C.; Xue, X.; Zhu, X.: Tracking user-preference varying speed in collaborative filtering. Twenty-Fifth AAAI Conference on Artificial Intelligence, San Francisco (2011)
63. Ling, C.; Huang, J.; Zhang, H.: AUC: A better measure than accuracy in comparing learning algorithms. In: Xiang, Y.; Chaib-draa B. (eds.). Advances in Artificial Intelligence 2671, pp. 329-341, LNCS (2003)
64. Lipczak, M.; Hu, Y.; Kollet, Y.; Milios, E.: Tag sources for recommendation in collaborative tagging systems. ECML PKDD Discovery Challenge 2009, pp. 157-172, Paris (2009)
65. Liu, N.N.; Cao, B.; Zhao, M.; Yang, Q.: Adapting neighborhood and matrix factorization models for context aware recommendation. Workshop on Context-Aware Movie Recommendation, pp. 7-13, Barcelona (2010a)
66. Liu, N.N.; Zhao, M.; Xiang, E.; Yang, Q.: Online evolutionary collaborative filtering. Fourth ACM Conference on Recommender Systems, pp. 95-102, Barcelona (2010b)
67. Lu, Z.; Agarwal, D.; Dhillon, I.S.: A spatio-temporal approach to collaborative filtering. Third ACM Conference on Recommender Systems, pp. 13-20, New York (2009)
68. Ma, S.; Li, X.; Ding, Y.; Orlowska, M.E.: A recommender system with interest-drifting. Eighth International Conference on Web Information Systems Engineering, pp. 633-642, Nancy (2007)
69. Min, S.; Han, I.: Detection of the customer time-variant pattern for improving recommender systems. Expert Syst. Appl. 28(2), 189-199 (2005)
70. Montañés, E.; Quevedo, J.R.; Díaz, I.; Ranilla, J.: Collaborative tag recommendation system based on logistic regression. ECML PKDD Discovery Challenge 2009, pp. 173-188, Paris (2009)
71. Oku, K.; Nakajima, S.; Miyazaki, J.; Uemura, S.: Context-aware SVM for context-dependent information recommendation. Seventh International Conference on Mobile Data Management, pp. 109-109, Nara (2006)
72. Palmisano, C.; Tuzhilin, A.; Gorgoglione, M.: Using context to improve predictive modeling of customers in personalization applications. IEEE Trans. Knowl. Data Eng. 20(11), 1535-1549 (2008)
73. Panniello, U.; Gorgoglione, M.; Palmisano, C.: Comparing pre-filtering and post-filtering approach in a collaborative contextual recommender system: an application to e-commerce. Tenth International Conference on E-Commerce and Web Technologies, pp. 348-359, Linz (2009a)
74. Panniello, U.; Tuzhilin, A.; Gorgoglione, M.; Palmisano, C.; Pedone, A.: Experimental comparison of pre- vs. post-filtering approaches in context-aware recommender systems. Third ACM Conference on Recommender Systems, pp. 265-268, New York (2009b)
75. Panniello, U.; Tuzhilin, A.; Gorgoglione, M.: Comparing context-aware recommender systems in terms of accuracy and diversity: which contextual modeling, pre-filtering and post-filtering methods perform the best. User Model. User-Adapt. Interact, Special Issue on Context-Aware Recommender Systems (2013)
76. Park, M.-H.; Hong, J.-H.; Cho, S.-B.: Location-based recommendation system using Bayesian user's preference model in mobile devices. In: Indulska, J.; Yang, L.; Ungerer, L.; Cao, J. (eds.) Proceedings of Ubiquitous Intelligence and Computing, 4611, pp. 1130-1139. Springer, Berlin (2007)
77. Parra, D.; Amatriain, X.: Walk the talk: analyzing the relation between implicit and explicit feedback for preference elicitation. Nineteenth International Conference on User Modeling, Adaption, and Personalization, pp. 255-268, Girona (2011)
78. Pradel, B.; Sean, S.; Delporte, J.; Guérif, S.; Rouveirol, C.; Usunier, N.; Fogelman-Soulié, F.; Dufau-Joel, F.: A case study in a recommender system based on purchase data. Seventeenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377-385, San Diego (2011)
79. Rendle, S.: Time-variant factorization models. In Rendle, S. (ed.), Context-aware ranking with factorization models. ISBN 978-3-642-16897-0 (2011)
80. Rendle, S.; Gantner, Z.; Freudenthaler, C.; Schmidt-Thieme, L.: Fast context-aware recommendations with factorization machines. Thirty-Fourth International ACM SIGIR Conference on Research and Development in Information, pp. 635-644, Beijing (2011)
81. Sarwar, B.; Karypis, G.; Konstan, J.; Riedl.; J.: Analysis of recommendation algorithms for e-commerce. Second ACM Conference on Electronic Commerce, pp. 158-167, Minneapolis (2000)
82. Shani, G.; Gunawardana, A.: Evaluating recommendation systems. In: Ricci, F.; Rokach, L.; Shapira, B.; Kantor, P.B. (eds.), Recommender Systems Handbook. ISBN 978-0-387-85820-3 (2011).
83. Stormer, H.: Improving e-commerce recommender systems by the identification of seasonal products. Twenty-second Conference on Artificial Intelligence, pp. 92-99, Vancouver, British Columbia (2007).
84. Töscher, A.; Jahrer, M.: The bigchaos solution to the Netflix prize 2008. Netflix Prize, Report (2008)
85. Töscher, A.; Jahrer, M.; Legenstein, R.: Improved neighborhood-based algorithms for large-scale recommender systems. Second KDD Workshop on Large-Scale Recommender Systems and the Netflix Prize Competition, pp. 4:1-4:6, Las Vegas (2008)
86. Tang, T.Y.; Winoto, P.; Chan, K.C.C.: On the temporal analysis for improved hybrid recommendations. 2003 IEEE/WIC International Conference on Web Intelligence, pp. 214-220, Beijing (2003).
87. Vargas, S.; Castells, P.: Rank and relevance in novelty and diversity metrics for recommender systems. Fifth ACM Conference on Recommender Systems, pp. 109-116, Chicago (2011).
88. Vildjiounaite, E.; Kyllönen, V.; Hannula, T.; Alahuhta, P.: Unobtrusive dynamic modelling of TV program preferences in a household. In: Tscheligi, M.; Obrist, M.; Lugmayr, A. (eds.), Changing Television Environments. ISBN 978-3-540-69477-9 (2008)
89. Weng, S.-S.; Binshan, L.; Chen, W.-T.: Using contextual information and multidimensional approach for recommendation. Expert Syst. Appl. 36(2009), 1268-1279 (2009)
90. Whitrow, G. J.: Time in history: views of time from prehistory to the present day. Oxford University Press, Oxford. ISBN 0-19-285211-6 (1988)
91. Wu, Y.; Yan, Q.; Bickson, D.; Low, Y.; Yang, Q.: Efficient multicore collaborative filtering. KDD-Cup Report, San Diego (2011)
92. Xiang, L.; Yang, Q.: Time-dependent models in collaborative filtering based recommender system. 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, pp. 450-457, Milano (2009)
93. Xiang, L.; Yuan, Q.; Zhao, S.; Chen, L.; Zhang, X.; Yang, Q.; Sun, J.: Temporal recommendation on graphs via long- and short-term preference fusion. Sixteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 723-732, Washington (2010)
94. Xiong, L.; Chen, X.; Huang, T.; Schneider, J.; Carbonell, J.G.: Temporal collaborative filtering with Bayesian probabilistic tensor factorization. SIAM Data Mining 2010, 211-222 (2010)
95. Zhan, S.; Gao, F.; Xing, C.; Zhou, L.: Addressing concept drift problem in collaborative filtering systems. ECAI 2006 Workshop on Recommender Systems, pp. 34-39, Riva del Garda (2006)
96. Zhang, M.; Hurley, N.: Novel item recommendation by user profile partitioning. 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology, pp. 508-515, Milano (2009)
97. Zheng, N.; Li, Q.: A recommender system based on tag and time information for social tagging systems. Expert Syst. Appl. 38(4), 4575-4587 (2011)
98. Zhou, T.; Kuscsik, Z.; Liu, J.-G.; Medo, M.; Wakeling, J.R.; Zhang, Y.-C.: Solving the apparent diversity-accuracy dilemma of recommender systems. Natl. Acad. Sci. 107(10), 4511-4515 (2010)
99. Ziegler, C.; McNee, S.M.; Konstan, J.A.; Lausen, G.: Improving recommendation lists through topic diversification. Fourteenth International Conference on World Wide Web, pp. 22-32, Chiba (2005)
100. Zimdars, A.; Chickering, D.M.; Meek, C.: Using temporal data for making recommendations. Seventeenth Conference in Uncertainty in Artificial Intelligence, pp. 580-588, Seattle (2001)