Creating personalized systems that people can scrutinize and control: Drivers, principles and experience

ACM Transactions on Interactive Intelligent Systems

Volumn 2, Issue 4, 2012, Pages

  Kay, Judy ^a   Kummerfeld, Bob ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Augmented cognition; Inconsistency; Open learner modeling; Personalization; Reasoning under uncertainty; Scrutability; Understandability; User control; User model; User modeling

Indexed keywords

COMPUTER PROGRAMMING; DIGITAL DEVICES; LOCATION BASED SERVICES; PERSONAL COMPUTERS;

AUGMENTED COGNITION; INCONSISTENCY; OPEN LEARNER MODELING; PERSONALIZATIONS; REASONING UNDER UNCERTAINTY; SCRUTABILITY; UNDERSTANDABILITY; USER CONTROL; USER MODELING; USER MODELS;

MATHEMATICAL MODELS;

EID: 84983587906     PISSN: 21606455     EISSN: 21606463     Source Type: Journal    
DOI: 10.1145/2395123.2395129     Document Type: Article

References (107)

1. AHN, J.-W., BRUSILOVSKY, P., GRADY, J.,HE, D., AND SYN, S. Y. 2007. Open user profiles for adaptive news systems: Help or harm? In Proceedings of the 16th International Conference onWorldWideWeb (WWW '07). ACM, New York, 11-20.
2. APTED, T. AND KAY, J. 2004. Mecureo ontology and modelling tools. Int. J. Contin. Engin. Educ. Lifelong Learn. 14, 3, 191-211.
3. APTED, T., KAY, J., LUM, A., AND UTHER, J. 2003. Visualisation of ontological inferences for user control of personal web agents. In Proceedings of 7th International Conference on Information Visualization (IV'03), E. Banissi, K. Borner, C. Chen, G. Clapworthy, C. Maple, A. Lobben, C. Moore, J. Roberts, A. Ursyn, and J. Zhang, Eds. IEEE Computer Society, 306-311.
4. ASSAD, M.,CARMICHAEL, D.,KAY, J., AND KUMMERFELD, B. 2007. Personisad: Distributed, active, scrutable model framework for context-aware services. In Pervasive Computing, A. LaMarca, M. Langheinrich, and K. Truong, Eds. Lecture Notes in Computer Science, vol. 4480. Springer, 55-72.
5. BARKHUUS, L. AND DEY, A. 2003. Is context-aware computing taking control away from the user? three levels of interactivity examined. In Proceedings of the Conference on Ubiquitous Computing (UbiComp '03), A. Dey, A. Schmidt, and J. McCarthy, Eds. Lecture Notes in Computer Science, vol. 2864. Springer, 149-156.
6. BERKOVSKY, S., EYTANI, Y., KUFLIK, T., AND RICCI, F. 2007. Enhancing privacy and preserving accuracy of a distributed collaborative filtering. In Proceedings of the ACM Conference on Recommender Systems. ACM, 9-16.
7. BLEIHOLDER, J. AND NAUMANN, F. 2009. Data fusion. ACM Comput. Surv. 41, 1:1-1:41.
8. BRAR, A. 2004. Privacy and security in ubiquitous personalised applications. Honours thesis, Unversity of Sydney.
9. BROWNE, D., TOTTERDELL, P., AND NORMAN, M. 1990. Adaptive User Interfaces. Academic Press.
10. BRUSILOVSKY, P. 1996. Methods and techniques of adaptive hypermedia. User Model. User-Adapt. Interact. 6, 2, 87-129.
11. BRUSILOVSKY, P., KOBSA, A., AND NEJDL, W. 2007. The Adaptive Web: Methods and Strategies of Web Personalization. Springer.
12. BRUSILOVSKY, P. ANDMILĹAN, E. 2007. User models for adaptive hypermedia and adaptive educational systems. In The Adaptive Web, P. Brusilovsky, A. Kobsa, and W. Nejdl, Eds. Lecture Notes in Computer Science, vol. 4321. Springer, 3-53.
13. BULL, S., BRNA, P., AND PAIN, H. 1995. Extending the scope of the student model. User Model. User-Adapt. Interact. 5, 1, 45-65.
14. BULL, S. AND GARDNER, P. 2009. Highlighting learning across a degree with an independent open learner model. In Artificial Intelligence in Education. IOS Press, 275-282.
15. BULL, S. AND KAY, J. 2007. Student models that invite the learner in: The SMILI open learner modelling framework. Int. J. Artif. Intell. 17, 2, 89-120.
16. BULL, S. AND KAY, J. 2008. Metacognition and open learner models. In Proceedings of the 3rd Workshop on Meta-Cognition and Self-Regulated Learning in Educational Technologies, at ITS'08, Intelligent Tutoring Systems. 7-20.
17. BULL, S. AND MABBOTT, A. 2006. 20000 inspections of a domain-independent open learner model with individual and comparison views. In Proceedings of the Conference on Intelligent Tutoring Systems (ITS'06) M. Ikeda, K. D. Ashley, and T.-W. Chan, Eds. Vol. 4053, 422-432.
18. BULL, S., MABBOTT, A., AND ABU ISSA, A. 2007. Umpteen: Named and anonymous learner model access for instructors and peers. Int. J. Artif. Intell. Educ. 17, 3, 227-253.
19. CARMAGNOLA, F., CENA, F., AND GENA, C. 2011. User model interoperability: a survey. User Model. User-Adapt. Interact. 21, 285-331.
20. CARMAGNOLA, F. ANDDIMITROVA, V. 2008. An evidence-based approach to handle semantic heterogeneity in interoperable distributed user models. In Adaptive Hypermedia and Adaptive Web-Based Systems. Springer, 73-82.
21. CARMICHAEL, D. 2009. Myplace: Supporting scrutability and user control in location modelling. Ph.D. thesis, University of Sydney.
22. CARMICHAEL, D. J., KAY, J., AND KUMMERFELD, R. J. 2005. Consistent modelling of users, devices and sensors in a ubiquitous computing environment. User Model. User-Adapt. Interact. 15, 3-4, 197-234.
23. CHELLAPPA, R. AND SIN, R. 2005. Personalization versus privacy: An empirical examination of the online consumers dilemma. Inf. Technol. Manag. 6, 2, 181-202.
24. CIMOLINO, L., KAY, J., AND MILLER, A. 2004. Concept mapping for eliciting verified personal ontologies. Int. J. Contin. Engin. Educ. Life-Long Learn. 14, 3, 212-228.
25. CISCO. 2011. Cisco visual networking index: Forecast and methodology, White Paper, 2010-2015.
26. CLANCEY, W. 1983. The epistemology of a rule-based expert system-A framework for explanation. Artif. Intell. 20, 3, 215-251.
27. COOK, R. AND KAY, J. 1994. The justified user model: A viewable, explained user model. In Proceedings of the 4th International Conference on User Modeling (UM'94), A. Kobsa and D. Litman, Eds. Mitre, 145-150.
28. COOK, R., KAY, J., RYAN, G., AND THOMAS, R. 1995. A toolkit for appraising the long-term usability of a text editor. Softw. Qual. J. 4, 2, 131-154.
29. CORE, M., LANE, H., VAN LENT, M., GOMBOC, D., SOLOMON, S., AND ROSENBERG, M. 2006. Building explainable artificial intelligence systems. In Proceedings of the National Conference on Artificial Intelligence. Vol. 21.
30. CRAMER, H., EVERS, V., RAMLAL, S., VAN SOMEREN, M., RUTLEDGE, L., STASH, N., AROYO, L., AND WIELINGA, B. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Model. User-Adapt. Interact. 18, 5, 455-496.
31. CSINGER, A. 1995. User models for intent-based authoring. Ph.D. thesis, University of British Columbia.
32. CZARKOWSKI, M. 2006. A scrutable adaptive hypertext. Ph.D. thesis, University of Sydney.
33. CZARKOWSKI, M. AND KAY, J. 2000. Bringing scrutability to adaptive hypertext teaching. In Proceedings of the Conference on Intelligent Tutoring Systems (ITS'00), G. Gauthier, C. Frasson, and K. VanLehn, Eds. 423-432.
34. CZARKOWSKI, M. AND KAY, J. 2006. Giving learners a real sense of control over adaptivity, even if they are not quite ready for it yet. In Advances in Web-Based Education: Personalized Learning Environments. IDEA Information Science Publishing, Hershey, PA, USA, 93-125.
35. DIMITROVA, V. 2003. STyLE-OLM: Interactive open learner modelling. Int. J. Artif. Intell. Educ. 13, 1, 35-78.
36. GERBER, S., FRY, M., KAY, J., KUMMERFELD, B., PINK, G., AND WASINGER, R. 2010. PersonisJ: mobile, client-side user modelling. In Proceedings of the Conference on User Modeling, Adaptation and Personalisation (UMAP'10). Springer, 111-122.
37. GLASS, A., MCGUINNESS, D., AND WOLVERTON, M. 2008. Toward establishing trust in adaptive agents. In Proceedings of the International Conference on Intelligent User Interfaces. Vol. 8. 227-236.
38. HERLOCKER, J. L., KONSTAN, J. A., AND RIEDL, J. 2000. Explaining collaborative filtering recommendations. In Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW '00), ACM, New York, 241-250.
39. HILBERT, D. AND REDMILES, D. 2000. Extracting usability information from user interface events. ACMComput. Surv. 32, 4, 384-421.
40. HITCHENS, M., KAY, J., KUMMERFELD, R. J., AND BRAR, A. 2005. Secure identity management for pseudoanonymous service access. In Proceedings of the 2nd International Conference on Security in Pervasive Computing (SPC 2005), D. Hutter, Ed. Lecture Notes in Computer Science, vol. 3450. Springer, 48-55.
41. HOLDEN, S., KAY, J., POON, J., AND YACEF, K. 2005.Workflow-Based personalised document delivery (just-in-time training system). J. Interact. Learn. 4, 1, 131-148.
42. HOOK, K. 2000. Steps to take before intelligent user interfaces become real. Interact. Comput. 12, 4, 409-426.
43. HOOK, K., KARLGREN, J.,WAERN, A.,DAHLBACK, N., JANSSON, C.,KARLGREN, K., AND LEMAIRE, B. 1996. A glass box approach to adaptive hypermedia. User Model. User-Adapt. Interact. 6, 2, 157-184.
44. HORVITZ, E., BREESE, J., HECKERMAN, D., HOVEL, D., AND ROMMELSE, K. 1998. The Lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence (UAI'98). Morgan Kaufmann Publishers, San Francisco, CA, 256-265.
45. IACHELLO, G. AND HONG, J. 2007. End-User privacy in human-computer interaction. Found. Trends Hum.- Comput. Interact. 1, 1, 1-137.
46. JAMESON, A. AND SCHWARZKOPF, E. 2006. Pros and cons of controllability: An empirical study. In Adaptive Hypermedia and Adaptive Web-Based Systems. Springer, 193-202.
47. KAY, J. 1990. UM: A user modelling toolkit. In the 2nd International User Modeling Workshop.
48. KAY, J. 1995. The um toolkit for cooperative user modelling. User Model. User-Adapt. Interact. 4, 149-196.
49. KAY, J. 1997. Learner know thyself: Student models to give learner control and responsibility. In Proceedings of International Conference on Computers in Education. 17-24.
50. KAY, J. 1999. A scrutable user modelling shell for user-adapted interaction. Ph.D. thesis, University of Sydney.
51. KAY, J. 2000a. Accretion representation for scrutable student modelling. In Proceedings of the Conference on Intelligent Tutoring Systems (ITS'00), G. Gauthier, C. Frasson, and K. VanLehn, Eds. 514-523.
52. KAY, J. 2000b. Invited keynote: Stereotypes, student models and scrutability. In Proceedings of the Conference on Intelligent Tutoring Systems (ITS'00), G. Gauthier, C. Frasson, and K. VanLehn, Eds. Springer, 19-30.
53. KAY, J. AND KUMMERFELD, B. 2010. Tackling HCI challenges of creating personalised, pervasive learning ecosystems. In Sustaining TEL: From Innovation to Learning and Practice 5th European Conference on Technology Enhanced Learning, (EC-TEL '10), M. Wolpers, P. A. Kirschner, M. Scheffel, S. Lindstaedt, and V. Dimitrova, Eds. Springer, 1-16.
54. KAY, J., KUMMERFELD, B., AND LAUDER, P. 2002. In Proceedings of the 2nd International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems (AH '02), P. D. Bra, P. Brusilovsky, and R. Conejo, Eds. Vol. 2347. Springer, 203-212.
55. KAY, J. AND KUMMERFELD, R. 1994. An individualised course for the C programming language. In Proceedings of 2nd International World Wide Web Conference. 17-20.
56. KAY, J., LI, L., AND FEKETE, A. 2007. Learner reflection in student self-assessment. In Proceedings of the 9th Australasian Computing Education Conference (ACE '07). Australian Computer Society, Sydney, 89-95.
57. KAY, J. AND LUM, A. 2005. Exploiting readily available web data for reflective student models. In Proceedings of the Conference Artificial Intelligence in Education (AIED '05). IOS Press, Amsterdam, The Netherlands, 338-345.
58. KAY, J. AND THOMAS, R. C. 1995. Studying long-term system use. Comm. ACM 38, 7, 61-69.
59. KOBSA, A. 2001. Generic user modeling systems. User Model. User-Adapt. Interact. 11, 1, 49-63.
60. KOBSA, A. 2002. Personalized hypermedia and international privacy. Comm. ACM 45, 5, 64-67.
61. KOBSA, A. 2007. Privacy-Enhanced web personalization. In The Adaptive Web: Methods and Strategies of Web Personalization, P. Brusilovsky, A. Kobsa, and W. Nejdl, Eds. Springer, 628-670.
62. KOBSA, A. AND FINK, J. 2006. An LDAP-based user modeling server and its evaluation. User Model. User-Adapt. Interact. 16, 2, 129-169.
63. KOBSA, A.,KOENEMANN, J., AND POHL,W. 2001. Personalised hypermedia presentation techniques for improving online customer relationships. The Knowl. Engin. Rev. 16, 02, 111-155.
64. KOBSA, A. AND WAHLSTER, W. 1989. User Models in Dialog Systems. Springer.
65. KOIDL, K. AND CONLAN, O. 2008. Engineering information systems towards facilitating scrutable and configurable adaptation. Adapt. Hypermedia and Adapt. Web-Based Syst. 5149, 405-409.
66. KONO, Y., IKEDA, M., AND MIZOGUCHI, R. 1992. To contradict is human-student modeling of inconsistency. In Proceedings of the Conference on Intelligent Tutoring Systems (ITS'92). Springer, 451-458.
67. KR̈UGER, A., BAUS, J., HECKMANN, D., KRUPPA, M., AND WASINGER, R. 2007. Adaptive Mobile Guides. In The Adaptive Web: Methods and Strategies of Web Personalization, P. Brusilovsky, A. Kobsa, and W. Nejdl, Eds. Springer, 521-549.
68. LANGHEINRICH, M. 2002. A privacy awareness system for ubiquitous computing environments. In 4th International Conference on Ubiquitous Computing (Ubicomp '02), G. Borriello and L. E. Holmquist, Eds. Lecture Notes in Computer Science, vol. 2498. Springer, 237-245.
69. LEE, S. AND BULL, S. 2008. An open learner model to help parents help their children. Technol. Instruct. Cogn. and Learn. 6, 1, 29.
70. LI, I., DEY, A., AND FORLIZZI, J. 2010. A stage-based model of personal informatics systems. In Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI '10). 557-566.
71. LIM, B., DEY, A., AND AVRAHAMI, D. 2009. Why and why not explanations improve the intelligibility of contextaware intelligent systems. In Proceedings of the 27th International Conference on Human Factors In Computing Systems (CHI'09). ACM New York, 2119-2128.
72. LIM, B. Y. AND DEY, A. K. 2009. Assessing demand for intelligibility in context-aware applications. In Proceedings of the 11th International Conference on Ubiquitous Computing (Ubicomp '09). ACM, New York, 195-204.
73. LIM, B. Y. AND DEY, A. K. 2010. Toolkit to support intelligibility in context-aware applications. In Proceedings of the 12th ACM International Conference on Ubiquitous Computing (Ubicomp '10). ACM, New York, 13-22.
74. LINTON, F. AND SCHAEFER, H. 2000. Recommender systems for learning: Building user and expert models through long-term observation of application use. User Model. User-Adapt. Interact. 10, 2, 181-208.
75. LUM, A. 2007. Light-Weight ontologies for scrutable user modelling. Ph.D. thesis, University of Sydney.
76. MITROVIC, A. AND MARTIN, B. 2006. Evaluating the effects of open student models on learning. In Adaptive Hypermedia and Adaptive Web-Based Systems. Springer, 296-305.
77. NIU, W. AND KAY, J. 2008. Location conflict resolution with an ontology. In Proceedings of the 6th International Conference on Pervasive Computing (Pervasive '08). Springer, 162-179.
78. NIU, W. T. AND KAY, J. 2010. PERSONAF: Framework for personalised ontological reasoning in pervasive computing. User model. User-Adapt. Interact.: The J. Personal. Res. 20, 1, 1-40.
79. NOVAK, J. 1990. Concept maps and vee diagrams: Two metacognitive tools to facilitate meaningful learning. Instruct. Sci. 19, 1, 29-52.
80. ORWANT, J. 1994. Heterogeneous learning in the doppelg̈anger usermodeling system. User Model. User-Adapt. Interact. 4, 2, 107-130.
81. PALEN, L. AND DOURISH, P. 2003. Unpacking "privacy" for a networked world. In Proceedings of the Conference on Human Factors in Computing Systems (CHI'03), 129-136.
82. PARAMYTHIS, A., WEIBELZAHL, S., AND MASTHOFF, J. 2010. Layered evaluation of interactive adaptive systems: Framework and formative methods. User Model. User-Adapt. Interac. 1-71.
83. PARISER, E. 2011. The Filter Bubble: What the Internet is Hiding from You. Penguin.
84. PRICE, B., ADAM, K., AND NUSEIBEH, B. 2005. Keeping ubiquitous computing to yourself: A practical model for user control of privacy. Int. J. Human-Comput. Stud. 63, 1-2, 228-253.
85. RICH, E. 1979. User modeling via stereotypes. Cogn. Sci. 3, 329-354.
86. RICH, E. 1983. Users are individuals: Individualizing user models. Int. J. Man-Mach. Stud. 18, 199-214.
87. SCHNEIDERMAN, B. 1996. The eyes have it: A task by data type taxonomy for information visualizations. In Proceedings of the IEEE Symposium on Visual Languages. IEEE Computer Society, 336-343.
88. SELF, J. 1999. Open Sesame?: Fifteen variations on the theme of openness in learning environments. Int. J. Artif. Intell. Educ. 10, 1020-1029.
89. SELLEN, A. J. AND WHITTAKER, S. 2010. Beyond total capture: A constructive critique of lifelogging. Comm. ACM 53, 5, 70-77.
90. SHILTON, K. 2009. Four billion little brothers?: Privacy, mobile phones, and ubiquitous data collection. Queue 7, 7, 40-47.
91. SPIEKERMANN, S. 2008. User Control in Ubiquitous Computing: Design Alternatives and User Acceptance (Habilitation). Shaker.
92. SPIEKERMANN, S. AND CRANOR, L. 2009. Engineering privacy. IEEE Trans. Softw. Engin. 35, 1, 67-82.
93. TAMIR, P. 1984. What do learning theories and research have to say to practitioners in science education? In Research and Development in Higher Education, J. Lublin, Ed. Vol. 7. 162-166.
94. TINTAREV, N. AND MASTHOFF, J. 2007. Effective explanations of recommendations: User-Centered design. In Proceedings of the ACM Conference on Recommender Systems. ACM, 153-156.
95. TOCH, E., WANG, Y., AND CRANOR, L. F. 2012. Personalization and privacy: A survey of privacy risks and remedies in personalization-based systems. User Model. User-Adapt. Interact. 20, 1, 203-220.
96. UPTON, K. AND KAY, J. 2009. Narcissus: Interactive activity mirror for small groups. In Proceedings of the Conference on User Modeling, Adaptation and Personalisation (UMAP'09), 54-65.
97. UTHER, J. 2002. On the visualisation of large user models in web based systems. Ph.D. thesis, University of Sydney.
98. UTHER, J. AND KAY, J. 2003. Vlum, a web-based visualisation of large user models. In Proceedings of the Conference on User Modeling (UM '03), P. Brusilovsky, A. Corbett, and F. de Rosis, Eds. Springer, 198-202.
99. WANG, Y. AND KOBSA, A. 2007. Respecting users individual privacy constraints in web personalization. In Proceedings of the Conference on User Modeling Adaptation and Personalization (UMAP'07), 157-166.
100. WANG, Y. AND KOBSA, A. 2009. Performance evaluation of a privacy-enhancing framework for personalized websites. In Proceedings of the Conference on User Modeling, Adaptation, and Personalization (UMAP'09), 78-89.
101. WEBER, G. AND BRUSILOVSKY, P. 2001. ELM-ART: An adaptive versatile system for web-based instruction. Int. J. Artif. Intell. Educ. 12, 4, 351-384.
102. WESTIN, A. 2003. Social and political dimensions of privacy. Contemp. Perspect. Priv. Social, Psychol. Polit. 59, 2, 431-453.
103. WOLF, G. May, 2, 2010. The data-driven life. The New York Times 28, MM38 of the Sunday Magazine.
104. YUDELSON, M.,BRUSILOVSKY, P., AND ZADOROZHNY, V. 2007. A user modeling server for contemporary adaptive hypermedia: An evaluation of the push approach to evidence propagation. In Proceedings of the Conference on User Modeling (UM '07) Vol. 4511, 27-36.
105. ZAPATA-RIVERA, D., HANSEN, E., SHUTE, V., UNDERWOOD, J., AND BAUER, M. 2007. Evidence-based approach to interacting with open student models. Int. J. Artif. Intell. Educ. 17, 3, 273-303.
106. ZAPATA-RIVERA, J. AND GREER, J. 2004. Inspectable bayesian student modelling servers in multi-agent tutoring systems. Int. J. Human-Comput. Stud. 61, 4, 535-563.
107. ZASLOW, J. 2002. If TiVo thinks you are gay, here's how to set it straight. Wall St. J. 26.