Automatic detection of users' skill levels using high-frequency user interface events

User Modeling and User-Adapted Interaction

Volumn 20, Issue 2, 2010, Pages 109-146

  Ghazarian, Arin ^a   Noorhosseini, S Majid ^a  

^a AMIRKABIR UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

Adaptive user interfaces; Expertise; GOMS; Graphical user interfaces; Intelligent user interfaces; Machine learning; Skill; User modeling

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; LEARNING SYSTEMS;

ADAPTIVE USER INTERFACE; EXPERTISE; GOMS; INTELLIGENT USER INTERFACES; SKILL; USER MODELING;

GRAPHICAL USER INTERFACES;

EID: 77954088175     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-010-9073-5     Document Type: Article

References (70)

1. J.R. Anderson 1982 Acquisition of cognitive skill Psychol. Rev. 89 4 369 406 10.1037/0033-295X.89.4.369
2. Anderson J.R.: The Architecture of Cognition. Harvard University Press, Cambridge (1983)
3. W. Barfield 1986 Expert-novice differences for software: implications for problem-solving and knowledge acquisition Behav. Inf. Technol. 5 1 15 29 10.1080/01449298608914495
4. Beale, R., Finlay, J., Austin, J., Harrison, M.: User modelling by classification: a neural-based approach. New Developments in Neural Computing, In: Taylor, J.G., Mannion, C.L.T. (eds.) (Adam Hilger), pp. 103-110 (1989)
5. D. Benyon 1993 Adaptive systems: a solution to usability problems User Model. User-adapt. Interact. 3 1 65 87 10.1007/BF01099425
6. Brewer M.: Research design and issues of validity. In: Reis, H., Judd, C. (eds) Handbook of research methods in social and personality psychology, Cambridge University Press, Cambridge (2000)
7. Browne, D., Totterdell, P., Norman, M. (eds): Adaptive User Interfaces. Academic Press Ltd, London (1990)
8. P. Brusilovsky 1996 Methods and techniques of adaptive hypermedia User Model. User-adapt. Interact. 6 2-3 87 129 10.1007/BF00143964
9. S.K. Card T.P. Moran A. Newell 1980 The keystroke-level model for user performance time with interactive systems Commun. ACM. 23 7 396 410 10.1145/358886.358895 (Pubitemid 11433615)
10. R. Caruana 1997 Multitask learning Mach. Learn. 28 41 75 10.1023/A:1007379606734
11. W.G. Chase H.A. Simon 1973 Perception in chess Cogn. Psychol. 4 55 81 10.1016/0010-0285(73)90004-2
12. B. Cheung Chiu G.I. Webb 1998 Using decision trees for agent modeling: improving prediction performance User Model. User-adapt. Interact. 8 1-2 131 152 10.1023/A:1008296930163
13. Chin, D.N.: KNOME: modeling what the user knows in UC. In: Kobsa A., Wahlster W. (eds.) User Models in Dialog Systems, pp. 74-107. Springer, Berlin (Symbolic computation series edited by Loveland D.W.) (1989)
14. E.R.F.W. Crossman 1959 A theory of the acquisition of speed-skill Ergonomics 2 153 156 10.1080/00140135908930419
15. A. Darzi S. Mackay 2002 Skills assessment of surgeons Surgery 131 2 121 124 10.1067/msy.2002.115831 (Pubitemid 34169534)
16. V. Datta S. Mackay M. Mandalia A. Darzi 2001 The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in laboratory-based model J. Am. Coll. Surg. 193 479 485 10.1016/S1072-7515(01) 01041-9 (Pubitemid 33043449)
17. V. Datta M. Mandalia S. Mackay A. Chang N. Cheshire A. Darzi 2001 Relationship between skill and outcome in the laboratory-based model Surgery 131 3 318 323 10.1067/msy.2002.120235
18. Desmarais, M.C., Liu, J.: Exploring the applications of user-expertise assessment for intelligent interfaces. In: Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems, (pp. 308-313). Amsterdam, The Netherlands (1993)
19. Encarnação, L.M.: Multi-level user support through adaptive hypermedia: a highly application-independent help component. In: Proceedings of the 1997 International Conference on Intelligent User Interfaces, pp. 187-194. Orlando, Florida, United States (1997)
20. Eurostat: http://epp.eurostat.ec.europa.eu (2005)
21. L. Faulkner D. Wick 2005 Cross-user analysis: benefits of skill level comparison in usability testing Interact. Comput. 17 6 773 786 10.1016/j.intcom.2005.04.004 (Pubitemid 41586122)
22. G. Fischer 2001 User modeling in human-computer interaction User Model. User-adapt. Interact. 11 1-2 65 86 1030.68664 10.1023/A:1011145532042 (Pubitemid 32409025)
23. P.M. Fitts 1954 The information capacity of the human motor system in controlling the amplitude of movement J. Exp. Psychol. 47 6 381 391 10.1037/h0055392 (Reprinted in J. Exp. Psychol.: General, 121(3):262-269, 1992)
24. Fitts P.M.: Perceptual-motor skill learning. In: Melton, A.W. (eds) Categories of Human Learning, Academic Press, New York and London (1964)
25. Fitts P.M., Posner M.I.: Human Performance. Brooks/Cole Publishing Company, Belmont (1968)
26. E. Frias-Martinez S.Y. Chen R.D. Macredie X. Liu 2007 The role of human factors in stereotyping behavior and perception of digital library users: a robust clustering approach User Model. User-adapt. Interact. 17 3 305 337 10.1007/s11257-007-9028-7 (Pubitemid 46807641)
27. Ghazarian, A.: A case study of defect introduction mechanisms. In: Proceedings of the 21st International Conference on Advanced Information Systems (CAiSE'09), pp. 156-170. Amsterdam, The Netherlands (2009)
28. W. Hacker 1994 Action theory and occupational psychology. Review of German empirical research since 1987 Ger. J. Psychol. 18 2 91 120
29. Hackos J.T., Redish J.C.: User and task analysis for interface design. Wiley, New York (1998)
30. L. Hassel E. Hagen 2006 Adaptation of an automotive dialogue system to users' expertise and evaluation of the system Computers and the Humanities 40 1 67 85
31. Hilbert, D.M., Robinns, J.E., Redmiles, D.F.: Supporting ongoing user involvement in development via expectation-driven event monitoring. In: Tech. Rep. UCI-ICS-97-19, Department of Information and Computer Science, University of California, Irvine (1997)
32. D.M. Hilbert D.F. Redmiles 2000 Extracting usability information from user interface events ACM Comput. Surv. 32 4 384 421 10.1145/371578.371593
33. Horvitz, E., Breese, J., Heckerman, D., Hovel, D., Romelse, K.: The Lumière project: Bayesian user modeling for inferring the goals and needs of software users. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 256-265. Madison, WI (1998)
34. Hurst, A., Hudson, S.E., Mankoff, J.: Dynamic detection of novice versus skilled use without a task model. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 271-280. San Jose, California, USA (2007)
35. Jameson, A.: Generalizing the double-stereotype approach: a psychological perspective. In: Proceedings of the Third International Workshop on User Modeling, pp. 69-83. Wadern, Germany (1992)
36. B.E. John D.E. Kieras 1996 Using GOMS for user interface design and evaluation: which technique? ACM Trans. Comput. Hum. Interact. 3 4 287 319 10.1145/235833.236050
37. Jokinen, K., Kanto, K.: User expertise modelling and adaptivity in a speech-based e-mail system. In: Proceedings of the 42nd Annual Meeting on Association For Computational Linguistics, vol. 87. Barcelona, Spain (2004)
38. Kobsa, A.: User modeling: recent work, prospects and hazards. In: SchneiderHufschmidt, M., Kühme, T., Malinowski, U. (eds.) Adaptive User Interfaces: Principles and Practice, pp. 111-128. Amsterdam North-Holland (1993)
39. A. Kobsa 2001 Generic user modeling systems User Model. User-adapt. Interact. 11 1-2 49 63 1030.68731 10.1023/A:1011187500863 (Pubitemid 32409024)
40. Kurosu, M., Urokohara, H., Sato, D., Nishimura, T., Yamada, F.: A new quantitative measure for usability testing: NEM (novice expert ratio method). Poster session presented at the annual conference of the usability professionals' association on humanizing design, Orlando, Florida (2002)
41. P. Langley H.A. Simon 1995 Applications of machine learning and rule induction Commun. ACM 38 11 54 64 10.1145/219717.219768
42. S.C. Leung J. Fulcher 1997 Classification of user expertise level by neural networks Intl. J. Neural Syst. 8 2 155 171 10.1142/S0129065797000185
43. H.C. Lin I. Shafran D. Yuh G.D. Hager 2006 Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions Comput. Aided Surg. 11 5 220 230 10.1080/10929080600989189 (Pubitemid 44859645)
44. Mitchel T.M.: Machine Learning. McGrawHill, New York (1997)
45. Neves D.M., Anderson J.R.: Knowledge compilation: mechanisms for the automatization of cognitive skills. In: Anderson, J.R. (eds) Cognitive Skills and Their Acquisition, pp. 57-84. Wiley, Hillsdale (1981)
46. Newell A., Rosenbloom P.: Mechanisms of skill acquisition and the law of practice. In: Anderson, J.R. (eds) Cognitive Skills and Their Acquisition, pp. 2-55. Erlbaum Associates, Hillsdale (1981)
47. Nielsen J.: Usability Engineering. Academic Press, Boston (1993)
48. R.E. Nisbett T.D. Wilson 1977 Telling more than we can know: verbal reports on mental processes Psychol. Rev. 84 231 259 10.1037/0033-295X.84.3.231
49. Norman D.: Design of Everyday Things. Doubleday, New York (1988)
50. Olson J.R., Olson G.M.: The growth of cognitive modeling in human-computer interaction since GOMS. In: Baecker, R.M., Grudin, J., Buxton, W.A., Greenberg, S. (eds) Human-Computer Interaction: Toward the Year 2000, pp. 603-625. Morgan Kaufmann Publishers, San Francisco (1995)
51. J.G. Phillips T.J. Triggs 2001 Characteristics of cursor trajectories controlled by the computer mouse Ergonomics 44 527 536 10.1080/00140130010020530
52. Popovic, V.: Novice and Expert User Models, PhD dissertation, University of Sydney (1998)
53. Provost, F.J., Fawcett, T., Kohavi, R.: The Case against accuracy estimation for comparing induction algorithms. In: Proceedings of the Fifteenth International Conference on Machine Learning, pp. 445-453. Madison, Wisconsin, USA (1998)
54. Quinlan J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, Los Altos (1993)
55. J. Rasmussen 1983 Skills, rules and knowledge; signals, signs and symbols, and other distinctions in human performance models IEEE Trans. Syst. Man Cybern. 13 3 257 266
56. E. Rich 1999 Users are individuals: individualizing user models Int. J. Hum. Comput. Stud. 51 2 323 338 10.1006/ijhc.1981.0312 1292451
57. C. Richards J. Rosen B. Hannaford C. Pellegrini M. Sinanan 2000 Skills evaluation in minimally invasive surgery using force/torque signatures Surg. Endosc. 14 9 791 798 10.1007/s004640000230
58. Roberts M.: Integrating the Mind. Routledge, London (2007)
59. J. Rosen B. Hannaford C.G. Richards M.N. Sinanan 2001 Markov modeling of minimally invasive surgery based on tool/tissue interaction and force/torque signatures for evaluating surgical skills IEEE Trans. Biomed. Eng. 48 5 579 591 10.1109/10.918597 (Pubitemid 32378059)
60. P.M. Sanderson C. Fisher 1994 Exploratory sequential data analysis: foundations Hum. Comput. Interact. Special Issue ESDA 9 3-4 251 317
61. Santos P.J., Badre A.N.: Automatic chunk detection in human-computer interaction. In: Costabile, M.F., Catarci, T., Levialdi, S., Santucci, G. (eds) Proceedings of the Workshop on Advanced Visual Interfaces, pp. 69-77. Bari, Italy (1994)
62. H.A. Simon K. Gilmartin 1973 A simulation of memory for chess positions Cogn. Psychol. 5 29 46 10.1016/0010-0285(73)90024-8
63. Sukaviriya, P.N., Foley, J.D.: Supporting adaptive interfaces in a knowledge-based user interface environment. In: Proceedings of the 1st International Conference on Intelligent User Interfaces, pp. 107-113. Orlando, Florida, United States (1993)
64. R. Sun C.L. Giles 2001 Sequence learning: from recognition and prediction to sequential decision making" IEEE Intell. Syst. 16 4 67 70 10.1109/MIS.2001.1463065
65. J.E. Trumbly K.P. Arnett M.P. Martin 1993 Performance effect of matching computer interface characteristics and user skill level Int. J. Man-Mach. Stud. 38 4 713 724 10.1006/imms.1993.1033
66. Uehling, D.L., Wolf, K.: User action graphing effort (UsAGE). In: Conference on Human Factors in Computing Systems, pp. 290-291. Denver, Colorado, United States (1995)
67. K.P. Vaubel C.F. Gettys 1990 Inferring user expertise for adaptive interfaces Hum. Comput. Interact. 5 1 95 117 10.1207/s15327051hci0501-3
68. G.I. Webb M.J. Pazzani D. Billsus 2001 Machine learning for user modeling User Model. User-adapt. Interact. 11 1-2 19 29 1030.68889 10.1023/A: 1011117102175 (Pubitemid 32409022)
69. Witten I.H., Frank E.: Data Minning: Practical Machine Learning Tools and Techniques, 2nd Edn. Morgan Kaufmann, San Francisco (2005)
70. I. Zukerman D.W. Albrecht 2001 Predictive statistical models for user modeling User Model. User-adapt. Interact. 11 1-2 5 18 1030.68915 10.1023/A:1011175525451 (Pubitemid 32409021)