A paradigm for handwriting-based intelligent tutors

International Journal of Human Computer Studies

Volumn 70, Issue 11, 2012, Pages 866-887

  Anthony, Lisa ^a,b   Yang, Jie ^a   Koedinger, Kenneth R ^a  

^a Carnegie Mellon University ^*  (United States)

^b UNIVERSITY OF MARYLAND BALTIMORE COUNTY   (United States)

Author keywords

Cognitive tutors; Educational technology; Handwriting recognition; Human computer interaction; Intelligent tutoring systems; Interaction design; Mathematics; Pen input

Indexed keywords

COGNITIVE TUTORS; EDUCATIONAL TECHNOLOGY; HANDWRITING RECOGNITION; INTELLIGENT TUTORING SYSTEM; INTERACTION DESIGN; PEN INPUT;

ALGEBRA; COMPUTER AIDED INSTRUCTION; EDUCATION COMPUTING; ERROR DETECTION; MATHEMATICAL TECHNIQUES; STUDENTS;

CHARACTER RECOGNITION;

EID: 84866372231     PISSN: 10715819     EISSN: 10959300     Source Type: Journal    
DOI: 10.1016/j.ijhcs.2012.04.003     Document Type: Article

References (91)

1. Abowd, G.D.; 1999. Software engineering issues for ubiquitous computing. In: Proceedings of International Conference on Software Engineering, ISCE'99. ACM, NY, pp. 75-84.
2. Aleven, V.A.W.M.M.; Koedinger, K.R.; Popescu, O.A.; 2003. A tutorial dialog system to support self-explanation: evaluation and open questions. In: Proceedings of the International Conference on Articial Intelligence in Education, pp. 39-46. IOS Press, Amsterdam, the Netherlands.
3. AlphaCount, 2010. AlphaCount. 〈http://alphacount.wordpress.com/ 〉, (accessed September 10, 2011).
4. Anderson, R.H.; 1968. Syntax-directed Recognition of Hand-printed Two-dimensional Mathematics. PhD Thesis, Department of Engineering and Applied Mathematics, Harvard University.
5. Anthony, L.; Yang, J.; Koedinger, K.R.; 2005. Evaluation of multimodal input for entering mathematical equations on the computer. In: Proceedings of the ACM Conference on Human Factors in Computing (CHI), pp. 1184-1187. ACM Press, NY, USA.
6. Anthony, L.; Yang, J.; Koedinger, K.R.; 2007a. Benets of handwritten input for students learning algebra equation solving. In: Proceedings of the International Conference on Articial Intelligence in Education (AIEd), pp. 521-523. IOS Press, Amsterdamn, the Netherlands.
7. Anthony, L.; Yang, J.; Koedinger, K.R.; 2007b. Adapting handwriting recognition for applications in algebra learning. In: Proceedings of the ACM Workshop on Educational Multimedia and Multimedia Education, pp. 47-56. ACM Press, NY, USA.
8. L. Anthony, J. Yang, and K.R. Koedinger Toward next-generation, intelligent tutors: Adding natural handwriting input IEEE Multimedia 15 3 2008 64 68
9. Anthony, L. 2008. Developing Handwriting-based Intelligent Tutors to Enhance Mathematics Learning. PhD Thesis, Carnegie Mellon University, CMU-HCII-08-105.
10. Anthony, L.; Wobbrock, J.O.; 2010. A lightweight multistroke recognizer for user interface prototypes. In: Proceedings of Graphics Interface 2010, Gi '10. Canadian Information Processing Society, Toronto, Ont.; Canada, pp. 245-252.
11. Awal, A.-M.; Mouchere, H.; Viard-Gaudin, C.; 2010. The problem of handwritten mathematical expression recognition evaluation. In: 2010 12th International Conference on Frontiers in Handwriting Recognition. Kolkata, India, pp. 646-651.
12. Beal, C.; Johnson, W.L.; Rabrowski, R.; Wu, S.; 2005. Individualized feedback and simulation-based practice in the tactical language training system: an experimental evaluation. In: Proceedings of the Articial Intelligence in Education Conference (AIEd), pp. 749-749. IOS Press, Amsterdam, the Netherlands.
13. A. Belaid, and J.P. Haton A syntactic approach for handwritten mathematical formula recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 6 1 1984 105 111
14. D. Blostein, and A. Grbavec Recognition of mathematical notation P.S.P. Wang, H. Bunke, Handbook on Optical Character Recognition and Document Analysis 1996 World Scientic Publishing Company 557 582
15. D. Blostein, E. Lank, A. Rose, and R. Zanibbi User interfaces for on-line diagram recognition D. Blostein, Y.-B. Kwon, Graphics Recognition Algorithms and Applications 2002 Springer Berlin Heidelberg, Berlin 92 103
16. Borda, J.C.; 1781. Memoire sur les elections au scrutin. In: Histoire de l'Academie Royal des Sciences.
17. R.M. Brown On-line computer recognition of handprinted characters IEEE Transactions on Electronic Computing 13 1964 750 752
18. Buscaldi, D.; Rosso, P.; 2007. A bag-of-words based ranking method for the wikipedia question answering task. In: Peters, C.; Clough, P.; Gey, F.C.; Karlgren, J.; Magnini, B.; Oard, D.W.; de Rijke, M.; Stempfhuber, M. (Eds.), Evaluation of Multilingual and Multi-modal Information Retrieval: Seventh Workshop of the Cross-Language Evaluation Forum, Revised Selected Papers; Lecture Notes in Computer Science 4730, pp. 550-553. Springer Berlin, Heidelberg.
19. K.-F. Chan, and D.-Y. Yeung Mathematical expression recognition: a survey International Journal on Document Analysis and Recognition 3 2000 375 384
20. Chan, K.-F.; Yeung, D.-Y.; 2001. PenCalc: a novel application of on-line mathematical expression recognition technology. In: Proceedings of Sixth International Conference on Document Analysis and Recognition. Seattle, WA, USA, pp. 774-778.
21. Cheema, S.; LaViola J.; 2010. Towards intelligent motion inferencing in mathematical sketching. In: Proceedings of the 2010 International Conference on Intelligent User Interfaces, pp. 289-292.
22. M.T.H. Chi, M. Bassok, M.W. Lewis, P. Reimann, and R. Glaser Self-explanations: how students study and use examples in learning to solve problems Cognitive Science 13 1989 145 182
23. R.C. Clark, F. Nguyen, and J. Sweller Efciency in Learning: Evidence-Based Guidelines to Manage Cognitive Load 2006 Pfeiffer San Francisco, CA, USA
24. A.T. Corbett, K.R. Koedinger, and J.R. Anderson Intelligent tutoring systems M.G. Helander, T.K. Landauer, P.V. Prabhu, Handbook of Human-Computer Interaction 1997 Elsevier Science B.V Amsterdam, The Netherlands 849 874
25. Y.A. Dimitriadis, and J.L. Coronado Towards an art based mathematical editor that uses online handwritten symbol recognition Pattern Recognition 28 6 1995 807 822
26. Dimond, T.L.; 1957. Devices for reading handwritten characters. In: Proceedings of the Eastern Joint Computing Conferences, pp. 232-237.
27. E.S. Elliott, and C.S. Dweck Goals: an approach to motivation and achievement Journal of Personality and Social Psychology 54 1 1988 5 12
28. R. Fateman, T. Tokuyasu, B.P. Berman, and N. Mitchell Optical character recognition and parsing of typeset mathematics Journal of Visual Communication and Image Representation 7 1 1996 2 15
29. Frankish, C.; Hull, R.; Morgan, P.; 1995. Recognition accuracy and user acceptance of pen interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), pp. 503-510.
30. Freedman, R.; 1999. Atlas: a plan manager for mixed-initiative, multimodal dialogue. In: AAAI Workshop on Mixed-initiative Intelligence.
31. Goldberg, D.; Goodisman, A.; 1991. Stylus user interfaces for manipulating text. In: Proceedings of ACM Symposium on User Interface Software and Technology (UIST), pp. 127-135.
32. R. Glaser Components of a psychology of instruction: toward a science of design Review of Educational Research 46 1976 1 24
33. Graesser, A.; Moreno, K.N.; Marineau, J.C.; Adcock, A.B.; Olney, A.M.; Person, N.K.; 2003. Autotutor improves deep learning of computer literacy: is it the dialog or the talking head? In: Proceedings of the International Conference on Articial Intelligence in Education, pp. 47-54.
34. I. Guyon, and C. Warwick Handwriting as computer interface R.A. Cole, J. Mariani, H. Uszkoreit, G.B. Varile, A. Zaenen, A. Zampolli, Survey of the State of the Art in Human Language Technology 1998 Cambridge University Press Boston, MA 78 83
35. Hürst, W.; Yang, J.; Waibel, A.; 1998. Error repair in human handwriting: an intelligent user interface for automatic on-line handwriting recognition. In: Proceedings of the IEEE International Joint Symposia on Intelligence and Systems, pp. 389-395. IEEE.
36. Jackson, L.; 2004. Laptops, handhelds,or tabletpcs? Education World. 〈 http://www.education-world.com/a-tech/tech/tech198.shtmlS 〉 (accessed 09.09.2011)
37. Jiang, Y.; Tian, F.; Wang, H.; Zhang, X.; Wang, X.; Dai, G.; 2010. Intelligent understanding of handwritten geometry theorem proving. In: Proceedings of the 15th International Conference on Intelligent User Interfaces, IUI '10. ACM, NY, USA, pp. 119-128.
38. LaLomia, M.J.; 1994. User acceptance of handwritten recognition accuracy. In: Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI), p. 107.
39. Landay, J.A.; Myers, B.A.; 1995. Interactive sketching for the early stages of user interface design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co.; NY, USA, pp. 43-50.
40. K.R. Koedinger, and A.T. Corbett Cognitive tutors: technology bringing learning science to the classroom K. Sawyer, The Cambridge Handbook of the Learning Sciences 2006 Cambridge University Press 61 78
41. Koedinger, K.R.; Corbett, A.C.; Perfetti, C.; accepted. The knowledge-learning-instruction (KLI) framework: bridging the science-practice chasm to enhance robust student learning. Cognitive Science. 10.1111/j.1551-6709.2012.01245.x, in press.
42. A.L. Koerich, R. Sabourin, and C.Y. Suen Large vocabulary off-line handwriting recognition: a survey Pattern Analysis and Applications 6 2003 97 121
43. Labahn, G.; Lank, E.; Marzouk, M.S.; Bunt, A.; MacLean, S.; Tausky, D.; 2008. MathBrush: a case study for pen-based interactive mathematics. In: Proceedings of the Fifth Eurographics Symposium on Sketch-Based Interfaces and Modeling, SBIM '08. Annecy, France, pp. 143-150.
44. Lapointe, A.; Blostein, D.; 2009. Issues in performance evaluation: a case study of math recognition. In: 10th International Conference on Document Analysis and Recognition. Barcelona, Spain, pp. 1355-1359.
45. LaViola, J.J.; 2006. An initial evaluation of a pen-based tool for creating dynamic mathematical illustrations. In: Proceedings of Eurographics Workshop on Sketch-Based Interfaces and Modeling, EG Workshop Series, pp. 157-164.
46. 2: a system for the creation and exploration of mathematical sketches ACM Transactions on Graphics (Proceedings of SIGGRAPH 2004) 23 3 2004 432 440
47. Lee, W.; de Silva, R.; Peterson, E.J.; Calfee, R.C.; Stahovich, T.F.; 2007. Newton's pen: a pen-based tutoring system for statics. In: Proceedings of the Fourth Eurographics Workshop on Sketch-based Interfaces and Modeling, SBIM '07. ACM, NY, USA, pp. 59-66.
48. V.I. Levenshtein Binary codes capable of correcting deletions, insertions, and reversals Soviet Physics Doklady 10 8 1966 707 710
49. Li, C.; Miller, T.S.; Zeleznik, R.C.; Jr, J.J.L.; 2008. AlgoSketch: algorithm sketching and interactive computation. In: Proceedings of the Fifth Eurographics Symposium on Sketch-based Interfaces and Modeling, SBIM '08. Annecy, France, pp. 175-182.
50. Lin, J.; Newman, M.W.; Hong, J.I.; Landay, J.A.; 2000. DENIM: finding a tighter fit between tools and practice for web site design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '00). ACM, NY, USA, pp. 510-517.
51. Litman, D.J.; Silliman, S.; 2004. Itspoke: an intelligent tutoring spoken dialogue system. In: Proceedings of the Human Language Technology Conference: Fourth Meeting of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL) (Companion Proceedings), pp. 233-236.
52. Littin, R.H.; 1995. Mathematical Expression Recognition: Parsing Pen/Tablet Input in Real-Time Using LR Techniques. Master's Thesis, University of Waikato, Hamilton, New Zealand.
53. Liu, C.-L.; Yin, F.; Wang, D.-H.; Wang, Q.-F.; 2010. Chinese handwriting recognition contest 2010. In: 2010 Chinese Conference on Pattern Recognition (CCPR). IEEE, p. 5.
54. I.S. MacKenzie, and L. Chang A performance comparison of two handwriting recognizers Interacting with Computers 11 1999 283 297
55. J. Mankoff, and G. Abowd Error Correction Techniques for Handwriting, Speech, and Other Ambiguous or Error Prone Systems. Technical Report GIT-GVU-99-18 1999 Georgia Institute of Technology
56. Märgner, V.; Abed, H.E.; 2010. ICFHR 2010 - Arabic handwriting recognition competition. In: 2010 12th International Conference on Frontiers in Handwriting Recognition. Kolkata, India, pp. 709-714.
57. Matsakis, N.E.; 1999. Recognition of Handwritten Mathematical Expressions. Master's Thesis, Massachusetts Institute of Technology.
58. McLaren, B.M.; Lim, S.-J.; Koedinger, K.R.; 2008. When is assistance helpful to learning? Results in combining worked examples and intelligent tutoring. In: Woolf, B.P.; Aïmeur, E.; Nkambou, R.; Lajoie, S. (Eds.), Proceedings of the Ninth International Conference on Intelligent Tutoring Systems (ITS-08), Lecture Notes in Computer Science, 5091. Springer Berlin Heidelberg, Berlin, pp. 677-680.
59. Microsoft, 2005. Equation Pack for Windows XP Tablet PC Edition. 〈http://www.microsoft.com/windowsxp/downloads/tabletpc/educationpack/ overview4.mspx〉, (accessed September 10, 2011).
60. Miller, E.G.; Viola, P.A.; 1998. Ambiguity and constraint in mathematical expression recognition. In: Proceedings of the AAAI Conference on Articial Intelligence (AAAI), pp. 784-789. AAAI Press.
61. O'Connell, T.; Li, C.; Miller, T.S.; Zeleznik, R.C.; LaViola, J.J.; 2009. A usability evaluation of AlgoSketch. In: Proceedings of the Sixth Eurographics Symposium on Sketch-based Interfaces and Modeling, SBIM '09. New Orleans, Louisiana, p. 149.
62. S. Oviatt, and B. Adams Designing and evaluating conversational interfaces with animated characters J. Cassell, J. Sullivan, S. Prevost, E. Churchill, Embodied Conversational Agents 2000 MIT Press Cambridge, MA, USA 319 343
63. Oviatt, S.; Arthur, A.; Cohen, J.; 2006. Quiet interfaces that help students think. In: Proceedings of the ACM Symposium on User Interface Software and Technology, pp. 191-200.
64. S. Oviatt, M. MacEachern, and G.-A. Levow Predicting hyperarticulate speech during human-computer error resolution Speech Communication 24 2 1998 87 110 May 1998
65. H. Pashler, P. Bain, B. Bottge, A. Graesser, K. Koedinger, M. McDaniel, and J. Metcalfe Organizing Instruction and Study to Improve Student Learning (NCER 2007-2004) 2007 National Center for Education Research, Institute of Education Sciences, U.S. Department of Education Washington, DC
66. Purcell, S.C.; 1977. Understanding Hand-Printed Algebra for Computer Tutoring. Master's Thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.
67. J.C. Read, S.J. MacFarlane, and C. Casey Where's the 'm' on the keyboard, mummy? In Womens' Engineering Society 2000
68. J.C. Read, S.J. MacFarlane, and C. Casey Can natural language recognition technologies be used to enhance the learning experience of young children? Computers and Learning 2001
69. Read, J.C.; MacFarlane, S.J.; Casey, C.; 2001b. Measuring the usability of text input methods for children. In: Proceedings of BCS British-HCI, pp. 559-572. Springer Verlag.
70. Read, J.C.; 2002. Optimising text input for young children using computers for creative writing tasks. In: Proceedings of BCS British-HCI, pp. 503-519. Springer Verlag.
71. Read, J.C.; MacFarlane, S.J.; Casey, C.; 2002. Pens behaving badly - usability of pens and graphics tablets for text entry with children. In: Adjunct Proceedings of the ACM Symposium on User Interface Software and Technology, pp. 21-22. ACM Press.
72. Read, J.C.; MacFarlane, S.J.; Casey, C.; 2003a. A comparison of two on-line handwriting recognition methods for unconstrained text entry by children. In: Proceedings of BCS British-HCI, pp. 29-32. Research Press International, Bristol, UK.
73. Read, J.C.; MacFarlane, S.J.; Casey, C.; 2003b. Good enough for what?: acceptance of handwriting recognition errors by child users. In: Proceedings of the 2003 conference on Interaction design and children, pp. 155-155, NY, USA. ACM Press.
74. J. Read A study of the usability of handwriting recognition for text entry by children Interacting with Computers 19 2007 57 69
75. Renda, M.E.; Straccia, U.; 2003. Web metasearch: rank vs. score based rank aggregation methods. In: Proceedings of the ACM symposium on Applied computing, pp. 841-846, NY, USA. ACM.
76. J. Roschelle, D. Tatar, S.R. Chaudbury, Y. Dimitriadis, C. Patton, and C. DiGiano Ink, improvisation, and interactive engagement: learning with tablets Computer 40 2007 42 48
77. R.J.C.M. Salden, K.R. Koedinger, A. Renkl, V. Aleven, and B.M. McLaren Accounting for beneficial effects of worked examples in tutored problem solving Educational Psychology Review 22 4 2010 379 392
78. Santos, P.J.; Baltzer, A.J.; Badre, A.N.; Henneman, R.L.; Miller, M.S.; 1992. On handwriting recognition performance: some experimental results. In: Proceedings of the Human Factors Society 36th Annual Meeting, pp. 283-287.
79. Schomaker, L.; 1994. User-interface aspects in recognizing connected-cursive handwriting. In: Proceedings of the IEE Colloquium on Handwriting and Pen-based Input, 8/1-8/3.
80. S. Smithies, K. Novins, and J. Arvo Equation entry and editing via handwriting and gesture recognition Behaviour and Information Technology 20 2001 53 67
81. M. Suzuki, T. Kanahori, N. Ohtake, and K. Yamaguchi An integrated OCR software for mathematical documents and its output with accessibility K. Miesenberger, J. Klaus, W.L. Zagler, D. Burger, Computers Helping People with Special Needs 2004 Springer Berlin Heidelberg, Berlin, Heidelberg 648 655
82. J. Sweller Cognitive load during problem solving: effects on learning Cognitive Science 12 1988 257 285
83. Tabachneck, H.J.M.; Koedinger, K.R.; Nathan, M.J.; 1995. A cognitive analysis of the task demands of early algebra. In: Proceedings of the 17th Annual Conference of the Cognitive Science Society, pp. 397-402.
84. Tapia, E.; Rojas, R.; 2003. Recognition of on-line handwritten mathematical formulas in the e-chalk system. In: Proceedings of the International Conference on Document Analysis and Recognition (ICDAR), pp. 980-984. IEEE.
85. Tapia, E.; Rojas, R.; 2005. Recognition of on-line handwritten mathematical expressions in the e-chalk system - an extension. In: Proceedings of the Eighth International Conference on Document Analysis and Recognition (ICDAR'05). Seoul, South Korea, pp. 1206-1210 Vol. 2.
86. Valentino-Devries, J.; 2010. Using the ipad to connect:parents, therapists use apple tablet to communicate with special needs kids. Wall Street Journal 〈 http://online.wsj.com/article/ SB10001424052748703440004575547971877769154.htmlS 〉(accessed 09.09.11).
87. Wood, C.; 2002. Technologyandeducation. PC Magazine. 〈 http://www.pcmag.com/article2/0,4149,15154,00.aspS 〉 (accessed 09.09.11).
88. xThink, 2003. MathJournal. 〈http://www.xthink.com/MathJournal. html〉, (accessed 10.09.11).
89. R. Zanibbi, D. Blostein, and J.R. Cordy Recognizing mathematical expressions using tree transformation IEEE Transactions on Pattern Analysis and Machine Intelligence 24 2002 1455 1467
90. Zanibbi, R.; Pillay, A.; Mouchere, H.; Viard-Gaudin, C.; Blostein, D.; 2011. Stroke-based performance metrics for handwritten mathematical expressions. In: Proceedings of the 2011 International Conference on Document Analysis and Recognition (ICDAR'11), Beijing, China, pp. 334-338.
91. R. Zeleznik, T. Miller, C. Li, and J.J. LaViola MathPaper: mathematical sketching with fluid support for interactive computation A. Butz, B. Fisher, A. Krüger, P. Olivier, M. Christie, Smart Graphics 2008 Springer Berlin Heidelberg, Berlin, Heidelberg 20 32