Recognizing sketched multistroke primitives

ACM Transactions on Interactive Intelligent Systems

Volumn 1, Issue 1, 2011, Pages

  Hammond, Tracy ^a   Paulson, Brandon ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Intelligent user interfaces; Neural networks; Primitive recognition; Sketch recognition

Indexed keywords

NEURAL NETWORKS;

BASIC BUILDING BLOCK; ELECTRONIC MEDIUM; INTELLIGENT USER INTERFACES; LINEAR SEARCH ALGORITHMS; PRIMITIVE RECOGNITION; SEARCH TECHNIQUE; SKETCH RECOGNITION; VISUAL GRAMMAR;

ALGORITHMS;

EID: 84983585909     PISSN: 21606455     EISSN: 21606463     Source Type: Journal    
DOI: 10.1145/2030365.2030369     Document Type: Article

References (68)

1. ALVARADO, C. AND DAVIS, R. 2001. Resolving ambiguities to create a natural computer-based sketching environment. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). AAAI, Stanfords, CA, 1365-1371.
2. ALVARADO, C. AND DAVIS, R. 2004. Sketchread: A multi-domain sketch recognition engine. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (UIST'04), ACM, New York, 23-32.
3. ANTHONY, L. AND WOBBROCK, J. 2010. A lightweight multistroke recognizer for user interface prototypes. In Proceedings of the Graphics Interface Conference. Canadian Information Processing Society, Ottawa, Ontario, 245-252.
4. APTE, A., VO, V., AND KIMURA, T. D. 1993. Recognizing multistroke geometric shapes: An experimental evaluation. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, New York, 121-128.
5. BAILEY, B. P. AND KONSTAN, J. A. 2003. Are informal tools better? Comparing DEMAIS, pencil and paper, and authorware for early multimedia design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'03). ACM, New York, 313-320.
6. BLACK, A. 1990. Visible planning on paper and on screen: The impact of working medium on decision-making by novice graphic designers. Behav. Inf. Technol. 9, 4, 283-296.
7. BLICKENSTORFER, C. H. 1995. Graffiti: Wow! Pen Comput. Mag. (1/30/95).
8. CALHOUN, C., STAHOVICH, T. F., KURTOGLU, T., AND KARA, L. B. 2002. Recognizing multi-stroke symbols. In Proceedings of the AAAI Spring Symposium - Sketch Understanding. AAAI Press, Stanford, CA, 15-23.
9. COSTAGLIOLA, G., DEUFEMIA, V., POLESE, G., AND RISI, M. 2004. A parsing technique for sketch recognition systems. In Proceedings of the IEEE Symposium on Visual Languages and Human Centric Computing. IEEE, Los Alamitos, CA, 19-26.
10. COSTAGLIOLA, G., DEUFEMIA, V., AND RISI, M. 2005. Sketch grammars: A formalism for describing and recognizing diagrammatic sketch languages. In Proceedings of the 8th International Conference on Document Analysis and Recognition (ICDAR'05). IEEE. Los Alamitos, CA, 1226-1231.
11. DIXON, D. M. 2009. A methodology using assistive sketch recognition for improving a person's ability to draw. M.S. thesis, Texas A&M University, College Station, TX.
12. FONSECA, M. J. AND JORGE, J. A. 2000. Using fuzzy logic to recognize geometric shapes interactively. In Proceedings of the 9th IEEE International Conference on Fuzzy Systems. Vol. 1. IEEE, Los Alamitos, CA, 291-296.
13. FONSECA, M. J.,PIMENTEL, C., AND JORGE, J. 2002. Cali: An online scribble recognizer for calligraphic interfaces. In Proceedings of the AAAI Spring Symposium. AAAI Press, Stanford, CA, 51-58.
14. FORBUS, K. D., USHER, J., LOVETTE, A., LOCKWOOD, K., AND WETZEL, J. 2008. Cogsketch: Open-domain sketch understanding for cognitive science research and for education. In Proceedings of the Eurographics Symposium on Sketch-based Interfaces and Modeling (SBIM). ACM, New York.
15. FUTRELLE, R. P. AND NIKOLAKIS,N. 1995. Efficient analysis of complex diagrams using constraint-based parsing. In Proceedings of the International Conference on Document Analysis and Recognition. 782.
16. GOEL, V. 1995. Sketches of Thought. MIT Press, Cambridge, MA.
17. GOLDBERG, D. AND RICHARDSON, C. 1993. Touch-typing with a stylus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM, New York, 80-87.
18. GROSS, M. D. AND DO, E. Y.-L. 1996. Ambiguous intentions: A paper-like interface for creative design. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, New York, 183-192.
19. GROSS, M. D. AND DO, E. Y.-L. 2000. Drawing on the back of an envelope: A framework for interacting with application programs by freehand drawing. Comput. Graph. 24, 6, 835-849.
20. HAMMOND, T. 2007. Hammond, T. 2007. LADDER: A perceptually-based language to simplify sketch recognition user interfaces development. Ph.D. dissertation, MIT, Cambridge, MA.
21. HAMMOND, T. AND DAVIS, R. 2002. Tahuti: A geometrical sketch recognition system for UML class diagrams. In Proceedings of the AAAI Spring Symposium. AAAI Press, Stanford, CA, 59-66.
22. HAMMOND, T. AND DAVIS, R. 2005. LADDER, a sketching language for user interface developers. Comput. Graph. 29, 4, 518-532.
23. HAMMOND, T., LOGSDON, D., PESCHEL, J., JOHNSTON, J., TAELE, P., WOLIN, A., AND PAULSON, B. 2010. A sketch recognition interface that recognizes hundreds of shapes in course-of-action diagrams. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM, New York.
24. HAMMOND, T. A. AND DAVIS,R. 2009. Recognizing interspersed sketches quickly. In Proceedings of the Conference on the Graphics Interface (GI'09). ACM, New York, 157-166.
25. HSE, H. H. AND NEWTON, A. R. 2005. Recognition and beautification of multi-stroke symbols in digital ink. Comput. Graph. 29, 4, 533-546.
26. JOHNSTON, J. AND HAMMOND, T. 2010. Computing confidence values for geometric constraints for use in sketch recognition. In Proceedings of the Eurographics Symposium on Sketch-based Interfaces and Modeling (SBIM). ACM, New York.
27. KARA, L. B. AND STAHOVICH, T. F. 2004. Hierarchical parsing and recognition of hand-sketched diagrams. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, New York, 13-22.
28. KURTOGLU, T. AND STAHOVICH, T. F. 2002. Interpreting schematic sketches using physical reasoning. In Proceedings of the AAAI Spring Symposium. AAAI Press, Stanford, CA, 78-85.
29. LAVIOLA, J. J. AND ZELEZNIK, R. C. 2004. Mathpad2: A system for the creation and exploration of mathematical sketches. ACM Trans. Graph. 23, 3, 432-440.
30. LI, X. AND YEUNG, D.-Y. 1997. On-line handwritten alphanumeric character recognition using dominant points in strokes. Patt. Recogn. 30, 1, 31-44.
31. LONG JR., A. C., LANDAY, J. A., ROWE, L. A., AND MICHIELS, J. 2000. Visual similarity of pen gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM, New York, 360-367.
32. MAHONEY, J. V. AND FROMHERZ, M. P. J. 2001. Interpreting sloppy stick figures by graph rectification and constraint-based matching. In Proceedings of the 4th IAPR International Workshop on Graphics Recognition. ACM, New York.
33. MAHONEY, J. V. AND FROMHERZ, M. P. J. 2002. Three main concerns in sketch recognition and an approach to addressing them. In Proceedings of the AAAI Spring Symposium. AAAI Press, Stanford, CA, 105-112.
34. NEWMAN, W. M. AND SPROULL, R. F. 1973. Principles of Interactive Computer Graphics 2nd Ed., McGraw-Hill, New York, 202-209.
35. OLTMANS, M. 2007. Envisioning sketch recognition: A local feature-based approach to recognizing informal sketches. Ph.D. dissertation, MIT, Cambridge, MA
36. OLTMANS, M. AND DAVIS, R. 2001. Naturally conveyed explanations of device behavior. In Proceedings of the Workshop on Perceptive User Interfaces (PUI'01). ACM, New York, 1-8.
37. OUYANG, T. Y. AND DAVIS, R. 2007. Recognition of hand drawn chemical diagrams. In Proceedings of theNational Conference on Artificial Intelligence. AAAI Press, 846-851.
38. OUYANG, T. Y. AND DAVIS, R. 2009. A visual approach to sketched symbol recognition. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press, Stanford, CA, 1463-1468.
39. PASTERNAK, B. AND NEUMANN, B. 1993. Adaptable drawing interpretation using object-oriented and constraintbased graphic specification. In Proceedings of the 2nd International Conference on Document Analysis and Recognition. IEEE, Los Alamitos, CA, 359-364.
40. PAULSON, B. 2010. Rethinking pen input interaction: Enabling freehand sketching through improved primitive recognition. Ph.D. dissertation, Texas A&M University, College Station, TX.
41. PAULSON, B., EOFF, B., WOLIN, A., JOHNSTON, J., AND HAMMOND, T. 2008. Sketch-based educational games: Drawing kids away from traditional interfaces. In Proceedings of the 7th International Conference on Interaction Design and Children (IDC'08), ACM, New York, 133-136.
42. PAULSON, B. AND HAMMOND, T. 2008a. MARQS: Retrieving sketches using domain and style-independent features learned from a single example using a dual-classifier. J. Multi-Modal User Interfaces 2, 1, 3-11.
43. PAULSON, B. AND HAMMOND, T. 2008b. PaleoSketch: Accurate primitive sketch recognition and beautification. In Proceedings of the International Conference on Intelligent User Interfaces. ACM, New York, 1-10.
44. PAULSON, B., RAJAN, P.,DAVALOS, P.,GUTIERREZ-OSUNA, R., AND HAMMOND, T. 2008. What!?! No Rubine features?: Using geometric-based features to produce normalized confidence values for sketch recognition. In Proceedings of the VL/HCC Sketch Tools for Diagramming Workshop. IEEE, Los Alamitos, CA, 57-63.
45. PAULSON, B.,WOLIN, A., JOHNSTON, J., AND HAMMOND, T. 2008. SOUSA: Sketch-based online user study applet. In Proceedings of the Eurographics Symposium on Sketch-based Interfaces and Modeling (SBIM). ACM, New York, 81-88.
46. PESCHEL, J. AND HAMMOND, T. 2008. STRAT: A sketched-truss recognition and analysis tool. In Proceedings of the International Conference on Distributed Multimedia Systems. Knowledge Systems Institute, Boston, MA, 282-287.
47. PITTMAN, J. A.,SMITH, I. A.,COHEN, P. R.,OVIATT, S. L., AND YANG, T.-C. 1996. Quickset: A multimodal interface for military simulation. In Proceedings of the 6th Conference on Computer-Generated Forces and Behavioral Representation (CGF-BR'96). 217-224.
48. PLIMMER, B. AND FREEMAN, I. 2007. A toolkit approach to sketched diagram recognition. In Proceedings of the 21st British CHI Group Annual Conference on HCI (BCS-HCI'07). ACM, New York, 205-213.
49. RUBINE, D. 1991. Specifying gestures by example. In Proceedings of the International Conference and Exhibition on Computer Graphics and Interactive Techniques (SIGGRAPH). ACM, New York, 329-337.
50. SAUND, E. AND LANK, E. 2003. Stylus input and editing without prior selection of mode. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (UIST '03). ACM, New York, 213-216.
51. SAUND, E.,MAHONEY, J., FLEET, D., LARNER, D., AND LANK, E. 2002. Perceptual organization as a foundation for intelligent sketch editing. In Proceedings of the AAAI Spring Symposium. AAAI Press, Stanford, CA, 118-125.
52. SAUND, E. AND MORAN, T. P. 1995. Perceptual organization in an interactive sketch editing application. In Proceedings of the IEEE 5th InternationalConference on Computer Vision (ICCV'95). IEEE, LosAlamitos, CA, 597-607.
53. SEZGIN, T. M., STAHOVICH, T., AND DAVIS, R. 2001. Sketch based interfaces: Early processing for sketch understanding. In Proceedings of the Workshop on Perceptive User Interfaces (PUI'01). ACM, New York, 1-8.
54. SHILMAN, M., PASULA, H.,RUSSELL, S., AND NEWTON, R. 2002. Statistical visual language models for ink parsing. In Proceedings of the AAAI Spring Symposium. AAAI Press, Stanford, CA, 126-132.
55. STAHOVICH, T. F. 1996. Sketchit: A sketch interpretation tool for conceptual mechanical design. Tech. rep. 1573, Artificial Intelligence Laboratory.
56. TAELE, P. AND HAMMOND, T. 2008. Using a geometric-based sketch recognition approach to sketch Chinese radicals. In Proceedings of the National Conference on Artificial Intelligence. AAAI Press, 1832-1833.
57. TAELE, P. AND HAMMOND, T. 2009. Hashigo: A next-generation sketch interactive system for Japanese Kanji. In Proceedings of the 21st Innovative Applications of Artificial Intelligence Conference (IAAI'09). AAAI Press, 153-158.
58. TAELE, P., PESCHEL, J., AND HAMMOND, T. 2009. A sketch interactive approach to computer-assisted biology instruction. In Proceedings of the Workshop on Sketch Recognition at the International Conference of Intelligent User Interfaces (IUI). ACM, New York.
59. TARJAN, R. 1972. Depth-first search and linear graph algorithms. SIAM J. Computing 1, 2, 146-160.
60. TENNESON, D. AND BECKER, S. 2005. Chempad: Generating 3D molecules from 2D sketches. In Proceedings of the International Conference and Exhibition on Computer Graphics and Interactive Techniques (SIGGRAPH). ACM, New York, 87.
61. VAN SOMMERS, P. 1984. Drawing and Cognition: Descriptive and Experimental Studies of Graphic Production Processes. Cambridge University Press, Cambridge, UK.
62. WOBBROCK, J. O.,WILSON, A. D., AND LI, Y. 2007. Gestures without libraries, toolkits or training: A $1 recognizer for user interface prototypes. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). ACM, New York, 159-168.
63. WOLIN, A., EOFF, B., AND HAMMOND, T. 2009. Search your mobile sketch: Improving the ratio of interaction to information onmobile devices. In Proceedings of theWorkshop on Sketch Recognition at the International Conference of Intelligent User Interfaces (IUI), ACM, New York.
64. WONG, Y. Y. 1992. Rough and ready prototypes: Lessons from graphic design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM, New York, 83-84.
65. YEUNG, L.,PLIMMER, B.,LOBB, B., AND ELLIFFE,D. 2008. Effect of fidelity in diagram presentation. In Proceedings of the 22nd British HCI Group Annual Conference on HCI (BCS-HCI'08). British Computer Society, 35-44.
66. YU, B. AND CAI, S. 2003. A domain-independent system for sketch recognition. In Proceedings of the 1st International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE'03). ACM, New York, 141-146.
67. ZELEZNIK, R. AND MILLER, T. 2006. Fluid inking: Augmenting the medium of free-form inking with gestures. In Proceedings of Graphics Interface Conference (GI'06). Canadian Information Processing Society, Ottawa, Ontario, 155-162.
68. ZHAO, R. 1993. Incremental recognition in gesture-based and syntax-directed diagram editors. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM, New York, 95-100.