Waken: Reverse engineering usage information and interface structure from software videos

UIST'12 - Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology

Volumn , Issue , 2012, Pages 83-92

  Banovic, Nikola ^a,b   Grossman, Tovi ^a   Matejka, Justin ^a   Fitzmaurice, George ^a  

^a AUTODESK RESEARCH   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Pixel based reverse engineering; Tutorials; Videos

Indexed keywords

FRAME DIFFERENCING; INTERFACE STRUCTURES; PRIOR KNOWLEDGE; TOOLTIPS; TUTORIALS; UI COMPONENTS; VIDEO PLAYERS; VIDEOS;

GRAPHICAL USER INTERFACES;

REVERSE ENGINEERING;

EID: 84869005919     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Bergman, L., Castelli, V., Lau, T., and Oblinger, D. 2005. DocWizards: a system for authoring follow-me documentation wizards. ACM UIST, 191-200.
2. Chang, T., Yeh, T., and Miller, R. C. 2010. GUI testing using computer vision. ACM CHI, 1535-1544.
3. Chang, T., Yeh, T., and Miller, R. C. 2011. Associating the visual representation of user interfaces with their internal structures and metadata. ACM UIST, 245-256.
4. Cheng, K., Luo, S., and Chen, B. 2009. SmartPlayer: usercentric video fast-forwarding. ACM CHI, 789-798.
5. Dixon M. and Fogarty, J. 2010. Prefab: implementing advanced behaviors using pixel-based reverse engineering of interface structure. ACM CHI, 1525-1534.
6. Dixon, M., Leventhal, D., and Fogarty, J. 2011. Content and hierarchy in pixel-based methods for reverse engineering interface structure. ACM CHI, 969-978.
7. Dixon M., Fogarty, J., and Wobbrock, J. 2012. A general-purpose target-aware pointing enhancement using pixel-level analysis on graphical interfaces. ACM CHI, 3167-3176.
8. Dong, T., Dontcheva, M., Joseph, D., Karahalios, K., Newman, M. W., and Ackerman, M. S. 2012. Discovery-based Games for Learning Software. ACM CHI, 2083-2086.
9. Dragicevic, P., Ramos, G., Bibliowitcz, J., Nowrouzezahrai, D., Balakrishnan, R., and Singh, K. 2008. Video browsing by direct manipulation. ACM CHI, 237-246.
10. Fernquist, J., Grossman, T., and Fitzmaurice, G. 2011. Sketch-sketch revolution: an engaging tutorial system for guided sketching and application learning. ACM UIST, 373-382.
11. Grabler, F., Agrawala, M., Li, W., Dontcheva, M., and Igarashi, T. 2009. Generating photo manipulation tutorials by demonstration. ACM SIGGRAPH. 66. 9 pages.
12. Grossman, T. and Fitzmaurice, G. 2010. ToolClips: an investigation of contextual video assistance for functionality understanding. ACM CHI, 1515-1524.
13. Grossman, T., Matejka, J., and Fitzmaurice, G. 2010. Chronicle: capture, exploration, and playback of document workflow histories. ACM UIST, 143-152.
14. Harrison, S. M. 1995. A comparison of still, animated, or nonillustrated online help with written or spoken instructions in a graphical user interface. ACM CHI, 82-89.
15. Hategekimana, C., Gilbert, S., and Blessing, S. 2008. Effectiveness of using an intelligent tutoring system to train users on off-the-shelf software. Society for Info. Tech. and Teacher Education Int'l Conf., AACE.
16. Hu, M. 1962. Visual pattern recognition by moment invariants. IRE Trans. Inf. Theory IT-8, 8, 1409-1420.
17. Huang, J. and Twidale, M. B. 2007. Graphstract: minimal graphical help for computers. ACM UIST, 203-212.
18. Hurst, A., Hudson, S. E., and Mankoff, J. 2010. Automatically identifying targets users interact with during real world tasks. ACM IUI, 11-20.
19. Kelleher, C. and Pausch, R. 2005. Stencils-based tutorials: design and evaluation. ACM CHI, 541-550.
20. Knabe, K. 1995. Apple guide: a case study in user-aided design of online help. ACM CHI, 286-287.
21. Matejka, J., Grossman, T., and Fitzmaurice, G. 2011. Ambient help. ACM CHI, 2751-2760.
22. Nakamura, T. and Igarashi, T. 2008. An application-independent system for visualizing user operation history. ACM UIST, 23-32.
23. Palmiter, S. and Elkerton, J. 1991. An evaluation of animated demonstrations of learning computer-based tasks. ACM CHI, 257-263.
24. Petrovic, N., Jojic, N., and Huang, T. S. 2005. Adaptive Video Fast Forward. Multimed. Tools Appl. 26, 3.
25. Pongnumkul, S., Wang, J., Ramos, G., and Cohen, M. 2010. Content-aware dynamic timeline for video browsing. ACM UIST, 139-142.
26. Pongnumkul, S., Dontcheva, M., Li, W., Wang, J., Bourdev, L., Avidan, S., and Cohen, M. F. 2011. Pause-and-play: automatically linking screencast video tutorials with applications. ACM UIST, 135-144.
27. Ramesh, V., Hsu, C., Agrawala, M., and Hartmann, B. 2011. ShowMeHow: translating user interface instructions between applications. ACM UIST, 127-134.
28. Shneiderman, B. 1983. Direct Manipulation: A Step Beyond Programming Languages. Compt. 16, 8, 57-69.
29. Suzuki, S. and Abe, K. 1985. Topological structural analysis of digitized binary images by border following. Comput. Vision Graph., 30(1), 32-46.
30. Yeh, T., Chang, T., and Miller R. C. 2009. Sikuli: using GUI screenshots for search and automation. ACM UIST, 183-192.
31. Yeh, T., Chang, T., Xie, B., Walsh, G., Watkins, I., Wongsuphasawat, K., Huang, M., Davis, L. S., and Bederson, B. B. 2011. Creating contextual help for GUIs using screenshots. ACM UIST, 145-154.