Where are my intelligent assistant's mistakes? A systematic testing approach

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6654 LNCS, Issue , 2011, Pages 171-186

  Kulesza, Todd ^a   Burnett, Margaret ^a   Stumpf, Simone ^b   Wong, Weng Keen ^a   Das, Shubhomoy ^a   Groce, Alex ^a   Shinsel, Amber ^a   Bice, Forrest ^a   McIntosh, Kevin ^a  

^a Oregon State University   (United States)

^b CITY UNIVERSITY   (United Kingdom)

Author keywords

end user development; end user programming; end user software engineering; Intelligent assistants; machine learning; testing

Indexed keywords

END USER DEVELOPMENT; END USER PROGRAMMING; END-USER SOFTWARE ENGINEERING; INTELLIGENT ASSISTANTS; MACHINE-LEARNING;

COMPUTER PROGRAMMING; ENGINEERING RESEARCH; FORECASTING; HUMAN COMPUTER INTERACTION; LEARNING SYSTEMS; SAFETY TESTING; SOFTWARE ENGINEERING; SOFTWARE TESTING;

STATISTICAL TESTS;

EID: 79959982085     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-21530-8_14     Document Type: Conference Paper

References (32)

1. Abraham, R., Erwig, M.: AutoTest: A tool for automatic test case generation in spreadsheets. In: Proc. VL/HCC, pp. 43-50. IEEE, Los Alamitos (2006)
2. Baeza-Yates, R., Ribeiro-Neto, B.: Modern Information Retrieval. Addison-Wesley Longman Publishing Co., Inc., Boston (1999)
3. Beizer, B.: Software Testing Techniques. International Thomson Computer Press (1990)
4. Blackwell, A.: First steps in programming: A rationale for attention investment models. In: Proc. HCC, pp. 2-10. IEEE, Los Alamitos (2002)
5. Burnett, M., Cook, C., Rothermel, G.: End-user software engineering. Comm. ACM 47(9), 53-58 (2004)
6. Chang, C., Lin, C.: LIBSVM: A library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm
7. Fisher, M., Cao, M., Rothermel, G., Brown, D., Cook, C., Burnett, M.: Integrating automated test generation into the WYSIWYT spreadsheet testing methodology. ACM Trans. Software Engineering and Methodology 15(2), 150-194 (2006)
8. Frankl, P., Weiss, S.: An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Trans. Software Eng. 19(3), 202-213 (1993)
9. Glass, A., McGuinness, D., Wolverton, M.: Toward establishing trust in adaptive agents. In: Proc. IUI, pp. 227-236. ACM, New York (2008)
10. Gmail Priority Inbox: Get through your email faster, http://google.com/ mail/help/priority-inbox.html (accessed September 16, 2010)
11. Green, T., Petre, M.: Usability analysis of visual programming environments: A cognitive dimensions framework. J. Visual Languages and Computing 7(2) (June 1996)
12. Grigoreanu, V., Cao, J., Kulesza, T., Bogart, C., Rector, K., Burnett, M., Wiedenbeck, S.: Can feature design reduce the gender gap in end-user software development environments? In: Proc. VL/HCC, pp. 149-156. IEEE, Los Alamitos (2008)
13. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, Heidelberg (2003)
14. IEEE, IEEE Standard Glossary of Software Engineering Terminology (IEEE Std610.12-1990) (1990)
15. Klann, M., Paterno, F., Wulf, V.: Future perspectives in end-user development. In: Lieberman, H., Paterno, F., Wulf, V. (eds.) End-User Development. Springer, Heidelberg (2006)
16. Kniesel, G., Rho, T.: Newsgroup data set (2005), http://www.ai.mit.edu/ jrennie/20newsgroups
17. Kulesza, T., Wong, W., Stumpf, S., Perona, S., White, R., Burnett, M., Oberst, I., Ko, A.: Fixing the program my computer learned: Barriers for end users, challenges for the machine. In: Proc. IUI, pp. 187-196. ACM, New York (2009)
18. Kulesza, T., Stumpf, S., Burnett, M., Wong, W., Riche, Y., Moore, T., Oberst, I., Shinsel, A., McIntosh, K.: Explanatory debugging: Supporting end-user debugging of machinelearned programs. In: Proc. VL/HCC. IEEE, Los Alamitos (2010)
19. Lawrance, J., Bogart, C., Burnett, M., Bellamy, R., Rector, K., Fleming, S.: How programmers debug, revisited: An information foraging theory perspective. IEEE Trans. Software Engineering (2011)
20. Lim, B., Dey, A., Avrahami, D.: Why and why not explanations improve the intelligibility of context-aware intelligent systems. In: Proc. CHI, pp. 2119-2128. ACM, New York (2009)
21. Lim, B., Dey, A.: Toolkit to support intelligibility in context-aware applications. In: Proc. Int. Conf. Ubiquitous Computing. ACM, New York (2010)
22. Miller, R., Myers, B.: Outlier finding: Focusing user attention on possible errors. In: Proc. UIST, pp. 81-90. ACM, New York (2001)
23. Raghavan, H., Madani, O., Jones, R.: Active learning with feedback on both features and instances. JMLR 7, 1655-1686 (2006)
24. Raz, O., Koopman, P., Shaw, M.: Semantic anomaly detection in online data sources. In: Proc. ICSE, pp. 302-312. IEEE, Los Alamitos (2002)
25. Rothermel, G., Burnett, M., Li, L., Dupuis, C., Sheretov, A.: A methodology for testing spreadsheets. ACM Trans. Software Engineering and Methodology 10(1) (January 2001)
26. Rowan, J., Mynatt, E.: Digital family portrait field trial: Support for aging in place. In: Proc. CHI, pp. 521-530. ACM, New York (2005)
27. Scaffidi, C.: Unsupervised inference of data formats in human-readable notation. In: Proc. Int. Conf. Enterprise Integration Systems, pp. 236-241 (2007)
28. Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin - Madison (2009)
29. Shen, J., Dietterich, T.: Active EM to reduce noise in activity recognition. In: Proc. IUI, pp. 132-140. ACM, New York (2007)
30. Talbot, J., Lee, B., Kapoor, A., Tan, D.: EnsembleMatrix: Interactive visualization to support machine learning with multiple classifiers. In: Proc. CHI, pp. 1283-1292. ACM, New York (2009)
31. Tullio, J., Dey, A., Chalecki, J., Fogarty, J.: How it works: A field study of non-technical users interacting with an intelligent system. In: Proc. CHI, pp. 31-40. ACM, New York (2007)
32. Wong, W.-K., Oberst, I., Das, S., Moore, T., Stumpf, S., McIntosh, K., Burnett, M.: Enduser feature labeling: A locally-weighted regression approach. In: Proc IUI. ACM, New York (2011)