Benefits and barriers of user evaluation in software engineering research

Proceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA

Volumn , Issue , 2011, Pages 643-656

  Buse, Raymond P L ^a   Sadowski, Caitlin ^b   Weimer, Westley ^a  

^a UNIVERSITY OF VIRGINIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Human study; User evaluation

Indexed keywords

HUMAN STUDY; USER EVALUATIONS;

COMPUTER SYSTEMS PROGRAMMING; ENGINEERING RESEARCH; OBJECT ORIENTED PROGRAMMING; SURVEYS; TECHNOLOGY TRANSFER;

SOFTWARE ENGINEERING;

EID: 81455159341     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2048066.2048117     Document Type: Conference Paper

References (28)

1. J. W. Backus. The IBM 701 Speedcoding system. Journal of the ACM, 1:4-6, January 1954.
2. V. Basili, R. Selby Jr, and D. Hutchens. Experimentation in software engineering. In P. Oman and S. L. Pfleeger, editors, Applying Software Metrics. 1997.
3. I. Campbell. Chi-squared and Fisher-Irwin tests of two-bytwo tables with small sample recommendations. Statistics in Medicine, 26:3661-3675, 2007. (Pubitemid 47180703)
4. T. Couronne, M. Zelkowitz, and D. Wallace. Experimental validation in software engineering. Information and Software Technology, 39(11):735-743, 1997. (Pubitemid 127371611)
5. I. Deligiannis, M. Shepperd, S. Webster, and M. Roumeliotis. A review of experimental investigations into object-oriented technology. Empirical Software Engineering, 7(3):193-231, 2002. (Pubitemid 34959711)
6. T. Dybå and T. Dingsøyr. Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9-10):833-859, 2008.
7. A. Egyed. Instant consistency checking for the UML. In International Conference on Software Engineering, 2006.
8. R. Glass, I. Vessey, and V. Ramesh. Research in software engineering: an analysis of the literature. Information and Software Technology, 44(8):491-506, 2002. (Pubitemid 34704253)
9. S. Hanenberg. Faith, hope, and love: an essay on software science's neglect of human factors. In Object-Oriented Programming, Systems, Languages, and Applications, 2010.
10. M. Host, C.Wohlin, and T. Thelin. Experimental context classification: incentives and experience of subjects. In International Conference on Software Engineering, 2005.
11. IDEO. Method Cards: 51 Ways to Inspire Design. William Stout, 2003.
12. M. Jørgensen. A review of studies on expert estimation of software development effort. Journal of Systems and Software, 70(1-2):37-60, 2004.
13. N. Juristo, A. Moreno, and S. Vegas. Reviewing 25 years of testing technique experiments. Empirical Software Engineering, 9(1):7-44, 2004.
14. B. Kitchenham. Evaluating software engineering methods and tools. Part 1: the evaluation context and evaluation methods. ACM SIGSOFT Software Engineering Notes, 21(1):11-14, 1996.
15. H. Lam, E. Bertini, P. Isenberg, C. Plaisant, and S. Carpendale. Seven guiding scenarios for information visualization evaluation. Technical Report 2011-992-04, University of Calgary, 2011.
16. J. Lazar, J. H. Feng, and H. Hochheiser. Research Methods in Human Computer Interaction. Wiley, 2010.
17. J. Lung, J. Aranda, S. M. Easterbrook, and G. V. Wilson. On the difficulty of replicating human subjects studies in software engineering. In International Conference on Software Engineering, 2008.
18. J. McGrath. Methodology matters: Doing research in the behavioral and social sciences. In Human-computer interaction, pages 152-169. Morgan Kaufmann Publishers Inc., 1995.
19. J. R. Quinlan. C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., 1993.
20. M. Robnik-Šikonja and I. Kononenko. Theoretical and empirical analysis of ReliefF and RReliefF. Machine Learning, 53:23-69, October 2003.
21. M. Shaw. Writing good software engineering research papers: minitutorial. In International Conference on Software Engineering, 2003.
22. D. Sjoberg, B. Anda, E. Arisholm, T. Dyba, M. Jorgensen, A. Karahasanovic, E. Koren, and M. Vokác. Conducting realistic experiments in software engineering. In Empirical Software Engineering and Measurement, 2003.
23. D. Sjoberg, T. Dyba, and M. Jorgensen. The future of empirical methods in software engineering research. In Future of Software Engineering Workshop (FoSER), 2007.
24. D. Sjøeberg, J. Hannay, O. Hansen, V. Kampenes, A. Karahasanovic, N.-K. Liborg, and A. Rekdal. A survey of controlled experiments in software engineering. IEEE Transactions on Software Engineering, 31(9):733 - 753, 2005. (Pubitemid 41555415)
25. M. Svahnberg, A. Aurum, and C. Wohlin. Using students as subjects-an empirical evaluation. In Empirical Software Engineering and Measurement, 2008.
26. L. Teo and B. John. Cogtool-Explorer: towards a tool for predicting user interaction. In Conference on Human factors In computing systems (CHI), 2008.
27. W. Tichy, P. Lukowicz, L. Prechelt, and E. Heinz. Experimental evaluation in computer science: A quantitative study. Journal of Systems and Software, 28(1):9-18, 1995.
28. M. Zelkowitz. Techniques for Empirical validation. In V. Basili, D. Rombach, K. Schneider, B. Kitchenham, D. Pfahl, and R. Selby, editors, Empirical Software Engineering Issues. Critical Assessment and Future Directions, pages 4-9. Springer, 2007. (Pubitemid 350143426)