Trends in the quality of human-centric software engineering experiments - A quasi-experiment

IEEE Transactions on Software Engineering

Volumn 39, Issue 7, 2013, Pages 1002-1017

  Kitchenham, Barbara ^a   Sjoberg, Dag I K ^a   Dyba, Tore ^b   Brereton, O Pearl ^c   Budgen, David ^d   Host, Martin ^e   Runeson, Per ^e  

^a KEELE UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF OSLO   (Norway)

^c SINTEF   (Norway)

^d DURHAM UNIVERSITY   (United Kingdom)

^e LUND UNIVERSITY   (Sweden)

Author keywords

empirical studies; experimentation; human centric experiments; Quality evaluation; software engineering

Indexed keywords

PAPER PRODUCTS; PAPERMAKING MACHINERY; QUALITY CONTROL; REGRESSION ANALYSIS; SOFTWARE ENGINEERING; SURVEYS; TEXTBOOKS;

EMPIRICAL SOFTWARE ENGINEERING; EMPIRICAL STUDIES; EXPERIMENTAL SOFTWARE ENGINEERING; EXPERIMENTATION; HUMAN-CENTRIC; HUMAN-CENTRIC SOFTWARE ENGINEERINGS; QUALITY EVALUATION; SOFTWARE ENGINEERING JOURNALS;

SOFTWARE QUALITY;

EID: 84880239200     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2012.76     Document Type: Article

References (28)

1. D.T. Campbell and J.C. Stanley, Experimental and Quasi- Experimental Designs for Research. Houghton Mifflin Company, 1966.
2. T.D. Cook and D.T. Campbell, Quasi-Experimentation: Design and Analysis Issues for Field Settings. Rand McNally Collage, 1979.
3. I.K. Crombie, The Pocket Guide to Appraisal. BMJ Books, 1996.
4. O. Dieste and A.G. Padua, "Developing Search Strategies for Detecting Relevant Experiments for Systematic Reviews," Proc. First Int'l Symp. Empirical Software Eng. and Measurement, pp. 215- 224, 2007.
5. O. Dieste, A. Grima'n, N. Juristo, and H. Saxena, "Quantitative Determination of the Relationship between Internal Validity and Bias in Software Engineering: Consequences for Systematic Literature Reviews," Proc. Int'l Symp. Empirical Software Eng. and Metrics, pp. 285-288, 2011.
6. T. Dyba, V.B. Kampenes, and D.I.K. Sjøberg, "A Systematic Review of Statistical Power in Software Engineering Experiments," Information and Software Technology, vol. 48, no. 8, pp. 745- 755, 2006. (Pubitemid 44218930)
7. L.D. Fisher, D.O. Dixon, J. Herson, R.K. Frankowski, M.S. Hearon, and K.E. Pearce, "Intention to Treat in Clinical Trials," Statistical Issues in Drug Research and Development, K.E. Pearce, ed., pp. 331- 350, Marcel Dekker, 1990.
8. A. Fink, Conducting Research Literature Reviews: From the Internet to Paper. Sage Publication, Inc., 2005.
9. T. Greenhalgh, How to Read a Paper: The Basics of Evidence-Based Medicine. BMJ Books, 2000.
10. A. Jedlitschka, M. Ciolkowski, and D. Pfahl, "Reporting Experiments in Software Engineering," Guide to Advanced Empirical Software Eng., F. Shull, J. Singer, and D.I.K. Sjøberg, eds., Springer- Verlag, 2008.
11. J. Juristo and A. Moreno, Basics of Software Engineering Experimentation. Kluwer Academic Publishers, 2001.
12. P. Ju? ni, A. Witschi, R. Bloch, and M. Egger, "The Hazards of Scoring the Quality of Clinical Trials for Meta-Analysis," J. Am. Medical Assoc., vol. 282, no. 11, pp. 1054-1060, 1999. (Pubitemid 29436453)
13. H. Liu and H.B.K. Tan, "Testing Input Validation in Web Applications through Automated Model Recovery," J. Systems and Software, vol. 81, pp. 222-233, 2007.
14. V.B. Kampenes, T. Dyba, J.E. Hannay, and D.I.K. Sjøberg, "A Systematic Review of Effect Size in Software Engineering Experiments," Information and Software Technology, vol. 49, no. 11/12, pp. 1073-1086, 2007. (Pubitemid 47464910)
15. V.B. Kampenes, "Quality of Design Analysis and Reporting of Software Engineering Experiments: A Systematic Review," PhD thesis, Dept. of Informatics, Univ. of Oslo, 2007.
16. V.B. Kampenes, T. Dyba, J.E. Hannay, and D.I.K. Sjøberg, "A Systematic Review of Quasi-Experiments in Software Engineering," Information and Software Technology, vol. 51, no. 1, pp. 71-82, 2009.
17. B. Kitchenham, S.L. Pfleeger, L.M. Pickard, P. Jones, D. Hoaglin, K. El Emam, and J. Rosenberg, "Preliminary Guidelines for Empirical Research in Software Engineering," IEEE Trans. Software Eng., vol. 28, no. 8, pp. 721-734, Aug. 2002.
18. B.A. Kitchenham, D.I.K. Sjøberg, T. Dyba, D. Pfhal, P. Brereton, D. Budgen, M. Ho? st, and P. Runeson, "Three Empirical Studies on the Agreement of Reviewers about the Quality of Software Engineering Experiments," Information and Software Technology, vol. 54, pp. 804-819, 2012.
19. B.A. Kitchenham, D.I.K. Sjøberg, O.P. Brereton, D. Budgen, T. Dyba, M. Høst, D. Pfahl, and P. Runeson, "Can We Evaluate the Quality of Software Engineering Experiments?" Proc. Conf. Empirical Software Eng. and Metrics, 2010.
20. W.F. Rosenberger, "Dealing with Multiplicities in Pharmacoepidermioloical Studies," Pharmacoepidemiology and Drug Safety, vol. 5, pp. 95-100, 1996.
21. R.L. Rosnow and R. Rosenthal, People Studying People. Artifacts and Ethics in Behavioural Research. W.H. Freeman and Company, 1997.
22. J. Singer, "Using the APA Style Guidelines to Report Experimental Results," Proc. Workshop Empirical Studies in Software Maintenance, pp. 71-75, 1999.
23. D.I.K. Sjøberg, J.E. Hannay, O. Hansen, V.B. Kampenes, A. Karahasanovic, N.K. Liborg, and A.C. Rekdal, "A Survey of Controlled Experiments in Software Engineering," IEEE Trans. Software Eng., vol. 31, no. 9, pp.733-753, Sept. 2005. (Pubitemid 41555415)
24. W.R. Shadish, T.D Cook, and D.T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, 2002.
25. P.E. Shrout and J.L. Fleiss, "Intraclass Correlations: Uses in Assessing Rater Reliability," Psychological Bull., vol. 86, no. 2, pp. 420-428, 1979.
26. A.K. Wagner, S.B. Soumerai, F. Zhang, and D. Ross-Degnan, "Segmented Regression Analysis of Interrupted Time Series Studies in Medication Use Research," J. Clinical Pharmacy Therapeutics, vol. 27, pp. 299-309, 2002. (Pubitemid 34989269)
27. C. Wohlin, P. Runeson, M. Ho? st, M.C. Ohlsson, B. Regnell, and A. Wessle'n, Experimentation in Software Engineering - An Introduction. Kluwer, Academic Press, 2000.
28. M.A. Wojcicki and P. Strooper, "Maximising the Information Gained by a Study of Static Analysis Technologies for Current Software," Empirical Software Eng., vol. 12, no. 6, pp. 617-645, 2007. (Pubitemid 350045890)