Towards a decision-making structure for selecting a research design in empirical software engineering

Empirical Software Engineering

Volumn 20, Issue 6, 2015, Pages 1427-1455

  Wohlin, Claes ^a   Aurum, Aybüke ^b  

^a BLEKINGE INSTITUTE OF TECHNOLOGY   (Sweden)

^b NONE   (Australia)

Author keywords

Empirical software engineering research; Research design; Research methods; Selecting research method

Indexed keywords

DECISION MAKING; DESIGN; SOCIAL SCIENCES COMPUTING; SOFTWARE ENGINEERING;

DECISION POINTS; DECISION-MAKING STRUCTURE; EMPIRICAL RESEARCH; EMPIRICAL RESEARCH IN SOFTWARE ENGINEERING; EMPIRICAL SOFTWARE ENGINEERING; INFORMED DECISION; RESEARCH DESIGNS; RESEARCH METHODS;

ENGINEERING RESEARCH;

EID: 84944353479     PISSN: 13823256     EISSN: 15737616     Source Type: Journal    
DOI: 10.1007/s10664-014-9319-7     Document Type: Article

References (96)

1. Adolp A, Hall W, Kruchten P (2011) Using grounded theory to study the experience of software development. J Empir Softw Eng 16(4):487–513
2. Allison I, Merali Y (2007) Software process improvement as emergent change: a structurational analysis. Inf Softw Technol 49(6):668–681
3. Barney S, Mohankumar V, Chatzipetrou P, Aurum A, Wohlin C, Angelis L (2014) Software quality across borders: three case studies on company internal alignment. Inf Softw Technol 56(1):20–38
4. Basili V (1993) The experimental paradigm in software engineering. Proceedings of the International Workshop on Experimental Software Engineering Issues: Critical Assessment and Future Directions. Springer-Verlag, LNCS 706, London, UK, pp 3–12 Link: http://link.springer.com/content/pdf/10.1007/3-540-57092-6_91.pdf
5. Baskerville R (2008) What design science is not. Eur J Inf Syst 17:441–443
6. Benbasat I, Goldstein DK, Mead M (1987) The case research strategy in studies of information systems. MIS Q 11(3):369–386
7. Bertelsen OW (1997) Towards a unified field of se research and practice. IEEE Softw 14:87–88
8. Bhattacherjee A (2012) Social science research: principles, methods, and practices. USF Open Access Textbooks Collection. Book 3 University of South Florida Link http://scholarcommons.usf.edu/oa_textbooks/3
9. Birks DF, Fernandez W, Levina N, Nasirin S (2013) Grounded theory method in information systems research: its nature, diversity and opportunities. Guest editorial. Eur J Inf Syst 22:1–8
10. Boell S, Cecez-Kecmanivic D (2010) Literature reviews and the hermeneutic circle. Aust Acad Res Libr 41(2):129–144
11. Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101
12. Brooke C (2002) What does it mean to be ‘critical’ in IS research? J Inf Technol 17(2):49–57
13. Butler T (1998) Towards a hermeneutic method for interpretive research in information systems. J Inf Technol 13:285–300
14. Carver J, Seaman C, Jeffery R (2004) Using qualitative methods in software engineering. International Advanced School of Empirical Software Engineering (IASESE04), August 18, 2004, LA CA Link: http://chess.cs.umd.edu/class/fall2004/cmsc735/CMSC735%2011a%20Qualitative%20Analysis.pdf
15. Cecez-Kecmanovic D (2011) Doing critical information systems research–arguments for a critical research methodology. Eur J Inf Syst 20(4):440–455
16. Checkland P (1981) Systems thinking, systems practice. Wiley, UK
17. Chen WS, Hirschheim R (2004) A paradigmatic and methodological examination of information systems research from 1991 to 2001. Inf Syst J 14(3):197–235
18. Coleman G, O’Connor R (2007a) Investigating software process in practice: a grounded theory perspective. J Syst Softw 81:772–784
19. Coleman G, O’Connor R (2007b) Using grounded theory to understand software process improvement: a study of Irish software product companies. Inf Softw Technol 49(6):654–667
20. Collis J, Hussey R (2009) Business research. Palgrave MacMillan, UK
21. Creswell J (2013) Research design: qualitative, quantitative and mixed methods approach. Sage Publication, Thousand Oaks
22. Davison RM, Martinsons MG, Kock N (2004) Principles of canonical action research. Inf Syst J 14(1):65–86
23. Dunne C (2011) The place of the literature review in grounded theory research. Int J Soc Res Methodol 14(2):111–124
24. Dybå T, Prikladnicki R, Rönkkö K, Seaman CB, Sillito J (2011) Qualitative research in software engineering. Empir Softw Eng 16(4):425–429
25. Easterbrook S, Singer J, Storey MA, Damian D (2008) Selecting empirical methods for software engineering research. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering, Springer Germany, pp 285–311
26. Eisenhardt KM (1989) Building theories from case study research. Academy of management. Acad Manag Rev 14(4):532–550
27. Engel RJ, Schutt RK (2010) The practice of research in social work. Sage Publication, Thousand Oaks
28. Ghanam Y, Maurer F, Abrahamsson P (2012) Making a leap to a software platform strategy: issues and challenges. Inf Softw Technol 54(9):968–984
29. Ghapanchi AH (2011) Dynamic capabilities and project characteristics contributing to the success of open source software projects. PhD Dissertation Thesis. The University of New South Wales, Sydney, NSW Australia
30. Gilbert GN (1995) Dagstuhl seminar on social science microsimulation: a challenge to computer science, SchjoB, May 1–5, 1995
31. Glaser BG (1992) Emergence vs. forcing: basics of grounded theory analysts. Sociology Press, California
32. Glaser BG, Strauss AL (1967) The discovery of grounded theory: strategies for qualitative research. Aldine, New York
33. Gregg DG, Kulkarni UR, Vinze AS (2001) Understanding the philosophical underpinnings of software engineering research in information systems. Inf Syst Front 3(2):169–183
34. Grix J (2002) Introducing students to the generic terminology of social research. Politics 22(3):175–186
35. Hannah DR, Lautsch BA (2011) Counting in qualitative research: why to conduct it, when to avoid it, and when to closet it. J Manag Inq 20(1):14–22
36. Hansen S, Rennecker J (2010) Getting on the same page: collective hermeneutics in a systems development team. Inf Organ 20(1):44–63
37. Harrison RJ, Lin Z, Caroll GR, Carley KM (2007) Simulation modeling in organization and management research. Acad Manag Rev 32(4):1229–1245
38. Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q 28(1):75–105
39. Jafarzadeh H, Aurum A, D’Ambra J (2011) Factors affecting the success of businesses in effective utilization of search engine advertising. International Conference on Information Systems, 3–7 December, Shanghai, China, 2011, sigIQ pre-ICIS workshop, pp 1 Link: http://www.northeastern.edu/iqworld/papers/Papers/Factors%20Affecting%20the%20Success%20of%20Businesses%20in%20Effective%20Utilization%20of%20Search%20Engine%20Advertising_abstract.pdf
40. Jafarzadeh H, Aurum A, D’ambra J, Abedin B (2013) Determinant of intention to use search engine advertising: a conceptual model. Int J Enterp Inf Syst 9(3):22–38
41. Johnson CF (1996) Deductive versus inductive reasoning: a closer look at economics. Soc Sci J 33(3):287–299
42. Johnson RB, Onwuegbuzie AJ (2004) Mixed methods research: a research paradigm whose time has come. Educ Res 33(7):14–26
43. Kachigan SK (1986) Statistical analysis: an interdisciplinary introduction to univariate & multivariate methods. Radius Press, New York
44. Kitchenham BA, Pfleeger SA (2002) Principles of survey research part 2: designing a survey. ACM SIGSOFT Softw Eng Notes 27(1):18–20
45. Kitchenham BA, Pickard LM, Pfleeger SL (1995) Case studies for method and tool evaluation. IEEE Softw 12(4):52–62
46. Kitchenham BA, Pfleeger SL, Pickard LM, Jones PW, Hoaglin DC, El Emam K, Rosenberg J (2002) Preliminary guidelines for empirical research in software engineering. IEEE Trans Softw Eng 28(8):721–734
47. Klein HK, Myers MD (1999) A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Q 23(1):67–89
48. Kline RB (2011) Principles and practice of structural equation modeling. Guilford Press, New York
49. Kontio J, Lehtola L, Bragge J (2004) Using the focus group method in software engineering: obtaining practitioner and user experience. Proceedings of International Symposium of Empirical Software Engineering, Los Angeles, CA, USA, August 2004, IEEE Computer Society Washington, DC, USA, pp 271–280
50. Kyburz-Graber R (2007) Does case–study methodology lack rigour? The need for quality criteria for sound case–study research, as illustrated by a recent case in secondary and higher education. Environ Educ Res 10(1):53–65
51. Law AM (2007) Simulation modeling and analysis, volume 4. McGraw-Hill, New York
52. Lee AS (1989) A scientific methodology for MIS case studies. MIS Q 13(1):33–54
53. Lethbridge TC, Sim SE, Singer J (2005) Studying software engineers: data collection techniques for software field studies. J Empir Softw Eng 10(3):311–341
54. Lyons H (2009) Case Study research methodology for publishing developments in ICT-facilitated learning in higher education—a perspective approach. Innov Educ Teach Int 46(1):27–39
55. Mack N, Woodsong C, MacQueen K, Guest G, Namey E (2005) Qualitative research methods: a data collector’s field guide. Family Health International, Research Triangle Park
56. Marascuilo LA, Serlin RC (1988) Statistical methods for the social and behavioral sciences. W.H. Freeman and Company, New York
57. March ST, Smith GF (1995) Design and natural science research on information technology. Decis Support Syst 15(4):251–266
58. McKernan J (1996) Curriculum action research: a handbook of methods and resources for the reflective practitioner. Kogan Page, London
59. McLeod L, MacDonell SG, Doolin B (2011) Qualitative research on software development: a longitudinal case study methodology. J Empir Softw Eng 16(4):430–459
60. Mingers J (2001) Combining IS research methods: towards a pluralist methodology. Inf Syst Res 12(3):240–259
61. Mkansi M, Acheampong EA (2012) Research philosophy debates and classification: students’ dilemma. Electron J Bus Res Methods 10(2):132–140
62. Moe NB, Aurum A, Dybå T (2012) Challenges of shared decision-making: a multiple case study of agile software development. Inf Softw Technol 54(8):853–865
63. Müller M, Pfahl D (2008) Simulation methods. In: Shull F, Singer J, Sjøberg DIK (eds) Guide to advanced empirical software engineering. Springer, Germany
64. Myers MD (1995) Dialectical hermeneutics: a theoretical framework for the implementation of information systems. Inf Syst J 5(1):51–70
65. Myers MD (1997) Qualitative research in information systems. MIS Q 21(2):241–242
66. Myers MD, Klein HK (2011) A set of principles for conducting critical research in information systems. MIS Q 35(1):17–36
67. Nunamaker JF Jr, Chen M, Purdin TDM (1991) Systems development in information systems research. J Manag Inf Syst 7(3):89–106
68. Orlikowski WJ, Baroudi JJ (1991) Studying information technology in organizations: research approaches and assumptions. Inf Syst Res 2(1):1–28
69. Ostrowski L, Helfert M (2011) Commonality in various design science methodologies. Proceedings of the Federated Conference on Computer Science and Information Systems, pp 317–320. ISBN 978-83-60810-39-2
70. Perry C (1998) Processes of a case study methodology for postgraduate research in marketing. Eur J Mark 32(9/10):785–802
71. Perry DE, Porter AA, Votta LG (2000) Empirical studies of software engineering: a roadmap. In: Finkelstein A (ed) The future of software engineering. ACM Press, New York
72. Pinsonneault A, Kraemer K (1993) Survey research methodology in management information systems: an assessment. J Manag Inf Syst 10(2):75–105
73. Rossi P, Freeman HF (1993) Evaluation: a systematic approach. Sage Publication, USA
74. Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. J Empir Softw Eng 14(2):131–164
75. Runeson P, Höst M, Rainer A, Regnell B (2012) Case study research in software engineering: guidelines and examples. Wiley, USA
76. Seaman CB (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557–573
77. Shanks G, Parr A (2003) Positivist, single case study research in information systems: a critical analysis. Proceedings of the 11th European Conference on Information Systems, Naples, Italy 16-21 June 2003, pp 1760–1774 Link: http://is2.lse.ac.uk/asp/aspecis/20030140.pdf
78. Shaw M (2002) What makes good research in software engineering? Int J Softw Tools Technol Transfer 4(1):1–7
79. Shaw M (2003) Writing good software engineering research papers. Proceedings of 25th Int Conference on Software Engineering, Portland, Oregon, USA, May 2003, IEEE Computer Society Washington, DC, USA, pp 726–736
80. Shull F, Singer J, Sjøberg DIK (2008) Guide to advanced empirical software engineering. Springer, Germany
81. Shye S (1988) Inductive and deductive reasoning: a structural reanalysis of ability tests. J Appl Psychol 73(2):308–311
82. Siegel S, Castellan NJ Jr (1988) Nonparametric statistics for the behavioral sciences. Mcgraw-Hill Book Company, New York
83. Sjøberg DIK, Dybå T, Jørgensen M (2007) The future of empirical methods in software engineering. IEEE Proceedings of Future of Software Engineering (FOSE)
84. Smite D, Wohlin C, Gorschek T, Feldt R (2010) Empirical evidence in global software engineering: a systematic review. Empir Softw Eng Int J 15(1):91–118
85. Smite D, Wohlin C, Aurum A, Jabangwe R, Numminen E (2013) Offshore insourcing in software development: structuring the decision-making process. J Syst Softw 86(4):1054–1067
86. Staron M, Kuzniarz L, Wohlin C (2006) Empirical assessment of using stereotypes to improve comprehension of UML models: a set of experiments. J Syst Softw 79(5):727–742
87. Strauss AL, Corbin JM (1998) Basics of qualitative research: techniques and procedures for developing grounded theory. Sage Publication, Thousand Oaks
88. Susman G, Evered R (1978) An assessment of the scientific merits of action research. Adm Sci Q 23(4):582–603
89. Tichy WF, Lukowicz P, Prechelt L, Heinz EA (1995) Experimental evaluation in computer science: a quantitative study. J Syst Softw 28(1):9–18
90. Tom E, Aurum A, Vidgen R (2013) An exploration of technical debt. J Syst Softw 86(6):1498–1516
91. Urquhart C (2002) Regrounding grounded theory–or reinforcing old prejudices? a brief reply to bryant. J Inf Technol Theory Appl 4(3):43–54
92. Walker D, Myrick F (2006) Grounded theory: an exploration of process and procedure. Qual Health Res 16(4):547–559
93. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in software engineering. Springer, ISBN 978-3-642-29043-5
94. Wynekoop JL, Russo NL (1997) Studying system development methodologies: an examination of research methods. Inf Syst J 7(1):47–65
95. Yin RK (2002) Case study research. Sage Publication, CA
96. Zelkowitz MV, Wallace D (1997) Experimental validation in software engineering. Inf Softw Technol 31(5):23–31