Six strategies for generalizing software engineering theories

Science of Computer Programming

Volumn 101, Issue , 2015, Pages 136-152

  Wieringa, Roel ^a   Daneva, Maya ^a  

^a UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Architectural mechanisms; External validity; Generalization; Scaling up; Statistical inference

Indexed keywords

LABORATORIES; SOFTWARE ENGINEERING;

EMPIRICAL SOFTWARE ENGINEERING; ENGINEERING SCIENCE; EXTERNAL VALIDITIES; GENERALIZATION; LABORATORY CONDITIONS; SCALING-UP; SOFTWARE ENGINEERING THEORIES; STATISTICAL INFERENCE;

ENGINEERING RESEARCH;

EID: 84925075873     PISSN: 01676423     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.scico.2014.11.013     Document Type: Conference Paper

References (107)

1. N. Cartwright The Dappled World. A Study of the Boundaries of Science 1999 Cambridge University Press
2. C. Craver Structure of scientific theories P. Machamer, M. Silberstein, The Blackwell Guide to the Philosophy of Science 2002 Blackwell 55 79
3. R. Nola, and H. Sankey Theories of Scientific Method 2007 Acumen
4. R. Merton The normative structure of science Social Theory and Social Structure enlarged edition 1968 The Free Press 267 278
5. A. Cournand, and M. Meyer The scientist's code Minerva 14 1 1976 79 96
6. S. Gordon History and Philosophy of Social Science 1992 Routledge
7. C. Hempel Aspects of Scientific Explanation and Other Essays in the Philosophy of Science 1965 Free Press
8. E. Nagel The Structure of Science 1961 Routledge and Kegan Paul
9. R. Giere Theories W. Newton-Smith, A Companion to the Philosophy of Science 2000 Blackwell 515 524
10. P. Machamer A brief historical introduction to the philosophy of science P. Machamer, M. Silberstein, The Blackwell Guide to the Philosophy of Science 2002 Blackwell 1 17
11. P. Godfrey-Smith Theory and Reality. An Introduction to the Philosophy of Science 2003 The University of Chicago Press
12. Z. Li, N. Madhavji, S. Murtaza, M. Gittens, A. Miranskyy, D. Godwin, and E. Cialini Characteristics of multiple-component defects and architectural hotspots: a large system case study Empir. Softw. Eng. 16 2011 667 702
13. P. Machamer, L. Darden, and C. Craver Thinking about mechanisms Philos. Sci. 67 2000 1 25
14. W. Bechtel, and A. Abrahamsen Explanation: a mechanistic alternative Stud. Hist. Philos. Biol. Biomed. Sci. 36 2005 421 441
15. S. Glennan Mechanisms and the nature of causation Erkenntnis 44 1996 49 71
16. M. Bunge How does it work? The search for explanatory mechanisms Philos. Soc. Sci. 34 2 2004 182 210
17. P. Hedström, R. Swedberg (Eds.), Social Mechanisms. An Analytical Approach to Social Theory, Cambridge University Press, 1998.
18. R. Merton, and A. Kitt Contributions to the theory of reference group behavior R. Merton, P. Lazarsfeld, Continuities in Social Research: Studies in the Scope and Method of "The American Soldier" 1950 Free Press 40 105
19. P. Thagard Explaining disease: correlations, causes, and mechanisms F. Keil, R. Wilson, Explanation and Cognition 2000 MIT Press 255 276
20. J. Hannay, D. Sjøberg, and T. Dybå A systematic review of theory use in software engineering experiments IEEE Trans. Softw. Eng. 30 2 2007 87 107
21. A.v. Mayrhauser, A. Vans, and A. Howe Program understanding behavior during enhancement of large-scale software J. Softw. Maint. Evol. 9 1997 299 327
22. R. Laymon Experimentation and the legitimacy of idealization Philos. Stud. 77 1995 353 375
23. R. Wieringa Design Science Methodology for Information Systems and Software Engineering 2014 Springer
24. E. McMullin A case for scientific realism J. Leplin, Scientific Realism 1984 University of California Press 8 40
25. E. McMullin Galilean idealization Stud. Hist. Philos. Sci. 16 3 1985 247 273
26. N. Cartwright How the Laws of Physics Lie 1983 Oxford University Press
27. R. Laymon Applying idealized scientific theories to engineering Synthese 81 1989 353 371
28. M. Boon How science is applied in technology Int. Stud. Philos. Sci. 20 1 2006 27 47
29. R. Merton On sociological theories of the middle range Social Theory and Social Structure enlarged edition 1968 The Free Press 39 72
30. A. Davis 201 Principles of Software Development 1995 McGraw-Hill
31. A. Endres, and D. Rombach A Handbook of Software and Systems Engineering: Empirical Observations, Laws and Theories 2003 Pearson Addison Wesley
32. P. Van Strien Towards a methodology of psychological practice: the regulative cycle Theory Psychol. 7 5 1997 683 700
33. T. Huynh, and J. Miller An empirical investigation into open source web applications' implementation vulnerabilities Empir. Softw. Eng. 15 5 2010 556 576
34. R. Wieringa, M. Daneva, and N. Condori-Fernandez The structure of design theories, and an analysis of their use in software engineering experiments International Symposium on Empirical Software Engineering and Measurement (ESEM) 2011 IEEE Computer Society 295 304
35. M. Parascandola, and D. Weed Causation in epidemiology J. Epidemiol. Community Health 55 2001 905 912
36. J. Goldthorpe Causation, statistics, and sociology Eur. Sociol. Rev. 17 1 2001 1 20
37. M.M. Marini, and B. Singer Causality in the social sciences Sociol. Method. 18 1988 347 409
38. J. Woodward Making Things Happen. A Theory of Causal Explanation 2003 Oxford University Press
39. J. Woodward Causation and manipulability E.N. Zalta, The Stanford Encyclopedia of Philosophy summer 2013 edition 2013
40. P. Hedström, and R. Swedberg Social mechanisms: an introductory essay P. Hedström, R. Swedberg, Social Mechanisms. An Analytical Approach to Social Theory 1998 Cambridge University Press 1 31
41. S. Glennan Rethinking mechanistic explanation Philos. Sci. 69 2002 S342 S353
42. P. McKay Illari, and J. Williamson What is a mechanism? Thinking about mechanisms across the sciences Eur. J. Philos. Sci. 2 2012 119 135
43. D. Damian, and J. Chisan An empirical study of the complex relationships between requirements engineering processes and other processes that lead to payoffs in productivity, quality and risk management IEEE Trans. Softw. Eng. 32 7 2006 433 453
44. W. Bechtel, and R. Richardson Discovering Complexity: Decomposition and Localization as Strategies in Scientific Research 2010 MIT Press reissue of the 1993 edition with a new introduction
45. A. Kaplan The Conduct of Inquiry. Methodology for Behavioral Science 1998 Transaction Publishers first edition 1964 by Chandler Publishers
46. D. Sjøberg, T. Dybå, B. Anda, and J. Hannay Building theories in software engineering F. Shull, J. Singer, D. Sjøberg, Guide to Advanced Empirical Software Engineering 2008 Springer 312 336
47. M. Boon, and T. Knuuttila Models as epistemic tools in engineering sciences A. Meijers, Philosophy of Technology and Engineering Sciences 2009 Elsevier 693 726
48. W. Vincenti What Engineers Know and How They Know It Anal. Stud. Aeronaut. Hist. 1990 Johns Hopkins
49. R.J. Wieringa Relevance and problem choice in design science Global Perspectives on Design Science Research (DESRIST), 5th International Conference St. Gallen Lecture Notes in Computer Science vol. 6105 2010 Springer Verlag London 61 76
50. W. Shadish, T. Cook, and D. Campbell Experimental and Quasi-Experimental Designs for Generalized Causal Inference 2002 Houghton Mifflin Company
51. N. Cartwright Nature's Capacities and Their Measurement 1989 Clarendon Press
52. J. Hannay, and M. Jørgensen The role of deliberate artificial design elements in software engineering experiments IEEE Tran. Softw. Eng. 34 2 2008
53. E. Gellenbeck, and C. Cook An investigation of procedure and variable names as beacons during program comprehension J. Koenemann-Belliveau, T. Moher, A. Robertson, Proceedings of the Fourth Annual Workshop on Empirical Studies of Programmers 1991 Ablex 1991 65 81
54. L. Prechelt, B. Unger-Lamprecht, M. Philippsen, and W. Tichy Two controlled experiments assessing the usefulness of design pattern documentation in program maintenance IEEE Trans. Softw. Eng. 28 6 2002 595 606
55. E. Constant The Origins of the Turbojet Revolution 1980 Johns Hopkins
56. R. Wieringa Empirical research methods for technology validation: scaling up to practice J. Syst. Softw. 95 2014 19 31 http://dx.doi.org/10.1016/j.jss.2013.11.1097
57. M. Zelkowitz, and D. Wallace Experimental validation in software engineering Inf. Softw. Technol. 39 1997 735 743
58. R. Glass, I. Vessey, and V. Ramesh Research in software engineering: an empirical study Tech. Rep. TR105-1 September 2001 Information Systems Department, Indiana University
59. S. Green, A. Kouchakdjian, and V. Basili Evaluation of the Cleanroom methodology in the software engineering laboratory Proc. 14th Annual Software Engineering Workshop 1989 NASA, Goddard Space Flight Center Greenbelt, MD 20771
60. N. Fenton, S. Pfleeger, and R. Glass Science and substance: a challenge to software engineers IEEE Softw. 11 4 1994 86 95
61. M. Lubars, G. Meredith, C. Potts, and C. Richter Object-oriented analysis for evolving systems Proceedings of the 14th International Conference on Software Engineering ICSE '92 1992 ACM 173 185
62. M. Lubars, C. Potts, and C. Richter Developing initial OOA models Proceedings of the 15th International Conference on Software Engineering 1993 ACM 255 264
63. C. Potts Software engineering research revisited IEEE Softw. 10 1993 19 28
64. N. Salleh, E. Mendes, J. Grundy, and G. Burch The effects of neuroticism on pair programming: an empirical study in the higher education context Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement ESEM '10 2010 ACM 22:1 22:10
65. J. Hannay, A. Arisholm, H. Engvik, and D. Sjøberg Effects of personality on pair programming IEEE Trans. Softw. Eng. 36 1 2010 61 80
66. G. Gigerenzer, Z. Swijtink, T. Porter, L. Daston, J. Beatty, and L. Krüger The Empire of Chance: How Probability Changed Science and Everyday Life 1989 Cambridge University Press
67. I. Hacking The Taming of Chance 1990 Cambridge University Press
68. T. Porter The Rise of Statistical Thinking, 1820-1900 1986 Princeton University Press
69. H. Inman, Karl Pearson, and R.A. Fisher on statistical tests: a 1935 exchange from Nature Am. Stat. 48 1 1994 2 11
70. R. Fisher Statistical Methods and Scientific Inference 1956 Oliver & Boyd
71. D. Freedman, R. Pisani, and R. Purves Statistics fourth edition 2007 Norton & Company
72. L. Wasserman All of Statistics. A Concise Course in Statistical Inference 2004 Springer
73. I. Hacking An Introduction to Probability and Inductive Logic 2001 Cambridge University Press
74. P. Holland Statistics and causal inference J. Am. Stat. Assoc. 81 396 1986 945 960
75. P. Sedgwick Random sampling versus random allocation Br. Med. J. 343 2011 d7453 10.1136/bmj.d7453
76. A. Nehlig, J.-L. Daval, and G. Debry Caffeine and the central nervous system: mechanisms of action, biochemical, metabolic and psychostimulant effects Brains Res. Rev. 17 1992 139 170
77. B. Kitchenham, S. Pfleeger, D. Hoaglin, K. Emam, and J. Rosenberg Preliminary guidelines for empirical research in software engineering IEEE Trans. Softw. Eng. 28 8 2002 721 733
78. B. Anda, D. Sjøberg, and A. Mockus Variability and reproducibility in software engineering: a study of four companies that developed the same system IEEE Trans. Softw. Eng. 35 1 2009 407 429
79. D. Sjøberg, J. Hannay, O. Hansen, V. Kampenes, A. Karahasanović, N.-K. Liborg, and A. Rekdal A survey of controlled experiments in software engineering IEEE Trans. Softw. Eng. 31 9 2005 733 753
80. D. Campbell, and J. Stanley Experimental and Quasi-Experimental Designs for Research 1963 Rand McNally
81. R. Berk Causal inference for sociological data N. Smelser, Handbook of Sociology 1988 Sage 155 172
82. S. West, J. Biesanz, and S. Pitts Causal inference and generalization in field settings: experimental and quasi-experimental designs H. Reis, C. Judd, Handbook of Research Methods in Social and Personality Psychology 2000 Cambridge University Press 40 84
83. M. Jørgensen, and D. Sjøberg Generalization and theory-building in software engineering research S. Linkman, 8th International Conference on Empirical Assessment in Software Engineering EASE 2004, Workshop 26th International Conference on Software Engineering IEE Proc. vol. 1 2004 29 35
84. O. Laitenberger, K. El Emam, and T. Harbich An internally replicated quasi-experimental comparison of checklist and perspective-based reading of code documents IEEE Trans. Softw. Eng. 27 5 2001 387 421
85. K. Dejaeger, W. Verbeke, D. Martens, and B. Baesens Data mining techniques for software effort estimation: a comparative study IEEE Trans. Softw. Eng. 38 2 2012 375 397
86. E. Kocaguneli, T. Menzies, and J. Keung On the value of ensemble effort estimation IEEE Trans. Softw. Eng. 38 2012 1403 1416
87. B. Boehm, C. Abts, A. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece Software Cost Estimation with COCOMO II 2000 Prentice-Hall
88. T. Menzies, B. Caglayan, E. Kocaguneli, J. Krall, F. Peters, and B. Turhan The PROMISE repository of empirical software engineering data http://promisedata.googlecode.com 2012 West Virginia University, Department of Computer Science
89. B. Kitchenham, E. Mendes, and G. Travassos Cross versus within-company cost estimation studies: a systematic review IEEE Trans. Softw. Eng. 33 5 2007 316 329
90. E. Mendes, and B. Kitchenham Further comparison of cross-company and within-company effort estimation models for web applications Proceedings, 10th International Symposium on Software Metrics 2004 348 357
91. R. Sabherwal The evolution of coordination in outsourced software development projects: a comparison of client and vendor perspectives Inf. Organ. 13 2003 153 202
92. F. Znaniecki The Method of Sociology 1968 Octagon Books first edition 1934
93. W. Robinson The logical structure of analytic induction Am. Sociol. Rev. 16 6 1951 812 818
94. J. Tacq Znaniecki's analytical induction as a method of sociological research Pol. Sociol. Rev. 158 2 2007 187 208
95. R. Yin Case Study Research: Design and Methods third edition 2003 Sage Publications
96. A. Mockus, R. Fielding, and J. Herbsleb A case study of open source software development: the Apache server Proceedings of the 22nd International Conference on Software Engineering ICSE 2000 2000 ACM Press 263 272
97. A. Mockus, R. Fielding, and J. Herbsleb Two case studies of open source software development: Apache and Mozilla ACM Trans. Softw. Eng. Methodol. 11 3 2002 309 346
98. S. Gregor The nature of theory in information systems Manag. Inf. Syst. Q. 30 3 2006 611 642
99. S. Gregor, and D. Jones The anatomy of a design theory J. Assoc. Inf. Syst. 8 5 2007 312 335
100. P. Seddon, and R. Scheepers Towards the improved treatment of generalization from knowledge claims in IS research: drawing general conclusions from samples Eur. J. Inf. Syst. 2011 1 16 10.1057/ejis.2011.9
101. A. Lee, and R. Baskerville Generalizing generalizability in information systems research Inf. Syst. Res. 14 3 2003 221 243
102. S. Ghaisas, P. Rose, M. Daneva, K. Sikkel, and R. Wieringa Generalizing by similarity: lessons learnt from industrial case studies X. Franch, N. Madhavji, W. Curtis, L. Votta, 1st International Workshop on Conducting Empirical Studies in Industry CESI 2013 2013 IEEE Computer Science Press 37 42
103. M. Daneva, E. van der Veen, C. Amrit, S. Ghaisas, K. Sikkel, R. Kumar, N. Ajmeri, U. Ramteerthkar, and R.J. Wieringa Agile requirements prioritization in large-scale outsourced system projects: an empirical study J. Syst. Softw. 86 5 2013 1333 1353
104. M. Höst, B. Regnell, and C. Wohlin Using students as subjects - a comparative study of students and professionals in lead-time impact assessment Empir. Softw. Eng. 5 3 2000 201 214
105. P. Runeson Using students as experiment subjects - an analysis on graduate and freshmen student data Proceedings of the Seventh International Conference Empirical Assessment and Evaluation in Software Engineering EASE '03 2003 95 102
106. M. Svahnberg, A. Aurum, and C. Wohlin Using students as subjects - an empirical evaluation Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement ESEM '08 2008 ACM 288 290
107. J. Krein, C. Knutson, and C. Bird Report from the 3rd international workshop on replication in empirical software engineering research (RESER 2013) SIGSOFT Softw. Eng. Notes 39 1 2014 31 35