Addressing the challenges of replications of surveys in software engineering research

2005 International Symposium on Empirical Software Engineering, ISESE 2005

Volumn , Issue , 2005, Pages 204-213

  Cater Steel, Aileen ^a   Toleman, Mark ^a   Rout, Terry ^b  

^a UNIVERSITY OF SOUTHERN QUEENSLAND   (Australia)

^b GRIFFITH UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ENGINEERING RESEARCH; INDUSTRIAL MANAGEMENT; SOCIETIES AND INSTITUTIONS; SURVEYS;

RESEARCH TOOLS; SOFTWARE DEVELOPMENT ORGANISATIONS; SOFTWARE ENGINEERING RESEARCH;

SOFTWARE ENGINEERING;

EID: 33749144025     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISESE.2005.1541829     Document Type: Conference Paper

References (67)

1. B. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K. El Emam, and J. Rosenberg, "Preliminary guidelines for empirical research in software engineering", IEEE Transactions on Software Engineering, vol. 28, no. 8, 2002, pp. 721-734.
2. ABS, "8126.0 Information and Communication Technology, Australia 2002-03", accessed 15 Oct, http://www.abs.gov.au/Ausstats/abs@nsf/ 1020492cfcd63696ca2568a1002477b5/60c99311996ef774ca256f070079c995! openDacument, 2004.
3. R.F. Bornstein, "Publication politics, experimenter bias and the replication process in social science research", in Replication Research in the Social Sciences, J.W. Neuliep, Ed. Sage, Newbury Park, Ca, USA, 1991, pp. 71-81.
4. K.E. Stanovich, How to think straight about psychology, 2nd ed., Scott Foresman & Co., Glenview IL, 1989.
5. A. Brooks, J. Daly, J. Miller, M. Roper, and M. Wood, "Replication of experimental results in software engineering", accessed 16 January, 1996, http://www2.umassd.edu/SWPI/ISERN/ISERN-96-10.pdf.
6. th international conference on software engineering", DACS, accessed 19 January, 1996, http://www.softwaretechnews.com/summer96/icse18.html
7. W.G. Zikmund, Business Research Methods, Dryden Press, Fort Worth, 1994.
8. W.S. Humphrey and W.L. Sweet, "A Method for Assessing the Software Engineering Capability of Contractors", Software Engineering Institute, CMU, Pittsburgh CMU/SEI-87-TR-23, Septenber, 1987.
9. D. Zubrow, W. Hayes, and J. Siegel, "A Method for Assessing the Software Engineering Capability of Contractors", Software Engineering Institute, CMU, Pittsburgh SEI-94-SR-007, 1994.
10. P. Fusaro, K. El Emam, and B. Smith, "The internal consistencies of the 1987 SEI maturity questionnaire and the SPICE capability dimension", Empirical Software Engineering, vol. 3, no. 2, 1997, pp. 179-201.
11. W.S. Humphrey, D.H. Kitson, and J. Gale, "A comparison of U.S. and Japanese software process maturity", presented at 13th International Conference on Software Engineering, Austin, Texas, 1991.
12. M. Tan and C.Y. Yap, "Impact of organisational maturity on software quality", in Software Quality and Productivity: Theory Practice, Education and Training, M. Lee, B. Barta, and P. Juliff, Eds., Chapman and Hall for IFIP, London, 1995, pp. 231-234.
13. A.P. Cater-Steel and E.P. Fitzgerald, "Quality Assurance Certification: Adoption by Australian Software Developers and its Association with Capability Maturity", presented at Asia Pacific Software Engineering Conference (APSEC), Hong Kong, 1997.
14. D.R. Goldenson and J.D. Herbsleb, "After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success", Software Engineering Institute, CMU CMU/SEI-95-TR-009, August, 1995.
15. K. El Emam and A. Birk, "Validating the ISO/EC 15504 measure of software requirements analysis process capability", IEEE Transactions on Software Engineering, vol. 26, no. 6, 2000, pp. 541-566.
16. F. Brooks, "No silver bullet: essence and accidents of software engineering", IEEE Computer, vol. 20, no. 4, 1987, pp. 10-19.
17. S. McConnell, "Closing the gap", IEEE Software, vol. 19, no. 1, 2002, pp. 3-5.
18. J. Ludewig, "Software engineering in the year 2000 minus and plus ten", in Informatics: 10 years back, 10 years ahead., R. Wilhelm, Ed., Springer-Verlag, Heidelberg, Berlin, 2001, pp. 102-111.
19. R.L. Glass, "The state of the practice of software engineering", IEEE Software, vol. 20, no. 6, 2003, pp. 20-21.
20. B. Fitzgerald, "The use of systems development methodologies in practice: a field study", Information Systems Journal, vol. 7, no. 3, pp. 201-212, 1997.
21. T. McBride, "Standards need more rigour", in Information Age, 2004, pp. 65-66.
22. N. Fenton, S.L. Pfleeger, and R.L. Glass, "Science and substance: a challenge to software engineers", IEEE Software, vol. 11, no. 4, 1994, pp. 86-95.
23. I. Richardson and K. Ryan, "Software Process Improvements in a Very Small Company", Software Quality Professional, vol. 3, no. 2, 2001, pp. 23-35.
24. F.G. Wilkie, D. McFall, and F. McCaffery, "The centre for software process technologies: a model for process improvement in geographical regions with small software industries", presented at Software Engineering Process Group, Orlando, Florida, USA, 2004.
25. G. Bucci, M. Campanai, and G. Cignoni, "Rapid assessment to solicit process improvement in small and medium-sized organisations", Software Quality Professional, vol. 4, no. 1, 2001, pp. 33-41.
26. S.L. Pfleeger, D.R. Jeffery, B. Curtis, and B. Kitchenham, "Status report on software management", IEEE Software, vol. 14, no. 2, 1997, pp. 33-43.
27. M.C. Paulk, D.R. Goldenson, and D.M. White, "The 1999 Survey of High Maturity Organizations", Software Engineering Institute, CMU, Pittsburgh CMU/SEI-2000-SR-002, February, 2000.
28. S. Bandinelli, A. Fuggetta, L. Lavazza, M. Loi, and G.P. Picco, "Modeling and improving an industrial software process", IEEE Transactions on Software Engineering, vol. 21, no. 5, 1995, pp. 440-454.
29. S. Dutta, L.N. Van Wassenhove, and S. Kulandaiswamy, "Benchmarking European software management practices", Communications of the ACM, vol. 41, no. 6, 1998, pp. 77-86,
30. A. Goldsworthy, "The Global Information Economy: The Way Ahead", DIST, Commonwealth of Australia, Canberra July, 1997.
31. D.N. Wilson, P. Petocz, and K. Roiter, "A study of software quality assurance in practice", in Software Quality Management III, vol. 1, Ross, Brebbia, Staples, and Stapleton, Eds., 1995, pp. 159-169.
32. M. Azuma and D. Mole, "Software management practice and metrics in the European Community and Japan: some results from a survey", Journal of Systems Software, vol. 26, pp. 5-18, 1994.
33. J. Blackburn, G. Scudder, and L. Van Wassenhove, "Improving speed and productivity of software development: a global survey of software developers", IEEE Transactions on Software Engineering, vol. 22, no. 12, 1996, pp. 875-885.
34. B. Curtis, H. Krasner, and N. Iscoe, "A field study of the software design process for large systems", Communications of the ACM, vol. 31, no. 11, 1988, pp. 1268-1287.
35. M. Zelkowitz, R. Yeh, R. Hamlet, J. Gannon, and V. Basili, "Software Engineering Practices in the U.S. and Japan", IEEE Computer, vol. 17, no. 6, 1984, pp. 57-66.
36. C.J. Davis, J.B. Thompson, P. Smith, and A. Gillies, "A Survey of Approaches to Software Quality within the United Kingdom", University of Sunderland, 1992.
37. C. McPhee and A. Eberlein, "Requirements Engineering for Time-to-Market Projects", presented at 9th IEEE International Conference on the Engineering of Computer Based Systems, Lund, Sweden, 2002.
38. A. Rouse, D. Watson, A. Gilbert, and R. Dilnutt, "A Comparison of the System Development Practices in Australian and United States' Organizations", Monash Information Systems Working Papers Series, no. 24/95 June, 1995.
39. D.R. Jeffery and B. Zucker, "The state of practice in software metrics", School of Info Systems, UNSW, Sydney Tech Report 97/1, 1997.
40. J.M. Verner and W.M. Evanco, "In-house software development: what software project management practices lead to success?", IEEE Software, vol. Jan/Feb, pp. 86-93, 2005.
41. ESI, "Software Best Practice Questionnaire Comparative Analysis of 1995 and 1996", European Software Institute, Bilboa ESI-PIA/96303, November, 1996.
42. S. Dutta, M. Lee, and L. Van Wassenhove, "Software engineering in Europe: a study of best practices", IEEE Software, vol. 16, no. 3, 1999, pp. 82-90.
43. ESI, "1997 Software Best Practice Questionnaire: Analysis of Results", European Software Institute, Bilboa 1997.
44. S. Dutta and L. N. Van Wassenhove, "An empirical study of adoption levels of software management practices within European firms", INSEAD, Fontainebleau, France 97/118/TM, 1997.
45. Gartner Group, "MIS 3000 (database) ", S.P. Group, Ed., Sydney, 1995.
46. Dependable Database Data, "Australia on Disc," 1998.
47. ISO/IEC TR 15504, "Information Technology -Software process assessment, Parts 1-9", ISO/IEC TR 15504:1998(E), 1998.
48. A.P. Cater-Steel, "An evaluation of software development practice and assessment-based process improvement in small software development firms", in EIT. Griffith University, Brisbane, 2004, pp. 465.
49. J.W. Cortada, Best Practices in Information Technology, Prentice Hall, Upper Saddle River, NJ, 1997.
50. P.B. Cragg, "Benchmarking information technology practices in small firms", European Journal of Information Systems, vol. 11, no. 4, 2002, pp. 267-282.
51. R.S. Pressman, "Software Engineering Resources", R.S. Pressman & Associates, 2003, accessed 27 July, http://www.rspa.com/spi/process-agile. html#general.
52. C. Jones, "Variations in software development practices", IEEE Software, vol. Nov/Dec, pp. 22-27, 2003.
53. M. A. Cusumano, A. MacCormack, C. F. Kemerer, and B. Crandall, "Software development worldwide: the state of the practice", IEEE Software, vol. 20, no. 6, 2003, pp. 28-34.
54. ESI, "Software Best Practice Questionnaire: Trend Analysis 1995-1997", European Software Institute, Bilboa ESI-1998-ESI-98-TR-001, 1998.
55. H. Mill, F. Mill, and A. Mill, "Reusing software: issues and research directions", IEEE Transactions on Software Engineering, vol. 21, no. 6, 1995, pp. 528-561.
56. D. A. de Vaus, Response Rate Calculation, 1996.
57. C. M. Paxson, "Follow-up mail survey"s, Industrial Marketing Management, vol. 21, no. 3, 1992, pp. 195-201.
58. T. Stalhane, P. Borgersen, and K. Arnesen, "In search of the customer's quality view", Journal of Systems Software, vol. 38, pp. 85-93, 1997.
59. E. A. Larsen and K. Kautz, "Quality assurance and software process improvement in Norway", Software Process Improvement and Practice, vol. 3, no. 2, 1997, pp. 71-86.
60. G.C. Moore, "An examination of the implementation of information technology for end-users: a diffusion of innovations perspective", University of British Columbia, Vancouver, BC, 1989.
61. M. Khurana, Z. He, I. Court, M. Ross, G. Staples, and D. Wilson, "Software quality practices - an empirical study", Software Quality Journal, no. 5, 1996, pp. 75-85.
62. G.M. Weinberg, "Just say no! Improving the requirements process", American Programmer, vol. 8, no. 10, 1995, pp. 19-23.
63. L. Briand, K. El Emam, and S. Morasca, "On the Application of Measurement Theory in Software Engineering", 1995, accessed 14 June, http://www2.umassd.edu/SWPI/ISERN/isern-95-04.pdf.
64. P.K. Lawlis, R.M. Flowe, and J.B. Thordahl, "A correlational study of the CMM and software development performance", in Crosstalk: The Journal of Defense Software Engineering, 1995.
65. J. Miller, "Statistical significance testing - a panacea for software technology experiments?", The Journal of Systems and Software, vol. 73, pp. 183-192, 2004.
66. P. Attewell and J.B. Rule, "Survey and other methodologies applied to IT impact research: experiences from a comparative study of business computing", in The Information Systems Research Challenge: Survey Research Methods, vol. 3, Harvard Business School Research Colloquium, K. Kraemer, Ed., 1991, pp. 299-315.
67. D.E. Perry, A.A. Porter, and L.G. Votta, "Empirical studies of software engineering: a roadmap", in Proceedings of the Conference on the Future of Software Engineering: ACM Press, 2000, pp. 345-355.