Measurement in software engineering: From the roadmap to the crossroads

International Journal of Software Engineering and Knowledge Engineering

Volumn 18, Issue 1, 2008, Pages 37-64

  Bellini, Carlo Gabriel Porto ^a   Pereira, Rita De Cássia De Faria ^a   Becker, João Luiz ^b  

^a FEDERAL UNIVERSITY OF PARAÍBA   (Brazil)

^b FEDERAL UNIVERSITY OF RIO GRANDE DO SUL   (Brazil)

Author keywords

Complexity; Interpretive data; Measurement theory; Software engineering; Software management; Software measurement; Triangulation

Indexed keywords

COMPUTATIONAL COMPLEXITY; DATA PROCESSING; INFORMATION SYSTEMS; MEASUREMENT THEORY;

INTERPRETIVE DATA; SOFTWARE MANAGEMENT; SOFTWARE MEASUREMENT; SOFTWARE METRICS;

SOFTWARE ENGINEERING;

EID: 44049095768     PISSN: 02181940     EISSN: None     Source Type: Journal    
DOI: 10.1142/S021819400800357X     Document Type: Article

References (122)

1. A. Abran, J. W. Moore. P. Bourque, R. Dupuis, and L. L. Tripp (eds.), Guide to the Software Engineering Body of Knowledge - Trial Version 1.00 (IEEE Computer Society Press, Los Alamitos, 2001).
2. A. Abran, A. Sellami, and W. Suryn, Metrology, measurement and metrics in software engineering, in Proc. 9th Int. Software Metrics Symposium, IEEE, Sydney, 3-5 September, 2003.
3. P. J. A˚gerfalk and B. Fitzgerald, Flexible and distributed software processes: Old petunias in new bowls? Commun. ACM 49(10) (2006) 27-34.
4. L. M. Applegate and J. L. King, Rigor and relevance: Careers on the line, MIS Quarterly 23(1) (1999) 17-18.
5. D. E. Avison and G. Fitzgerald, Information systems development, in Rethinking Management Information Systems: An Interdisciplinary Perspective, eds. W. L. Currie and B. Galliers (Oxford University Press, New York, 1999), pp. 250-278.
6. E. Babbie, Me´todos de Pesquisa de Survey (Universidade Federal de Minas Gerais, Belo Horizonte, Brazil, 1999) [in Portuguese: "Survey Research Methods"].
7. R. P. Bagozzi, Y. Yi, and L. W. Phillips, Assessing construct validity in organizational research, Administrative Science Quarterly 36(3) (1991) 421-458.
8. L. Bardin, Análise de Conteúdo, Edições 70 (Lisbon, 1977) [in Portuguese: "Content Analysis"].
9. M. O. Barros, C. M. L. Werner, and G. H. Travassos, Supporting risks in software project management, J. Systems and Software 70(1-2) (2004) 21-35.
10. V. R. Basili, Is there a future for empirical software engineering? in Proc. ISESE'06, ACM, Rio de Janeiro, Brazil, 21-22 September, 2006.
11. V. R. Basili and H. D. Rombach, The TAME project: Towards improvement-oriented software environments, IEEE Trans. Software Engineering 14(6) (1988) 758-773.
12. S. Beecham, T. Hall, C. Britton, M. Cottee, and A. Rainer, Using an expert panel to validate a requirements process improvement model, J. Systems and Software 76(3) (2005) 251-275.
13. S. Beecham, T. Hall, and A. Rainer, Software process improvement problems in twelve software companies: An empirical analysis, Empirical Software Engineering 8(1) (2003) 7-42.
14. C. G. P. Bellini, M.E.T.R.I.C.S. - Model for Eliciting Team Resources and Improving Competence Structures. A Socio-technical Treatise on Managing Customer Professionals in Software Projects for Enterprise Information Systems, Ph.D. dissertation, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, 2006.
15. C. G. P. Bellini, J. L. Becker, and D. Borenstein, Towards a better understanding of stakeholders' roles in software customization, Int. J. Computers, Systems and Signals 5(1) (2004) 16-31.
16. I. Benbasat, D. K. Goldstein, and M. Mead, The case research strategy in studies of information systems, MIS Quarterly 11(3) (1987) 369-386.
17. I. Benbasat and R. W. Zmud, Empirical research in information systems: The practice of relevance, MIS Quarterly 23(1) (1999) 3-16.
18. J. H. Blackstone Jr., L. R.. Gardiner, and S. C. Gardiner, A framework for the systemic control of organizations, Int. J. Production Research 35(3) (1997) 597-609.
19. J. T. Boardman, Wholes and parts - A systems approach, IEEE Trans. Systems, Man and Cybernetics 25(7) (1995) 1150-1161.
20. J. Bosch and L. Lundberg, Software architecture - Engineering quality attributes, J. Systems and Software 66(3) (2003) 183-186.
21. M. C. Boudreau, D. Gefen, and D. W. Straub, Validation in information systems research: A state-of-the-art assessment, MIS Quarterly 25(1) (2001) 1-16.
22. P. Bourque, R. Dupuis, A. Abran, J. W. Moore, and L. Tripp, The guide to the software engineering body of knowledge, IEEE Software 16(6) (1999) 35-44.
23. L. C. Briand, K. El Emam, and S. Morasca, Theoretical and empirical validation of software product measures, Technical Report ISERN-95-03, 1995.
24. L. C. Briand, K. El Emam, and S. Morasca, On the application of measurement theory in software engineering, Empirical Software Engineering 1(1) (1996) 61-88.
25. L. C. Briand, S. Morasca, and V. R. Basili, An operational process for goal-driven definition of measures, IEEE Trans. Software Engineering 28(12) (2002) 1106-1125.
26. M. E. Bush and N. E. Fenton, Software measurement: A conceptual framework, J. Systems and Software 12(3) (1990) 223-231.
27. J. K. Chhabra, K. K. Aggarwal, and Y. Singh, Code and data spatial complexity: Two important software understandability measures, Information and Software Technology 45(8) (2003) 539-546.
28. G. A. Churchill Jr., A paradigm for developing better measures of marketing constructs. J. Marketing Research 16 (1979) 64-73.
29. D. S. Corbin, Establishing the software development environment, J. Systems Management 42(9) (1991) 28-31.
30. W. L. Currie and B. Galliers (eds.), Rethinking Management Information Systems: An Interdisciplinary Perspective (Oxford University Press. New York. 1999).
31. W. L. Currie and I. A. Glover, Hybrid managers: An example of tunnel vision and regression in management research, in Rethinking Management Information Systems: An Interdisciplinary Perspective, eds. W. L. Currie and B. Galliers (Oxford University Press, New York, 1999), pp. 417-443.
32. T. H. Davenport and M. L. Markus, Rigor versus relevance revisited: Response to Benbasat and Zmud, MIS Quarterly 23(1) (1999) 19-23.
33. C. Dekkers and P. McQuaid, The dangers of using measurement to (mis)manage: Measuring the software process, in Proc. ASQ's Annual Quality Congress (American Society for Quality, Milwaukee, 2002), pp. 551-560.
34. R. F. DeVellis, Scale Development - Theory and Applications (Sage, Newbury Park, 1991).
35. J. A. Díez, A hundred years of numbers. A historical introduction to measurement theory 1887-1990. Part I: The formation period. Two lines of research: Axiomat-ics and real morphisms, scales and invariance, Studies in History and Philosophy Sciences 28(1) (1997) 167-185.
36. L. Donaldson, The normal science of structural contingency theory, in Handbook of Organization Studies, eds. S. R. Clegg, C. Hardy, and W. R. Nord (Sage, London, 1996), pp. 57-76.
37. L. M. Duvall, A study of software management: The state of practice in the United States and Japan, J. Systems and Software 31(2) (1995) 109-124.
38. T. Dybå, Enabling software process improvement: An investigation of the importance of organizational issues, Empirical Software Engineering 7(4) (2002) 387-390.
39. H. Eisner, Essentials of Project and Systems Engineering Management (John Wiley and Sons, New York, 1997).
40. K. El Emam, Software engineering process, in Guide to the Software Engineering Body of Knowledge - Trial Version 1.00, Chap. 9, eds. A. Abran, J. W. Moore, P. Bourque, R. Dupuis, and L. L. Tripp (IEEE Computer Society Press, Los Alamitos, 2001), pp. 137-154.
41. W. M. Evanco and R. Lacovara, A model-based framework for the integration of software metrics, J. Systems and Software 26(1) (1994) 77-86.
42. S. Faraj and L. Sproull, Coordinating expertise in software development teams, Management Science 46(12) (2000) 1554-1568.
43. A. V. Feigenbaum, How to manage for quality in today's economy, Quality Progress 34(5) (2001) 26-27.
44. N. E. Fenton and M. Neil, Software metrics: Successes, failures and new directions, J. Systems and Software 47(2-3) (1999) 149-157.
45. N. E. Fenton and M. Neil, Software metrics: Roadmap, in Proc. 22nd Int. Conf. on Software Engineering, ACM, Limerick. 4-11 June. 2000, pp. 357-370.
46. N. V. Flor. Globally distributed software development and programming, Commun. ACM 49(10) (2006) 57-58.
47. J. M. Ford, T. A. Stetz, M. M. Bott, and B. S. O'Leary, Automated content analysis of multiple-choice test item banks, Social Science Computer Review 18(3) (2000) 258-271.
48. M. B. Franzen and C. G. P. Bellini. Arte ou prática em teste de software? Revista Eletrônica de Administração 11(3) (2005) [in Portuguese: "Art or practice in software testing?"].
49. A. Fuggetta, L. Lavazza, S. Morasca, S. Cinti, G. Oldano, and E. Orazi, Applying GQM in an industrial software factory, ACM Trans. Software Engineering and Methodology 7(4) (1998) 411-448.
50. R. A. Gardner, Resolving the process paradox, Quality Progress 34(3) (2001) 51-59.
51. D. Gefen, D. W. Straub, and M.-C. Boudreau, Structural equation modeling and regression: Guidelines for research practice, Commun. AIS 4 (2000) 1-76.
52. C. Ghezzi, M. Jazayeri, and D. Mandrioli, Fundamentals of Software Engineering (Prentice-Hall, Englewood Cliffs, 1991).
53. R. L. Glass. I. Vessey. and V. Ramesh, Research in software engineering: An analysis of the literature, Information and Software Technology 44(8) (2002) 491-506.
54. A. Gopal, M. S. Krishnan, T. Mukhopadhyay, and D. R,. Goldenson, Measurement programs in software development: Determinants of success, IEEE Trans. Software Engineering 28(9) (2002) 863-875.
55. J. F. Hair Jr., R. E. Anderson, R. L. Tatham, and W. C. Black, Multivariate Data Analysis (Prentice Hall, Upper Saddle River, 1998).
56. N. Hanebutte. C. S. Taylor, and R. R. Dumke, Techniques of successful application of factor analysis in software measurement, Empirical Software Engineering 8(1) (2003) 43-57.
57. W. Harrison, A flexible method for maintaining software metrics data: A universal metrics repository, J. Systems and Software 72(2) (2004) 225-234.
58. W. Hetzel, The measurement process, in Applying Software Metrics, eds. P. Oman and S. L. Pfleeger (IEEE, Los Alamitos, 1997), pp. 72-93.
59. G. H. Hofstede. Cultures and Organizations (Harper Collins. London. 1994).
60. R. Hoving, Executive response: Project management process maturity as a "secret weapon", MIS Quarterly Executive 2(1) (2003) 29-30.
61. J. Hussey and R. Hussey, Business Research (Palgrave Macmillan, New York, 1997).
62. J. Iversen and L. Mathiassen. Cultivation and engineering of a software metrics program, Information Systems Journal 13(1) (2003) 3-19.
63. M. C. Jackson and P. Keys, Towards a system of systems methodologies, J. Operational Research Society 35(6) (1984) 473-486.
64. M. Jørgensen, Software quality measurement, Advances in Engineering Software 30(12) (1999) 907-912.
65. M. Jørgensen, A review of studies on expert estimation of software development effort, J. Systems and Software 70(1-2) (2004) 37-60.
66. M. Jørgensen. K. H. Teigen, and K. Moløkken. Better sure than safe? Over-confidence in judgement-based software development effort prediction intervals, J. Systems and Software 70(1-2) (2004) 79-93.
67. R. S. Kaplan and D. P. Norton, The Balanced Scorecard (Harvard Business School Press, Boston, 1996).
68. P. Katerattanakul and B. Han. Are European IS journals under-rated? An answer based on citation analysis, European Journal of Information Systems 12(1) (2003) 60-71.
69. M. Keil, J. Mann, and A. Rai, Why software projects escalate: An empirical analysis and test of four theoretical models, MIS Quarterly 24(4) (2000) 631-664.
70. T. M. Khoshgoftaar, E. B. Allen, and D. L. Lanning, An information theory-based approach to quantifying the contribution of a software metric, J. Systems and Software 36(2) (1997) 103-113.
71. T. M. Khoshgoftaar and N. Seliya, Fault prediction modeling for software quality estimation: Comparing commonly used techniques, Empirical Software Engineering 8(3) (2003) 255-283.
72. S. R. Kirk and S. Jenkins, Information theory-based software metrics and obfuscation, J. Systems and Software 72(2) (2004) 179-186.
73. B. Kitchenham, Procedures for performing systematic reviews, Joint Technical Report - Keele University Technical Report TR/SE-0401 and National ICT Australia Ltd., Technical Report 040011T.1, 2004.
74. A. S. Lee, Rigor and relevance in MIS research: Beyond the approach of positivism alone, MIS Quarterly 23(1) (1999) 29-34.
75. H. K. N. Leung, Quality metrics for intranet application, Information and Management 38(3) (2001) 137-152.
76. M. Lindvall, R. T. Tvedt, and P. Costa, An empirically-based process for software architecture evaluation, Empirical Software Engineering 8(1) (2003) 83-108.
77. F. Lopes and P. Morais, Lessons learned from the teaching of IS development, J. Information Technology Education 1(2) (2002) 103-112.
78. K. Lyytinen, Empirical research in information systems: On the relevance of practice in thinking of IS research, MIS Quarterly 23(1) (1999) 25-28.
79. S. G. MacDonell and A. R. Gray, Software engineering management, in Guide to the Software. Engineering Body of Knowledge - Trial Version 1.00, Chap. 8, eds. A. Abran, J. W. Moore, P. Bourque, R. Dupuis, and L. L. Tripp (IEEE Computer Society Press, Los Alamitos, 2001), pp. 121-135.
80. L. Mari, The meaning of "quantity" in measurement, Measurement 17(2) (1996) 127-138.
81. L. Mari, Epistemology of measurement, Measurement 34(1) (2003) 17-30.
82. J. G. McDaniel, Improving system quality through software evaluation, Computers in Biology and Medicine 32(3) (2002) 127-140.
83. S. Morasca, Foundations of a weak measurement-theoretic approach to software measurement, in Proc. 6th Int. Conf. on Fundamental Approaches to Software Engineering, Warsaw, 7-11 April, 2003, pp. 200-215.
84. J. Münch and J. Heidrich, Software project control centers: Concepts and approaches, J. Systems and Software 70(1-2) (2004) 3-19.
85. N. A. Mylonopoulos and V. Theoharakis, Global perceptions of IS journals, Commun. ACM 44(9) (2001) 29-33.
86. R. Nelson and S. G. Winter, An Evolutionary Theory of Economic Change (Belknap Press, Cambridge, 1982).
87. P. Oman and S. L. Pfleeger (eds.), Applying Software Metrics (IEEE, Los Alamitos, 1997).
88. J. S. Osmundson, J. B. Michael, M. J. Machniak, and M. A. Grossman, Quality management metrics for software development, Information and Management 40(8) (2003) 799-812.
89. P. Palvia, E. Mao, A. F. Salam, and K. S. Soliman, Management information systems research: What's there in a methodology? Commun. AIS 11 (2003) 289-309.
90. S. C. Palvia. R. S. Sharma, and D. W. Conrath. A socio-technical framework for quality assessment of computer information systems, Industrial Management and Data Systems 101(5) (2001) 237-251.
91. D. Parnas, Agile methods and GSD: The wrong solution to an old but real problem, Commun. ACM 49(10) (2006) 29.
92. R. Pavur, M. Jayakumar, and H. Clayton. Software testing metrics: Do they have merit? Industrial Management and Data Systems 99(1) (1999) 5-10.
93. J. M. Pearson, A. Pearson, and J. P. Shim, The relevancy of information systems research: The practitioner's view, Information Resources Management Journal 18(3) (2005) 50-67.
94. A. Peled, Creating winning information technology project teams in the public sector, Team, Performance Management 6(1/2) (2000) 6-14.
95. S. L. Pfleeger, Maturity, models, and goals: How to build a metrics plan, J. Systems and Software 31(2) (1995) 143-155.
96. M. T. Pich, C. H. Loch, and A. De Meyer, On uncertainty, ambiguity, and complexity in project management, Management Science 48(8) (2002) 1008-1023.
97. R. S. Pressman, Software Engineering: A Practitioner's Approach (McGraw-Hill, New York, 2001).
98. A. Rainer and T. Hall, A quantitative and qualitative analysis of factors affecting software processes, J. Systems and Software 66(1) (2003) 7-21.
99. B. Ramesh, L. Cao, K. Mohan, and P. Xu, Can distributed software development be agile? Commun. ACM 49(10) (2006) 41-46.
100. T. Ravichandran and A. Rai, Quality management in systems development: An organizational system perspective, MIS Quarterly 24(3) (2000) 381-415.
101. L. D. Richards and S. K. Gupta, The systems approach in an information society: A reconsideration, J. Operational Research Society 36(9) (1985) 833-843.
102. G. B. Rossi, A probabilistic model for measurement processes, Measurement 34(2) (2003) 85-99.
103. F. Ruiz, M. Genero, F. García, M. Piattini, and C. Calero, A proposal of a software measurement ontology, in Proc. 4th Argentine Symposium of Software Engineering, SADIO, Buenos Aires, 1-3 September, 2003.
104. W. S. Sarle, Measurement theory: Frequently asked questions, in Disseminations of the International Statistical Applications Institute, ed. W. S. Sarle, ACG, Wichita, 1995, pp. 61-66.
105. H. Scarbrough, The management of knowledge workers, in Rethinking Management Information Systems: An Interdisciplinary Perspective, eds. W. L. Currie and B. Galliers (Oxford University Press, New York, 1999), pp. 474-496.
106. F. L. Schmidt and J. E. Hunter, Theory testing and measurement error, Intelligence 27(3) (1999) 183-198.
107. A. H. Segars, Assessing the unidimensionality of measurement: A paradigm and illustration within the context of information systems research, Omega 25(1) (1997) 107-121.
108. N. G. Shaw, Identifying relationships among factors in IS implementation, Commun. AIS 11 (2003) 155-165.
109. H. J. Smith and M. Keil, The reluctance to report bad news on troubled software projects: A theoretical model, Information Systems Journal 13(1) (2003) 69-95.
110. I. Sommerville, Software Engineering (Addison-Wesley, Harlow, 2001).
111. I. Stamelos, L. Angelis, M. Morisio, E. Sakellaris, and G. L. Bleris, Estimating the development cost of custom software, Information and Management 40(8) (2003) 729-741.
112. P. Stein, By their measures shall ye know them, Quality Progress 34(5) (2001) 72-74.
113. D. W. Straub, Validating instruments in MIS research, MIS Quarterly 13(2) (1989) 147-169.
114. D. W. Straub. D. L. Hoffman, B. W. Weber, and C. Steinfield, Toward new metrics for net-enhanced organizations, Information Systems Research 13(3) (2002) 227-238.
115. P. A. Todd, J. D. Mckeen, and R. B. Gallupe, The evolution of IS job skills: A content analysis of IS job advertisements from 1970 to 1990, MIS Quarterly 19(1) (1995) 1-27.
116. P. S. Tolbert and L. G. Zucker, The institutionalization of institutional theory, in Handbook of Organization Studies, eds. S. R. Clegg, C. Hardy, and W. R. Nord, Sage, London, 1996, pp. 175-190.
117. P. Wagstrom and J. Herbsleb, Dependency forecasting in the distributed agile organization, Commun. ACM 49(10) (2006) 55-56.
118. D. Wallace and L. Reeker, Software quality, in Guide to the Software Engineering Body of Knowledge - Trial Version 1.00, Chap. 11, eds. A. Abran, J. W. Moore, P. Bourque, R. Dupuis, and L. L. Tripp, IEEE Computer Society Press, Los Alamitos, 2001, pp. 165-183.
119. I. Wieczorek, Improved software cost estimation - a robust and interpretable modelling method and a comprehensive empirical investigation, Empirical Software Engineering 7(2) (2002) 177-180.
120. C. Wohlin and A. A. Andrews, Prioritizing and assessing software project success factors and project characteristics using subjective data, Empirical Software Engineering 8(3) (2003) 285-308.
121. M. H. Zack, Managing codified knowledge, Sloan Management Review 40(4) (1999) 45-58.
122. K. Zhu and K. L. Kraemer, E-commerce metrics for Net-enhanced organizations: Assessing the value of e-commerce to firm performance in the manufacturing sector, Information Systems Research 13(3) (2002) 275-295.