Validating the ISO/IEC 15504 measure of software requirements analysis process capability

IEEE Transactions on Software Engineering

Volumn 26, Issue 6, 2000, Pages 541-566

  Emam, Khaled El ^a   Birk, Andreas ^b  

^a NATIONAL RESEARCH COUNCIL CANADA   (Canada)

^b Fraunhofer Institut for Experimental Software Engineering   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; INFORMATION TECHNOLOGY; REQUIREMENTS ENGINEERING; STANDARDS;

PREDICTIVE VALIDITY; SOFTWARE PROCESS IMPROVEMENT; SOFTWARE QUALITY; SOFTWARE REQUIREMENTS ANALYSIS PROCESS;

SOFTWARE ENGINEERING;

EID: 0034205641     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/32.852742     Document Type: Article

References (102)

1. B. Baker, C. Hardyck, and L. Petrinovich, "Weak Measurements vs. Strong Statistics: An Empirical Critique of S.S. Stevens' Proscriptions on Statistics," Educational and Psychological Measurement, vol. 26. pp. 291-309, 1966.
2. S. Benno and D. Frailey, "Software Process Improvement in DSEG: 1989-1995," Texas Instruments Technical ., vol. 12, no. 2, pp. 20-28, Mar.-Apr. 1995.
3. A. Bicego, M. Khurana, and P. Kuvaja, "Bootstrap 3.0: Software Process Assessment Methodology," Proc. Software Quality Managment, 1998.
4. G. Bohrnstedt and T. Carter, "Robustness in Regression Analysis," Sociological Methodology, H. Costner ed., Jossey-Bass, 1971.
5. L. Briand, K. El Emam, and S. Morasca, "On the Application of Measurement Theory in Software Engineering," Empirical Software Eng., An Int'l ., vol. 1, no. 1, pp. 61-88, 1996.
6. J. Brodman and D. Johnson, "What Small Businesses and Small Organizations Say about the CMM," Proc. 16th Int'l Conf. Software Eng., pp. 331-340, 1994.
7. J. Brodman and D. Johnson, "Return on Investment (ROI) from Software Process Improvement as Measured by US Industry," Software Process: Improvement and Practice, Pilot Issue, John Wiley & Sons, 1995.
8. K. Butler, "The Economic Benefits of Software Process Improvement," Crosstalk, vol. 8, no. 7, pp. 14-17, July 1995.
9. D. Card, "Understanding Process Improvement," IEEE Software, pp. 102-103, July 1991.
10. B. Clark, "The Effects of Software Process Maturity on Software Development Effort," PhD thesis, Univ. of Southern California, Apr. 1997.
11. F. Coallier, J. Mayrand, and B. Lague, "Risk Management in Software Product Procurement," Elements of Software Process Assessment and Improvement, K. El Emam and N.H. Madhavji eds., IEEE CS Press, 1999.
12. W. Cochran, Planning and Analysis of Observational Studies. John Wiley & Sons, 1983.
13. J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Assoc., 1988.
14. L. Cronbach, "Coefficient Alpha and the Internal Structure of Tests," Psychometrika, pp 297-334, Sept. 1951.
15. M. David, R. Little, M. Samuhel, and R. Triest, "Imputation Models Based on the Propensity to Respond," Proc. Business and Economics Section, Am. Statistical Assoc., pp. 168-173, 1983.
16. C. Deephouse, D. Goldenson, M. Kellner, and T. Mukhopadhyay, "The Effects of Software Processes on Meeting Targets and Quality," Proc. Hawaiian Int'l Conf. Systems Sciences, vol. 4, pp. 710-719, Jan. 1995.
17. R. Dion, "Elements of a Process Improvement Program," IEEE Software, vol. 9, no. 4, pp. 83-85, July 1992.
18. R. Dion, "Process Improvement and the Corporate Balance Sheet," IEEE Software, vol. 10, no. 4, pp. 28-35, July 1993. 19 S. Dutta and L. van Wassenhove, "An Empirical Study of Adoption Levels of Software Management Practices within European Firms," INSEAD Research Initiative in Software Excellence Working Paper, 1997
19. 20 K. El Emam, "The Internal Consistency of the ISO/IEC 15504 Software Process Capability Scale," Proc. Fifth Int'l Symp. Software Metrics, pp. 72-81, IEEE CS Press, 1998. 21 K. El Emam and D.R. Goldenson, "SPICE: An Empiricist's Perspective," Proc. Second IEEE Int'l Software Eng. Standards Symp., pp. 84-97, Aug. 1995.
20. K. El Emam and N.H. Madhavji, "The Reliability of Measuring Organizational Maturity," Software Process: Improvement and Practice, vol. 1, no. 1, pp. 3-25, 1995. 23 K. El Emam, S. Quintin, and N.H. Madhavji, "User Participation in the Requirements Engineering Process: An Empirical Study," Requirements Eng. J., vol. 1, pp. 4-26, 1996.
21. K. El Emam, J.-N. Drouin, and W. Melo, SPICE: The Theory and Practice of Software Process Improvement and Capability Determination. K. El Emam, J.-N. Drouin, and W. Melo, eds., IEEE CS Press,
22. 1998. 25 K. El Emam, J.-M. Simon, S. Rousseau, and E. Jacquet, "Cost Implications of Interrater Agreement for Software Process Assss-ments," Proc. Fifth Int'l Symp. Software Metrics, pp. 38-51, 1998.
23. K. El Emam and L. Briand, "Costs and Benefits of Software Process Improvement," Better Software Practice for Business Benefit: Principles and Experience, R. Messnarz and C. Tully, eds., IEEE CS Press, 1999.
24. K. El Emam and D. Goldenson, "An Empirical Review of Software Process Assessments," Advances in Computers, to appear, 2000.
25. R. Flowe and J. Thordahl, "A Correlational Study of the SEI's Capability Maturity Model and Software Development Performance in DOD Contracts," MSc thesis, U.S. Air Force Inst. of Technology, 1994.
26. B. Ford, "An Overview of Hot-Deck Procedures," Incomplete Data in Sample Surveys, Volume 2: Theory and Bibliographies, W. Madow, I. OIkin, and D. Rubin, eds., Academic Press, 1983.
27. C. Franz and D. Robey, "Organizational Context, User Involve-ment, and the Usefulness of Information Systems," Decision Sciences, vol. 17, pp. 329-356, 1986.
28. P. Fusaro, K. El Emam, and B. Smith, "The Internal Consistencies of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension," Empirical Software Eng.: An Int'l ., vol. 3, pp. 179-201, 1997.
29. D. Galletta and A. Lederer, "Some Cautions on the Measurement of User Information Satisfaction," Decision Sciences, vol. 20, pp. 419-438, 1989.
30. P. Gardner, "Scales and Statistics," Review of Educational Research, vol. 45, no. 1, pp. 43-57, Winter 1975.
31. D.R. Goldenson and J.D. Herbsleb, "After the Appraisal: A Systematic Survey of Process Improvement, Its Benefits, and Factors that Influence Success," Technical Report CMU/SEI-95-TR-009, Software Engineering Institute, 1995.
32. D. Goldenson, K. El Emam, J. Herbsleb, and C. Deephouse, "Empirical Studies of Software Process Assessment Methods," Elements of Software Process Assessment and Improvement, K. El Emam and N.H. Madhavji, eds., IEEE CS Press, 1999.
33. A. Gopal, T. Mukhopadhyay, and M. Krishnan, "The Role of Software Processes and Communication in Offshore Software Development," submitted for publication, 1997.
34. W. Harmon, "Benchmarking: The Starting Point for Process Improvement," Proc. ESI Workshop on Benchmarking and Software Process Improvement, Apr. 1998.
35. J. Herbsleb, A. Carleton, J. Rozum, J. Siegel, and D. Zubrow, "Benefits of CMM-Based Software Process Improvement: Initial Results," Technical Report, CMU-SEI-94-TR-13, Software Eng. Inst., 1994.
36. D. Hosmer and S. Lemeshow, Applied Logistic Regression. John Wiley & Sons, 1989.
37. W. Humphrey, "Characterizing the Software Process: A Maturity Framework," IEEE Software, pp. 73-79, vol. 5, no. 3, Mar. 1988.
38. W. Humphrey, T. Snyder, and R. Willis, "Software Process Improvement at Hughes Aircraft," IEEE Software, pp. 11-23, July 1991.
39. M. Ibanez and H. Rempp, "European User Survey Analysis," ESPITI Project Report, Feb. 1996. (available from http://www.esi.es),
40. B. Ives, M. Oison, and J. Baroudi, "The Measurement of User Information Satisfaction," Comm. ACM, vol. 26, no. 10, pp. 785-793, 1983.
41. C. Jones, Assessment and Control of Software Risks. Prentice Hall, 1994.
42. C. Jones, "The Pragmatics of Software Process Improvements," Software Process Newsletter, IEEE CS Technical Council on Software Eng., no. 5, pp. 1-4, Winter 1996. (available at http://www.seg.iit.nrc.ca/SPN),
43. C. Jones, "The Economics of Software Process Improvements," Elements of Software Process Assessment and Improvement, K. El Emam and N. H. Madhavji, eds., IEEE CS Press, 1999.
44. F. Kerlinger, Foundations of Behavioral Research. Holt, Rinehart, and Winston, 1986.
45. E. Kim and J. Lee, "An Exploratory Contingency Model of User Participation and MIS Use," Information and Management, vol. 11, pp. 87-97, 1986.
46. H. Krasner, "The Payoff for Software Process Improvement: What it is and How to Get it," Elements of Software Process Assessment and Improvement, K. El Emam and N. H. Madhavji, eds., IEEE CS Press, 1999.
47. M. Krishnan and M. Kellner, "Measuring Process Consistency: Implications for Reducing Software Defects," submitted for publication. Mar. 1998.
48. S. Labovitz, "Some Observations on Measurement and Statistics," Social Forces, vol. 46, no. 2, pp. 151-160, Dec. 1967.
49. S. Labovitz, "The Assignment of Numbers to Rank Order Categories," Am. Sociological Review, vol. 35, pp. 515-524, 1970.
50. P. Lawlis, R. Flowe, and J. Thordahl, "A Correlational Study of the CMM and Software Development Performance," Software Process Newsletter, IEEE CS Technical Council on Software Eng., no. 7, pp. 15, Fall 1996. (available at http://www.seg.iit.nrc.ca/SPN),
51. L. Lebsanft, "Bootstrap: Experiences with Europe's Software Process Assessment and Improvement Method," Software Process Newsletter, IEEE CS Technical Council on Software Eng., no. 5, pp. 610, Winter 1996. (available at http://www.seg.iit.nrc.ca/SPN),
52. J. Lee and S. Kim, "The Relationship between Procedural Formalization in MIS Development and MIS Success," Information and Management, vol. 22, pp. 89-111, 1992.
53. R. Lindsay and A. Ehrenberg, "The Design of Replicated Studies," The Am. Statistician, vol. 47, no. 3, pp. 217-228, 1993.
54. W. Lipke and K. Butler, "Software Process Improvement: A Success Story," Crosstalk, vol. 5, no. 9, pp. 29-39, Sept. 1992.
55. R. Little and R. Rubin, Statistical Analysis with Missing Data. John Wiley & Sons, 1987.
56. F. McGarry, S. Burke, and B. Decker, "Measuring the Impacts Individual Process Maturity Attributes Have on Software Projects," Proc. Fifth Int'l Software Metrics Symp., pp. 52-60, 1998.
57. J. Mclver and E. Carmines, Unidimensional Scaling. Sage Publications, 1981.
58. J. McKeen, T. Guimaraes, and J. Wetherbe, "The Relationship between User Participation and User Satisfaction: An Investigation of Four Contingency Factors," M/S Quarterly, pp. 427-451, Dec. 1994.
59. J. Nunnally and I. Bernstein, Psychometric Theory. McGraw-Hill,
60. M. Paulk, B. Curtis, M.-B. Chrissis, and C. Weber, "Capability
61. Maturity Model, Version 1.1," IEEE Software, vol. 10, no. 4,
62. pp. 18-27, July 1993. 64 M. Paulk and M. Konrad, "Measuring Process Capability versus
63. Organizational Process Maturity," Proc. Fourth Int l Conf. Software
64. Quality, Oct. 1994. 65 J. Rice, Mathematical Statistics and Data Analysis. Duxbury Press,
65. P. Rosenbaum and D. Rubin, "The Central Role of the Propensity
66. Score in Observational Studies for Causal Effects," Biometrika,
67. vol. 70, no. 1, pp. 41-55, 1983. 67 P. Rosenbaum and D. Rubin, "Constructing a Control Group
68. Using Multivariate Matched Sampling Methods that Incorporate
69. the Propensity Score," The Am. Statistician, vol. 39, no. 1, pp. 33-38,
70. R. Rosenthal, "Replication in Behavioral Research," Replication
71. Research in the Social Sciences, . Neuliep, ed., Sage Publications,
72. D. Rubin, "The Bayesian Bootstrap," The Annals of Statistics, vol. 9,
73. no. 1, pp. 130-134, 1981. 70 D. Rubin, Multiple Imputation for Nonresponse in Surveys. John
74. Wiley & Sons, 1987. 71 D. Rubin, "An Overview of Multiple Imputation," Proc. Survey
75. Research Section, Am. Statistical Assoc., pp. 79-84, 1988. 72 D. Rubin and N. Schenker, "Multiple Imputation for Interval
76. Estimation from Simple Random Samples with Ignorable Nonresponse," J. Am. Statistical Assoc., vol. 81, no. 394, pp. 366-374,
77. D. Rubin and N. Schenker, "Multiple Imputation in Health Care
78. Databases: An Overview," Statistics in Medicine, vol. 10,
79. pp. 585-598, 1991. 74 H. Rubin, "Software Process Maturity: Measuring its Impact on
80. Productivity and Quality," Proc. Int'l Conf. Software Eng.,
81. pp. 468-476, 1993. 75 H. Rubin, "Findings of the 1997 Worldwide Benchmark Project:
82. Worldwide Software Engineering Performance Summary," Meta
83. Group, 1998. 76 D. Rubin, H. Stern, and V. Vehovar, "Handling 'Don't Know'
84. Survey Responses: The Case of the Slovenian Plebiscite," J. Am.
85. Statistical Assoc., vol. 90, no. 431, pp. 822-828, 1995. 77 I. Sande, "Hot-Deck Imputation Procedures," Incomplete Data in
86. Sample Surveys, Volume 3: Proc. Symp., W. Madow and I. Olkin,
87. eds., Academic Press, 1983. 78 J. Schaefer, Analysis of Incomplete Multivariate Data. Chapman &
88. Hall, 1997.
89. V. Sethi and W. King, "Construct Measurement in Information Systems Research: An Illustration in Strategic Systems," Decision Sciences, vol. 22, pp. 455-472, 1991.
90. S. Siegel and J. Castellan, Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, 1988.
91. The Capability Maturity Model: Guidelines for Improving the Software Process. Software Eng. Inst. Addison Wesley, 1995.
92. "Software Engineering Institute C4 Software Technology Refer-ence Guide-A Prototype/'Handbook CMU/SEI-97-HB-OOl, Software Eng. Inst. 1997.
93. "Top-Level Standards Map,"Software Eng. Inst., Feb. 1998. available at http://www.sei.cmu.edu/activities/cmm/cmm.articles.html,
94. "CMMI A Specification Version 1.1," Software Eng. Inst., Apr. 1998. available at http://www.sei.cmu.edu/activities/cmm/cmmi/specs/aspecl.1.html,
95. I. Sommerville and P. Sawyer, Requirements Engineering: A Good Practice Guide. John Wiley & Sons, 1997.
96. P. Spector, "Ratings of Equal and Unequal Response Choice Intervals," J. Social Psychology, vol. 112, pp. 115-119, 1980.
97. "The SPIRE Handbook: Better Faster Cheaper Software Development in Small Companies/'The SPIRE Project, ESSI Project 23873, Nov. 1998.
98. S. Stevens, "Mathematics, Measurement, and Psychophysics," Handbook of Experimental Psychology, S. Stevens, ed., John Wiley & Sons, 1951.
99. A. Subramanian and S. Nilakanta, "Measurement: A Blueprint for Theory-Building in MIS," Information and Management, vol. 26, pp. 13-20, 1994.
100. D. Treiman, W. Bielby, and M. Cheng, "Evaluating a Multiple Imputation Method for Recalibrating 1970 U.S. Census Detailed Industry Codes to the 1980 Standard," Sociological Methodology, vol. 18, 1988.
101. P. Velleman and L. Wilkinson, "Nominal, Ordinal, Interval, and Ratio Typologies Are Misleading," The Am. Statistician, vol. 47, no. 1, pp. 65-72, Feb. 1993.
102. H. Wohlwend and S. Rosenbaum, "Software Improvements in an International Company," Proc. Int'l Conf. Software Eng., pp. 212-220, 1993.