Analysis of interrater agreement in ISO/IEC 15504-based software process assessment

Proceedings - 2nd Asia-Pacific Conference on Quality Software, APAQS 2001

Volumn , Issue , 2001, Pages 341-348

  Lee, Hye Young ^a   Jung, Ho Won ^a   Chung, Chang Shin ^a   Lee, Jong Moo ^a   Lee, Kyung Whan ^a   Jeong, Hack Jong ^a  

^a Korea University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

RELIABILITY; STANDARDS;

INTER-RATER AGREEMENTS; INTERNAL RELIABILITIES; ISO/IEC; ISO/IEC 15504; PROCESS ASSESSMENTS; SOFTWARE PROCESS; SOFTWARE PROCESS ASSESSMENT;

SOFTWARE RELIABILITY;

EID: 84954058529     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/APAQS.2001.990039     Document Type: Conference Paper

References (24)

1. Agresti, Categorical Data Analysis, Wiley, New York, 1990.
2. F. Coallier, TRILLIUM: A model for the assessment of telecom product development and support capability, Software Process Newsletter 2 (1995) 3-8.
3. J. Cohen, Weighted Kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 70 (4) (1968) 213-220.
4. J. Cohen, A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20 (1) (1960)
5. K. El Emam, Benchmarking Kappa: Interrater agreement in software process assessments, Empirical Software Engineering 4 (1999) 113-133.
6. K. El Emam, L. Briand, B. Smith, Assessor agreement in rating SPICE process, Software Process Improvement and Practice 2 (2) (1996) 123-148.
7. nd Int'l Software Engineering Standards Symposium (1995) 84-97.
8. K. El Emam, D.R. Goldenson, L. Briand, P. Marshall. Interrater agreement in SPICE-based assessments: Some preliminary results, Int'l Conference on the Software Process (1996) 149-156.
9. K. El Emam, H.-W. Jung, An empirical evaluation of the ISOAEC 15504 assessment model, to appear in Journal of Systems and Software.
10. K. El Emam, N.H. Madhavji, The reliability of measuring organizational maturity. Software Process Improvement and Practice, 1 (1) (1995) 3-25.
11. Everitt, The Analysis of Contingency Tables, Chapman & Hall, 1992.
12. J.L.Fleiss, J. Cohen, The equivalence of weighted Kappa and the interclass correlation coefficient as measured of reliability, Educational and Psychological Measurement 33 (1973) 613-619.
13. P. Fusaro, K. El Emam, B. Smith, The internal consistencies of the 1987 SEI maturity questionnaire and the SPICE capability dimension, Technical Report, International Software Engineering Research Network, H.-W. Jung, R. Hunter, D.R. Goldenson, K. El Emam, Findings from phase 2 of the SPICE trials, submitted to the publication.
14. H. Kitson, Why does 15504 matter, SPICE 2000 Conference (2000) 7-8.
15. P. Kuvaja, A. Bicego, A European assessment methodology, Software Quality Journal 3 (1994) 117-127.
16. P. Kuvaja, BOOTSTRAP 3.0 - A SPICE conformant software process assessment methodology, Software Quality Journal 8 (1999) 7-19
17. J.C. Nunnally, I.H. Bemstein, Psychometric Theory, McGraw-Hill, Inc., New York, 1994.
18. M.C. Paulk, C.V. Webber, S.M. Garcia, M. Chrissis, M. Bush, Key Practices of the Capability Maturity Model, version 1.1. Software Engineering Institute, CMU/SEI-93-TR-25(1993)
19. T.P Route, A.Tuffley, B. Cahill, B. Hodgen, The Rapid assessment of software process capability, SPICE 2000 Conference (2000), 47-55.
20. J.M. Simon, K. El Emam, S. Rousseau, E. Jacquet, F. Babey, The reliability of ISO/IEC PDTR 15504 assessments, Software Process Improvement and Practice 3 (3) (1997) 177-188.
21. R. Smith, K. El Emam, Transitioning to phase 2 of the SPICE trials, SPICE '96 Conference (1996), 45-55.
22. ISO/IEC DIS 14598-1, Information Technology - Software product Evaluation: Part 1 : General Overview, ISO/IEC, Geneva, 1999.
23. ISO, ISO/IEC TR2 15504, Part 1 - Part 9, Information Technology - Software Process Assessment, ISO, Geneva, 1993.
24. SPICE, Phase 2 Trials lnterim Report, Version 1.0, 1998.