On the Shortcomings of Shortened Tests: A Literature Review

International Journal of Testing

Volumn 13, Issue 3, 2013, Pages 223-248

  Kruyen, Peter M ^a   Emons, Wilco H M ^b   Sijtsma, Klaas ^b  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^b TILBURG UNIVERSITY   (Netherlands)

Author keywords

literature review; psychological tests; reliability of short tests; test shortening; validity of short tests

Indexed keywords

EID: 84879065308     PISSN: 15305058     EISSN: 15327574     Source Type: Journal    
DOI: 10.1080/15305058.2012.703734     Document Type: Review

References (60)

1. Alden, L. E., Wiggins, J. S. and Pincus, A. L. 1990. Construction of circumplex scales for the inventory of interpersonal problems. Journal of Personality Assessment, 55: 521 - 536. doi: 10.1080/00223891.1990.9674088
2. Allen, M. J. and Yen, W. M. 1979. Introduction to measurement theory, Monterey, CA: Brooks/Cole.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. 1999. Standards for educational and psychological testing, Washington, DC: American Educational Research Association.
4. Anastasi, A. 1982. Psychological testing (5th ed.), New York, NY: Macmillan.
5. Beck, A. T. and Beck, R. W. 1972. Screening depressed patients in family practice. A rapid technic. Postgraduate Medicine, 52: 81 - 85.
6. Bentler, P. A. and Woodward, J. A. 1980. Inequalities among lower bounds to reliability: With applications to test construction and factor analysis. Psychometrika, 45: 249 - 267. doi: 10.1007/BF02294079
7. Borsboom, D., Mellenbergh, G. J. and Van Heerden, J. 2004. The concept of validity. Psychological Review, 111: 1061 - 1071. doi: 10.1037/0033-295X.111.4.1061
8. Burisch, M. 1997. Test length and validity revisited. European Journal of Personality, 11: 303 - 315. doi: 10.1002/(SICI)1099-0984(199711)11:4<303::AID-PER292>3.0.CO;2-#
9. Butcher, J. N. and Hostetler, K. 1990. Abbreviating MMPI item administration: What can be learned from the MMPI for the MMPI-2?. Psychological Assessment, 2: 12 - 21. doi: 10.1037/1040-3590.2.1.12
10. Charter, R. A. 2003. A breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability. The Journal of General Psychology, 130: 290 - 304. doi: 10.1080/00221300309601160
11. Clark, L. A. and Watson, D. 1995. Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7: 309 - 319. doi: 10.1037//1040-3590.7.3.309
12. Cook, T. D. and Campbell, D. T. 1979. Quasi-experimentation: Design and analysis for field settings, Chicago, IL: Rand McNally.
13. Cooper, A. and Petrides, K. V. 2010. A psychometric analysis of the Trait Emotional Intelligence Questionnaire-Short Form (TEIQue-SF) using item response theory. Journal of Personality Assessment, 92: 449 - 457. doi: 10.1080/00223891.2010.497426
14. Costa, P. T. Jr. and McCrae, R. R. 1992. The NEO PI/FFI manual supplement, Odessa, FL: Psychological Assessment Resources.
15. Coste, J., Guillemin, F., Pouchot, J. and Fermanian, J. 1997. Methodological approaches to shortening composite measurement scales. Journal of Clinical Epidemiology, 50: 247 - 252. doi: 10.1016/S0895-4356(96)00363-0
16. Derogatis, L. R. 2001. Brief Symptom Inventory (BSI)-18 administration, scoring and procedures manual, Minneapolis, MN: NCS Pearson.
17. Derogatis, L. R. and Melisaratos, N. 1983. The Brief Symptom Inventory: An introductory report. Psychological Medicine, 13: 595 - 605. doi: 10.1017/S0033291700048017
18. Emons, W. H. M., Sijtsma, K. and Meijer, R. R. 2007. On the consistency of individual classification using short scales. Psychological Methods, 12: 105 - 120. doi: 10.1037/1082-989x.12.1.105
19. Evers, A. V. A. M., Sijtsma, K., Lucassen, W. and Meijer, R. R. 2010. The Dutch review process for evaluating the quality of psychological tests: History, procedure and results. International Journal of Testing, 10: 295 - 317. doi: 10.1080/15305058.2010.518325
20. George, S. M., Connor, J. P., Gullo, M. J. and Young, R. M. 2010. A prospective study of personality features predictive of early adolescent alcohol misuse. Personality and Individual Differences, 49: 204 - 209. doi: 10.1016/j.paid.2010.03.036
21. Goldberg, L. R. 1992. The development of markers for the big-five factor structure. Psychological Assessment, 4: 26 - 42. doi: 10.1037//1040-3590.4.1.26
22. Ham, L. S., Stewart, S. H., Norton, P. J. and Hope, D. A. 2005. Psychometric assessment of the Comprehensive Effects of Alcohol Questionnaire: Comparing a brief version to the original full scale. Journal of Psychopathology and Behavioral Assessment, 27: 141 - 158. doi: 10.1007/s10862-005-0631-9
23. Hofsess, C. D. and Tracey, T. J. G. 2005. The interpersonal circumplex as a model of interpersonal capabilities. Journal of Personality Assessment, 84: 137 - 147. doi: 10.1207/s15327752jpa8402_03
24. Kaiser, R. B., Lindberg, J. T. and Craig, S. B. 2007. Assessing the flexibility of managers: A comparison of methods. International Journal of Selection and Assessment, 15: 40 - 55. doi: 10.1111/j.1468-2389.2007.00366.x
25. Kline, P. 2000. The handbook of psychological testing (2nd ed.), London, UK: Routledge.
26. Kruyen, P. M., Emons, W. H. M. and Sijtsma, K. 2012. Test length and decision quality in personnel selection: When is short too short?. International Journal of Testing, 12, 321-344. doi: 10.1080/15305058.2011.643517,
27. Lawing, K., Frick, P. J. and Cruise, K. R. 2010. Differences in offending patterns between adolescent sex offenders high or low in callous-unemotional traits. Psychological Assessment, 22: 298 - 305. doi: 10.1037/a0018707
28. Levenson, H. 1973. Multidimensional locus of control in psychiatric patients. Journal of Consulting and Clinical Psychology, 41: 397 - 404. doi: 10.1037/h0035357
29. Levy, P. 1968. Short-form tests. A methodological review. Psychological Bulletin, 69: 410 - 416. doi: 10.1037/h0025736
30. Lissitz, R. W. 2009. The concept of validity: Revisions, new directions, and applications, Charlotte, NC: Information Age Publishing.
31. Lord, F. M. and Novick, M. R. 1968. Statistical theories of mental test scores, Reading, MA: Addison-Wesley.
32. Lovibond, S. H. and Lovibond, P. F. 1995. Manual for the Depression Anxiety Stress Scales (2nd ed.), Sydney, Australia: Psychology Foundation.
33. Mael, F. A. and Ashforth, B. E. 1992. Alumni and their alma mater: A partial test of the reformulated model of organizational identification. Journal of Organizational Behavior, 13: 103 - 123. doi: 10.1002/job.4030130202
34. Mason, O., Linney, Y. and Claridge, G. 2005. Short scales for measuring schizotypy. Schizophrenia Research, 78: 293 - 296. doi: 10.1016/j.schres.2005.06.020
35. McCarthy, D. M., Pedersen, S. L. and D'Amico, E. 2009. Analysis of item response and differential item functioning of alcohol expectancies in middle school youths. Psychological Assessment, 21: 444 - 449. doi: 10.1037/a0016319
36. Mellenbergh, G. J. 1996. Measurement precision in test score and item response models. Psychological Methods, 1: 293 - 299. doi: 10.1037//1082-989X.1.3.293
37. Nicewander, W. A. and Price, J. M. 1983. Reliability of measurement and the power of statistical tests: Some new result. Psychological Bulletin, 94: 52 - 533. doi: 10.1037/0033-2909.94.3.524
38. Noone, J. H., Stephens, C. and Alpass, F. 2010. The Process of Retirement Planning Scale (PRePS): Development and validation. Psychological Assessment, 22: 520 - 531. doi: 10.1037/a0019512
39. Nunnally, J. C. 1978. Psychometric theory (2nd ed.), New York, NY: McGraw-Hill.
40. Nunnally, J. C. and Bernstein, I. H. 1994. Psychometric theory (3rd ed.), New York, NY: McGraw-Hill.
41. Paap, M. C. S., Meijer, R. R., Van Bebber, J., Pedersen, G., Karterud, S., Hellem, F. M. and Haraldsen, I. R. 2011. A study of the dimensionality and measurement precision of the SCL-90-R using item response theory. International Journal of Methods in Psychiatric Research, 20: e39 - e55. doi: 10.1002/mpr.347
42. Putnam, S. P. and Rothbart, M. K. 2006. Development of short and very short forms of the Children's Behavior Questionnaire. Journal of Personality Assessment, 87: 103 - 113. doi: 10.1207/s15327752jpa8701_09
43. Reise, S. P. and Waller, N. G. 2009. Item response theory and clinical measurement. Annual Review of Clinical Psychology, 5: 27 - 48. doi: 10.1146/annurev.clinpsy.032408.153553
44. Reise, S. P., Waller, N. G. and Comrey, A. L. 2000. Factor analysis and scale revision. Psychological Assessment, 12: 287 - 297. doi: 10.1037//1040-3590.12.3.287
45. Richins, M. L. 2004. The material values scale: Measurement properties and development of a short form. Journal of Consumer Research, 31: 209 - 219. doi: 10.1086/383436
46. Saucier, G. 1994. Mini-markers: A brief version of Goldberg's Unipolar Big-Five Markers. Journal of Personality Assessment, 63: 506 - 516. doi: 10.1207/s15327752jpa6303_8
47. Scheier, M. F. and Carver, C. S. 1992. Effects of optimism on psychological and physical well-being: Theoretical overview and empirical update. Cognitive Therapy and Research, 16: 201 - 228. doi: 10.1007/BF01173489
48. Shacham, S. 1983. A shortened version of the Profile of Mood States. Journal of Personality Assessment, 47: 305 - 306. doi: 10.1207/s15327752jpa4703_14
49. Shrout, P. E. and Yager, T. J. 1989. Reliability and validity of screening scales: Effects of reducing scale length. Journal of Clinical Epidemiology, 42: 69 - 78. doi: 10.1016/0895-4356(89)90027-9
50. Sijtsma, K. 2009a. Correcting fallacies in validity, reliability, and classification. International Journal of Testing, 9: 167 - 194. doi: 10.1080/15305050903106883
51. Sijtsma, K. 2009b. On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika, 74: 107 - 120. doi: 10.1007/S11336-008-9101-0
52. Sijtsma, K. and Emons, W. H. M. 2011. Advice on total-score reliability issues in psychosomatic measurement. Journal of Psychosomatic Research, 70: 565 - 572. doi: 10.1016/j.jpsychores.2010.11.002
53. Sinharay, S., Puhan, G. and Haberman, S. 2010. Reporting diagnostic scores in educational testing: Temptations, pitfalls, and some solutions. Multivariate Behavioral Research, 45: 1 - 21. doi: 10.1080/00273171.2010.483382
54. Smith, G. T., McCarthy, D. M. and Anderson, K. 2000. On the sins of short-form development. Psychological Assessment, 12: 102 - 111. doi: 10.1037/1040-3590.12.1.102
55. Soldz, S., Budman, S., Demby, A. and Merry, J. 1995. A short form of the Inventory of Interpersonal Problems Circumples Scales. Assessment, 2: 53 - 63. doi: 10.1177/1073191195002001006
56. Stanton, J. M., Sinar, E. F., Balzer, W. K. and Smith, P. C. 2002. Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55: 167 - 194. doi: 10.1111/j.1744-6570.2002.tb00108.x
57. Strahan, R. and Gerbasi, K. C. 1972. Short, homogeneous versions of the Marlow-Crowne (sic) social desirability scale. Journal of Clinical Psychology, 28: 191 - 193. doi: 10.1037/h0047358
58. Tangirala, S. and Ramanujam, R. 2008. Employee silence on critical work issues: The cross level effects of procedural justice climate. Personnel Psychology, 61: 37 - 68. doi: 10.1111/j.1744-6570.2008.00105.x
59. Ten Berge, J. M. F. and Sočan, G. 2004. The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 69: 613 - 625. doi: 10.1007/BF0228-9858
60. Ware, J. E., Kosinski, M. and Keller, S. D. 1996. A 12-Item Short-Form Health Survey: Construction of scales and preliminary tests of reliability and validity. Medical Care, 34: 220 - 233. doi: 10.1097/00005650-199603000-00003