Job analysis and the specification of content for licensure and certification examinations

Applied Measurement in Education

Volumn 14, Issue 4, 2001, Pages 369-415

  Raymond, Mark R ^a  

^a American Registry of Radiologic Technologists   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0035603998     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1207/S15324818AME1404_4     Document Type: Article

References (131)

1. American College Testing. (1992). The CCNA practice analysis and test specifications project. Park Ridge, IL: Council on Certification of Nurse Anesthetists.
2. American Board of Physical Medicine and Rehabilitation. (1998, December). The SCI medicine exam outline: A closer look. Diplomate News, 6-8. Rochester, MN: Author.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
4. American Registry of Radiologic Technologists. (1996). Job analysis and content specifications for quality management. St. Paul, MN: Author.
5. Ash, R. K. (1988). Job analysis in the world of work. In S. Gael (Ed.), The job analysis handbook for business, industry, and government, Vols. I and II (pp. 3-13). New York: Wiley.
6. Berk, R. A. (Ed.). (1986). A guide to criterion-referenced test construction. Baltimore: Johns Hopkins University Press.
7. Bloom, B. S. (Ed.). (1956). Taxonomy of educational objectives, Handbook 1: The cognitive domain. New York: McKay.
8. Cascio, W. F. (1982). Applied psychology in personnel management. Reston, VA: Reston.
9. Chai, S. H., Dimick, M. P., & Kasch, M. C. (1987). A role delineation study of hand therapy. Journal of Hand Therapy, 1, 7-16.
10. Chornick, N. L., Yocum, C. J., & Jacobsen, J. (1995). Job analysis: Newly licensed practical/vocational nurses. Chicago: National Council on Measurement in Education.
11. Cizek, G. J., Webb, L. C., & Kalohn, J. C. (1995). The use of cognitive taxonomies in licensure and certification test development: Reasonable or customary? Evaluation and the Health Professions, 18, 77-91.
12. Colliver, J. A., Markwell, S. J., Vu, N. V., & Barrows, H. S. (1990). Case specificity of standardized patient examinations. Evaluation and the Health Professions, 13, 252-261.
13. Cooper, W. H. (1981). Ubiquitous halo. Psychological Bulletin, 90, 218-244.
14. Cornelius, E. T. (1988). Practical findings from job analysis research. In S. Gael (Ed.), The job analysis handbook for business, industry, and government, Vols. I and II (pp. 48-64). New York: Wiley.
15. Cornelius, E. T., & Lyness, K. S. (1980). A comparison of holistic and decomposed judgment strategies in job analysis by job incumbents. Journal of Applied Psychology, 65, 155-163.
16. Cranny, C. J., & Doherty, M. E. (1988). Importance ratings in job analysis: Note on the misinterpretation of factor analysis. Journal of Applied Psychology, 73, 320-322.
17. Cyr, A. B. (1982). Health services administrator role delineation methodology: An institutional long-term care example. Evaluation and the Health Professions, 5, 170-199.
18. Dawes, R. M., Faust, D., & Meehl, P. (1989, March 31). Clinical versus actuarial judgment. Science, 243, 1668-1674.
19. D'Costa, A. (1986). The validity of credentialing examinations. Evaluation and the Health Professions, 9, 137-169.
20. de Jong, T., & Ferguson-Hessler, M. G. M. (1996). Types and qualities of knowledge. Educational Psychologist, 31, 105-113.
21. Dunnette, M. D. (1976). Aptitudes, abilities, and skills. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 473-520). Chicago: Rand McNally.
22. Edwards, J. C., Currie, M. L., Wade, T. P., & Kaminski, D. L. (1993). Surgery resident selection and evaluation: A critical incident study. Evaluation and the Health Professions, 16, 73-86.
23. Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1978). Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press.
24. Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, & Department of Justice. (1978). Adoption by four agencies of uniform guidelines of employee selection procedures. Federal Register, 43(166), 38290-38315.
25. Federation of State Medical Boards & National Board of Medical Examiners. (1999). USMLE Bulletin of Information. Philadelphia: Authors.
26. Fidler, J. (1988). Medical assistants: An analysis of tasks performed in practical settings. Evaluation and the Health Professions, 11, 358-378.
27. Fine, S. (1986). Job analysis. In R. A. Berk (Ed.), Performance assessment: Methods and applications (pp. 53-81). Baltimore: Johns Hopkins University Press.
28. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358.
29. Fleishman, E. A. (1975). Toward a taxonomy of human performance. American Psychologist, 30, 1127-1149.
30. Fleishman, E. A. (1982). Systems for describing human tasks. American Psychologist, 37, 821-834.
31. Fleishman, E. A., & Quaintance, M. K. (1984). Taxonomies of human performance: The description of human tasks. New York: Academic.
32. Friedman, L. (1990). Degree of redundancy between time, importance and frequency task ratings. Journal of Applied Psychology, 75, 748-752.
33. Gael, S. (1983). Job analysis: A guide to assessing work activities. San Francisco: Jossey-Bass.
34. Gael, S. (1988). The job analysis handbook for business, industry, and government, Vols. I and II. New York: Wiley.
35. Glaser, R. (1963). Instructional technology and the measurement of learning outcomes: Some questions. American Psychologist, 18, 519-521.
36. Green, S. B., & Stutzman, T. (1986). An evaluation of methods to select respondents to structured job analysis questionnaires. Personnel Psychology, 39, 543-564.
37. Hackman, J. R., & Oldham, G. R. (1975). Development of the Job Diagnostic Survey. Journal of Applied Psychology, 60, 159-170.
38. Harvey, R. J. (1991). Job analysis. In M. Dunnette & L. Hough (Eds.), Handbook of industrial and organizational psychology (2nd ed., Vol. 2, pp. 71-163). Palo Alto, CA: Consulting Psychologists Press.
39. Hubbard, J. P., Levit, E. J., Schumacher, C. F., & Schnabel, T. G. (1965). An objective evaluation of clinical competence. New England Journal of Medicine, 272, 1321-1328.
40. Hughes, G. L., & Prien, E. P. (1989). Evaluation of task and job skill linkage judgments used to develop test specifications. Personnel Psychology, 42, 283-292.
41. Iowa Intervention Project. (1993). The NIC taxonomy structure. Image, 25, 187-192.
42. Jacobs, A. M., Fivars, G., Edwards, D. S., & Fitzpatrick, R. (1978). Critical requirements for safe/effective nursing practice. Washington, DC: American Nurses' Association.
43. Jacobs, L. C., & Chase, C. I. (1992). Developing and using tests effectively. San Francisco: Jossey-Bass.
44. Joseph, M. J., & Taranath, S. N. (1999, April). Obtaining task measures from job analysis data. Papar presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec, Canada.
45. Kane, M. T. (1982). The validity of licensure examinations. American Psychologist, 37, 911-918.
46. Kane, M. T. (1994). Validating interpretive arguments for licensure and certification examinations. Evaluation and the Health Professions, 17, 133-159.
47. Kane, M. T. (1997). Model-based practice analysis and test specifications. Applied Measurement in Education, 10, 5-18.
48. Kane, M. T., Estes, C. A., Colton, D. A., & Eltoft, C. S. (1990). Role delineation for dietetic practitioners: Empirical results. Journal of the American Dietetic Association, 90, 1124-1133.
49. Kane, M. T., Kingsbury, C., Colton, D., & Estes, C. (1986). A study of nursing practice and role delineation and job analysis of entry-level performance of registered nurses. Chicago: National Council of State Boards of Nursing.
50. Kane, M. T., Kingsbury, C., Colton, D., & Estes, C. (1989). Combining data on criticality and frequency in developing plans for licensure and certification examinations. Journal of Educational Measurement, 26, 17-27.
51. Kane, M. T., Miller, T., Trine, M., Becker, C., & Carson, K. (1995). The precision of practice analysis results in the professions. Evaluation and the Health Professions, 18, 29-50.
52. Klein, L. W., & Jarjoura, D. (1985). The importance of content representation for common-item equating with nonrandom groups. Journal of Educational Measurement, 22, 197-206.
53. Knapp, J., & Knapp, L. (1995). Practice analysis: Building the foundation for validity. In J. C. Impara (Ed.), Licensure testing: Purposes, procedures, and practices (pp. 93-116). Lincoln, NE: Buros Institute of Mental Measurements.
54. Kuehn, P. A., Stallings, W. M., & Holland, C. L. (1990). Court-defined job analysis requirements for validation of teacher certification tests. Educational Measurement: Issues & Practice, 9(4), 21-24.
55. Kunen, S., Cohen, R., & Solman, R. (1981). A levels of processing analysis of Bloom's taxonomy. Journal of Educational Psychology, 73, 202-211.
56. LaDuca, A. (1980). The structure of competence in the health professions. Evaluation and the Health Professions, 3, 253-288.
57. LaDuca, A. (1994). Validation of professional licensure examinations: Professions theory, test design, and construct validity. Evaluation and the Health Professions, 17, 178-197.
58. LaDuca, A., Downing, S., & Henzel, T. (1995). Test development: Systematic item writing and test construction. In J. C. Impara (Ed.), Licensure testing: Purposes, procedures, and practices (pp. 117-148). Lincoln, NE: Buros Institute of Mental Measurements.
59. LaDuca, A., Engle, J. D., & Risley, M. E. (1978). Progress toward development of a general model for competence definition in the health professions. Journal of Allied Health, 7, 149-155.
60. LaDuca, A., Taylor, D. D., & Hill, I. K. (1984). The design of a new physician licensure examination. Evaluation and the Health Professions, 7, 115-140.
61. Landy, F. J. (1988). Selection procedure development and usage. In S. Gael (Ed.), The job analysis handbook for business, industry, and government, Vols. I and II (pp. 271-287). New York: Wiley.
62. Landy, F. J., & Vasey, J. (1991). Job analysis: The composition of SME samples. Personnel Psychology, 44, 27-50.
63. Levine, E. L., Ash, R. A., & Bennett, N. (1980). Exploratory comparative study of four job analysis methods. Journal of Applied Psychology, 65, 524-535.
64. Levine, E. L., Thomas, J. N., & Sistrunk, F. (1988). Selecting a job analysis approach. In S. Gael (Ed.), The job analysis handbookfor business, industry, and government, Vols. I and II (pp. 339-352). New York: Wiley.
65. Levine, H. G., McGuire, C., Miller, G. E., & Larson, C. (1968). The orthopaedic training study: Final report. Chicago: University of Illinois, Center for the Study of Medical Education.
66. Lindell, M. K., Clause, C. S., Brandt, C. J., & Landis, R. S. (1998). Relationship between organizational context and job analysis task ratings, Journal of Applied Psychology, 83, 769-776.
67. Lunz, M. E., Stahl, J. A., & James, K. (1989). Content validity revisited: Transforming job analysis data into test specifications. Evaluation and the Health Professions, 12, 192-206.
68. McCloskey, J. C., & Bulachek, G. M. (Eds.). (1992). Nursing interventions classification. St. Louis, MO: Mosby.
69. McCormick, E. J. (1976). Job and task analysis. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 651-696). Chicago: Rand McNally.
70. McCormick, E. J., Jeanneret, P. R., & Mecham, R. C. (1972). A study of job characteristics and job dimensions as based on the Position Analysis Questionnaire. Journal of Applied Psychology, 56, 347-368.
71. McGaghie, W. C. (1980). The evaluation of competence: Validity issues in the health professions. Evaluation and the Health Professions, 3, 289-320.
72. McGaghie, W. C. (1991). Professional competence evaluation. Educational Researcher, 20(1), 3-9.
73. Mehrens, W. A., & Popham, W. J. (1992). How to evaluate the legal defensibility of high-stakes tests. Applied Measurement in Education, 5, 265-283.
74. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York: American Council on Education and Macmillan.
75. Miller, G. E. (1968). The orthopaedic training study. Journal of the American Medical Association, 206, 601-606.
76. Millman, J., & Greene, J. (1989). The specification and development of tests of achievement and ability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 335-366). New York: Macmillan.
77. Morgeson, F. P., & Campion, M. A. (1997). Social and cognitive sources of potential inaccuracy in job analysis. Journal of Applied Psychology, 82, 627-655.
78. National Commission of Health Certifying Agencies. (1981). Task force report on education and certification. Washington, DC: Author.
79. National Conference of Examiners for Engineering and Surveying. (1999a). Fundamentals of engineering (FE) examinations: Morning session specifications. Clemson, SC: Author.
80. National Conference of Examiners for Engineering and Surveying. (1999b). PE examination formats: Principles and practice of engineering examinations. Clemson, SC: Author.
81. National Council of Examiners for Engineering and Surveying. (1989). Analysis of professional activities and requirements of the engineering profession. Clemson, SC: Author.
82. Nelson, D. (1994). Job analysis for licensure and certification exams: Science or politics. Educational Measurement: Issues & Practice, 13(4), 29-35.
83. Newman, L. S., Slaughter, R. C., & Taranath, S. N. (1999, April). The selection and use of rating scales in task surveys: A review of current job analysis practice. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec, Canada.
84. Nitko, A. J. (1980). Distinguishing the many varieties of criterion-referenced tests. Review of Educational Research, 50, 461-485.
85. Osterlind, S. J. (1998). Constructing test items: Multiple choice, constructed response, performance, and other formats. Boston: Kluwer Academic.
86. Pearlman, K. (1980). Job families: A review and discussion of their implications for personnel selection. Psychological Bulletin, 87, 1-28.
87. Perkins, D. N., & Salomon, G. (1989). Are cognitive skills context-bound? Educational Researcher, 18(1), 16-25.
88. Popham, W. J. (1984). Specifying the domain of content or behaviors. In R. A. Berk (Ed.), A guide to criterion-referenced test construction (pp. 29-48). Baltimore: Johns Hopkins University Press.
89. Popham, W. J., & Husek, T. R. (1969). Implications of criterion-referenced measurement. Journal of Educational Measurement, 6, 1-9.
90. Prekeges, J. L., Sawyer, N. S., & Wells, P. C. (1999). Components of preparedness for nuclear medicine technologists. Journal of Nuclear Medicine Technology, 27, 237-245.
91. Primoff, E. J., & Fine, S. A. (1988). A history of job analysis. In S. Gael (Ed.), The job analysis handbook for business, industry, and government, Vols. I and II (pp. 14-29). New York: Wiley.
92. Raymond, M. R. (1989). Applications of multidimensional scaling to research in the health professions. Evaluation and the Health Professions, 12, 379-408.
93. Raymond, M. R. (1996). Establishing weights for test plans for licensure and certification examinations. Applied Measurement in Education, 9, 237-256.
94. Raymond, M. R., Buckwalter, J. A., & Steyers, C. M. (1989, March). Investigating the structure of orthopedic proactive using multidimensional scaling analysis. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
95. Rodgers, J. L. (1988). Structural models of the American Psychological Association in 1986: A taxonomy for reorganization. American Psychologist, 43, 372-382.
96. Rosenfeld, M., Shimberg, B., & Thornton, R. F. (1983). Job analysis of licensed psychologists in the United States and Canada. Princeton, NJ: Educational Testing Service.
97. Rosenfeld, M., Slater, S., & Goldsmith, S. (2000, April). Defining the scope of a new job and the curriculum requirements for competent entry-level practice: An application of job analysis methodology. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
98. Rowland, V. V., & Whaley, K. C. (1995). Core curriculum assessmentfor the radiologic and therapeutic sciences [Survey]. Philadelphia: Thomas Jefferson University, Department of Diagnostic Imaging.
99. Sanchez, J. I., & Fraser, S. L. (1992). On the choice of scales for task analysis. Journal of Applied Psychology, 77, 545-553.
100. Sanchez, J. I., & Levine, E. L. (1989). Determining important tasks within jobs: A policy-capturing approach. Journal of Applied Psychology, 74, 336-342.
101. Schaefer, L., Raymond, M. R., & White, A. S. (1992). A comparison of two methods for structuring performance domains. Applied Measurement in Education, 5, 321-335.
102. Schiffman, S. S., Reynolds, M. L., & Young, F. W. (1981). Introduction to multidimensional scaling: Theory, methods, and applications. New York: Academic.
103. Schmeiser, C. B. (1987, April). Effect of translating task analysis into test specifications. Paper presented at the annual meeting of the National Council on Measurement in Education, Washington, DC.
104. Schmeiser, C. B. (1988, April). Effects of statistical and judgmental weightings of task analysis data on test specifications. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans, LA.
105. Schmeiser, C. B., & Estes, C. A. (1986, April). Translating task analysis results into test specifications: Clarity or confusion? Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco.
106. Schmidt, G. R., & Sulzer, J. L. (1989, April). Defining the criticality of job content: A survey of job analysis procedures. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Boston.
107. Seddon, G. M. (1978). The properties of Bloom's taxonomy of educational objectives for the cognitive domain. Review of Educational Research, 48, 302-323.
108. Shimberg, B. (1981). Testing for licensure and certification. American Psychologist, 36, 1138-1146.
109. Shimberg, B. (1990). Social considerations in the validation of licensing and certification examinations. Educational Measurement: Issues and Practice, 9(4), 11-14.
110. Sireci, S. G., & Geisinger, K. F. (1992). Analyzing test content using cluster analysis and multidimensional scaling. Applied Psychological Measurement, 16, 17-31.
111. Sireci, S. G., & Geisinger, K. F. (1995). Using subject-matter experts to assess content representation: An MDS analysis. Applied Psychological Measurement, 19, 241-255.
112. Slaughter, R. C., & Newman, L. S. (1999, April). Developing test specifications from job analysis data. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec, Canada.
113. Smith, I. L., & Hambleton, R. K. (1990). Content validity studies of licensing examinations. Educational Measurement: Issues & Practice, 9(4), 7-10.
114. Smith, J. E., & Hakel, M. D. (1979). Convergence among data sources, response bias, and reliability and validity of a structured job analysis questionnaire. Personnel Psychology, 32, 677-692.
115. Society for Industrial and Organizational Psychology. (1987). Principles for the validation and use of personnel selection procedures (3rd ed.). College Park, MD: Author.
116. Sokal, R. (1974, September 27). Classification: Purpose, principles, progress. Science, 185, 1115-1123.
117. Spence, I. (1983). Monte Carlo simulation studies. Applied Psychological Measurement, 7, 405-425.
118. Spray, J. A., & Huang, C. (2000). Obtaining test blueprints weights from job analysis surveys. Journal of Educational Measurement, 37, 187-201.
119. Stocking, M. L., Swanson, L., & Pearlman, M. (1993). Application of item selection method to real data. Applied Psychological Measurement, 17, 167-176.
120. Tannenbaum, R. J., & Rosenfeld, M. (1994). Job analysis for teacher competency testing: Identification of basic skills important for all entry-level teachers. Educational and Psychological Measurement, 54, 199-211.
121. Tannenbaum, R. J., & Wesley, S. (1993). Agreement between committee-based and field-based job analyses: A study in the context of licensure testing. Journal of Applied Psychology, 78, 975-980.
122. Task Force on the Core Content for Emergency Medicine Revision. (1997). Core content for emergency medicine. Annals of Emergency Medicine, 29, 791-811.
123. Thomas, L., & Kalohn, J. J. (1996, April). Weighting tasks from job analysis to derive test specifications for licensure examinations: Some methodological and statistical considerations. Paper presented at the annual meeting of the American Educational Research Association, New York.
124. Thompson, D. E., & Thompson, T. A. (1982). Court standards for job analysis in test validation. Personnel Psychology, 35, 865-874.
125. Wang, N., Wiser, R. F., & Joseph, M. (1999, April). Examining the reliability and validity of job analysis survey data. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Quebec, Canada.
126. Wells, P., & Leahy, D. (1998). NMTCB 1997 task analysis report. Journal of Nuclear Medicine Technology, 26, 52-56.
127. Wilson, M. A., Harvey, R. J., & Macy, B. A. (1990). Repeating items to estimate the test-retest reliability of task inventory ratings. Journal of Applied Psychology, 75, 158-163.
128. Wightman, L. (1998). Practical issues in computerized test assembly. Applied Psychological Measurement, 22, 292-302.
129. Wilson-Pessano, S. (1988, April). Defining professional competence: The critical incident technique 40 years later. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
130. Yocum, C. J., Cayia, B. A., & Jacobsen, J. (1991). Job analysis of nurse aides employed in nursing999 homes, home health care agencies, and hospitals. Chicago: National Council on Measurement in Education.
131. Zaglaniczny, K. L. (1993). Council on certification professional practice analysis. Journal of the American Association of Nurse Anesthetists, 61, 241-255.