Testing the rules of justice: The effects of frame-of-reference and pre-test validity information on personality test responses and test perceptions

International Journal of Selection and Assessment

Volumn 13, Issue 1, 2005, Pages 75-86

  Holtz, Brian C ^a   Ployhart, Robert E ^b   Dominguez, Alexandria ^c  

^a GEORGE MASON UNIVERSITY   (United States)

^b UNIVERSITY OF SOUTH CAROLINA   (United States)

^c UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 15544378988     PISSN: 0965075X     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.0965-075X.2005.00301.x     Document Type: Article

References (55)

1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Anastasi, A. and Urbina, S. (1997) Psychological testing. New Jersey: Prentice-Hall.
3. Arvey, R.D. and Sackett, P.R. (1993) Fairness in selection: Current developments and perspectives. In N. Schmitt and W.C. Borman (Eds.), Personnel selection in organizations (pp. 171-202). San Francisco: Jossey-Bass.
4. Baggozzi, R.P. and Edwards, J.R. (1998) A general approach for representing constructs in organizational research. Organizational Research Methods, 1, 45-87.
5. Baggozzi, R.P. and Heatherton, T.F. (1994) A general approach to representing multifaceted personality constructs: Application to state self-esteem. Structural Equation Modeling, 1, 35-67.
6. Baron, R.M. and Kenny, D.A. (1986) The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Applied Psychology, 71, 1173-1182.
7. Barrick, M.R. and Mount, M.K. (1991) The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1-26.
8. Bauer, T.N., Maertz, C.P., Dolen, M.R. and Campion, M.A. (1998) Longitudinal assessment of applicant reactions to employment testing and test outcome feedback. Journal of Applied Psychology, 83, 892-903.
9. Bing, M.N., Whanger, J.C., Davison, H.K. and VanHook, J.B. (2004) Incremental validity of the frame-of-reference effect in personality scale scores: A replication and extension. Journal of Applied Psychology, 89, 150-157.
10. Brown, F.G. (1983) Principles of educational and psychological testing (3rd Edn.). New York: Holt, Rinehart, & Winston.
11. Browne, M.W. and Cudeck, R. (1993) Alternative ways of assessing model fit. In K.A. Bollen and J.S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
12. Byrne, B.M., Shavelson, R.J. and Muthen, B. (1989) Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
13. Chan, D. (1997) Racial subgroup differences in predictive validity perceptions on personality and cognitive ability tests. Journal of Applied Psychology, 82, 311-320.
14. Chan, D. and Schmitt, N. (1997) Video-based versus paper-and-pencil method of assessment in situational judgement tests: Subgroup differences in test performance and face validity perceptions. Journal of Applied Psychology, 82(1), 143-159.
15. Chan, D., Schmitt, N., DeShon, R.P., Clause, C.S. and Delbridge, K. (1997) Reactions to cognitive ability tests: The relationships between race, test performance, face validity perceptions, and test-taking motivation. Journal of Applied Psychology, 82, 300-310.
16. Cohen, J. (1988) Statistical power analysis for the behavioral sciences (2nd Edn.). Hillsdale, NJ: Lawrance Erlbaum Associates.
17. Costa, P.T. Jr. and McCrae, R.R. (1992) NEO-PI-R professional manual. Odessa, FL: Psychological Assessment Resources, Inc.
18. Cronbach, L.J. (1970) Essentials of psychological testing. New York: Harper & Row.
19. Elkins, T.J. and Phillips, J.S. (2000) Job context, selection decision outcome, and the perceived fairness of selection tests: Biodata as an illustrative case. Journal of Applied Psychology, 85(3), 479-484.
20. Fink, A.M. and Butcher, J.N. (1972) Reducing objections to personality inventories with special instructions. Educational and Psychological Measurement, 32, 631-639.
21. Gilliland, S.W. (1993) The perceived fairness of selection systems: An organizational justice perspective. Academy of Management Review, 18(4), 694-734.
22. Gilliland, S.W. (1994) Effects of procedural and distributive justice on reactions to a selection system. Journal of Applied Psychology, 79(5), 691-701.
23. Harland, L.K., Rauzi, T. and Biasotto, M.M. (1995) Perceived fairness of personality tests and the impact of explanations on their use. Employee Responsibilities and Rights Journal, 8, 183-192.
24. Hogan, R.T. (1991) Personality and personality measurement. In M.D. Dunnette and L.M. Hough (Eds.), Hanbook of industrial and organizational psychology (Vol. 2, pp. 873-919). Palo Alto, CA: Consulting Psychologists Press.
25. Horvath, M., Ryan, A.M. and Stierwalt, S. (2000) Explanations for selection test use, outcome favorability and self-efficacy: What influences test-taker perceptions? Organizational Behavior and Human Decision Processes, 83, 310-330.
26. Hough, L.M. (1998) Personality at work: Issues and evidence. In M.D. Hakel (Ed.), Beyond multiple choice: Evaluating alternatives to traditional testing for selection (pp. 131-159). Hillsdale, NJ: Erlbaum.
27. Hough, L.M. and Oswald, F.L. (2000) Personnel selection: Looking toward the future - remembering the past. Annual Review of Psychology, 51, 631-664.
28. Hu, L.T. and Bender, P.M. (1995) Evaluating model fit. In R.H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76-99). Thousand Oaks, CA: Sage.
29. Hunthausen, J.M., Truxillo, D.M., Bauer, T.N. and Hammer, L.B. (2003) A field study of frame-of-reference effects on personality test validity. Journal of Applied Psychology, 88, 545-551.
30. Jones, J.W. (1991) Assessing privacy invasiveness of psychological test items: Job relevant versus clinical measures of integrity. Journal of Business and Psychology, 5(4), 531-535.
31. Joreskog, K. and Sorbom, D. (1993) LISREL 8: Structural equation modeling with the SIMPLIS command language. Hillsdale, NJ: Lawrence Erlbaum and Associates.
32. Kluger, A.N. and Rothstein, H.R. (1993) The influence of selection test type on applicant reactions to employment testing. Journal of Business and Psychology, 8, 3-25.
33. Macan, T.H., Avedon, M.J., Paese, M. and Smith, D.E. (1994) The effects of applicants' reactions to cognitive ability tests and an assessment center. Personnel Psychology, 47, 715-738.
34. Mischel, W. and Shoda, Y. (1995) A cognitive-affective system theory of personality: Reconceptualizing situations, dispositions, dynamics, and invariance in personality structure. Psychological Review, 102(2), 246-268.
35. Mone, M.A., Mueller, G.C. and Mauland, W. (1996) The perceptions and usage of statistical power in applied psychology and management research. Personnel Psychology, 49, 103-120.
36. Mount, M.K. and Barrick, M.R. (1995) The big five personality dimensions: Implications for research and practice in human resources management. Research in Personnel and Human Resource Management, 13, 153-200.
37. Nevo, B. (1985) Face validity revisited. Journal of Educational Measurement, 22, 287-293.
38. Ployhart, R.E. and Harold, C.M. (2004) The applicant attribution-reaction theory (AART): An integrative theory of applicant attributional processing. International Journal of Selection and Assessment, 12, 84-98.
39. Ployhart, R.E. and Ryan, A.M. (1998) Applicants' reactions to the fairness of selection procedures: The effects of positive rule violations and time of measurement. Journal of Applied Psychology, 83, 3-16.
40. Ployhart, R.E., Ryan, A.M. and Bennett, M. (1999) Explanations for selection decisions: Applicants' reactions to informational and sensitivity features of explanations. Journal of Applied Psychology, 84, 87-106.
41. Ployhart, R.E., Ziegert, J.C. and McFarland, L.A. (2003) Understanding racial differences on cognitive ability tests in selection contexts: An integration of stereotype threat and applicant reactions research. Human Performance, 16, 231-259.
42. Robie, C., Schmit, M.J., Ryan, A.M. and Zickar, M.J. (2000) Effects of item context specificity on the measurement equivalence of a personality inventory. Organizational Research Methods, 3, 348-365.
43. Ryan, A.M. (2001) Explaining the black-white test score gap: The role of test perceptions. Human Performance, 14, 45-75.
44. Ryan, A.M. and Ployhart, R.E. (2000) Applicants' perceptions of selection procedures and decisions: A critical review and agenda for the future. Journal of Management, 26, 565-606.
45. Rynes, S.L. (1993) Who's selecting whom? Effects of selection practices on applicant attitudes and behaviors. In N. Schmitt and W. Borman (Eds.), Personnel selection in organizations (pp. 240-274). San Francisco: Jossey-Bass.
46. Rynes, S.L. and Connerley, M.L. (1993) Applicant reactions to alternative selection procedures. Journal of Business and Psychology, 7, 261-277.
47. Schmit, M.J. and Ryan, A.M. (1993) The big five in personnel selection: Factor structure in applicant and nonapplicant populations. Journal of Applied Psychology, 78, 966-974.
48. Schmit, M.J., Ryan, A.M., Stierwalt, S.L. and Powell, A.B. (1995) Frame-of-Reference effects on personality scale scores and criterion-related validity. Journal of Applied Psychology, 80, 607-620.
49. Schmitt, N., Clause, C.S. and Pulakos, E.D. (1996) Subgroup differences associated with different measures of some common job-relevant constructs. International review of industrial and organizational psychology (Vol. 11, pp. 115-139). New York: John Wiley & Sons Ltd.
50. Smith, D.B., Hanges, P.J. and Dickson, M.W. (2001) Personnel selection and the five-factor model: Reexamining the effects of applicant's frame of reference. Journal of Applied Psychology, 86, 304-315.
51. Smither, J.W., Reilly, R.R., Pearlman, K. and Stoffey, R.W. (1993) Applicant reactions to selection procedures. Personnel Psychology, 46, 49-76.
52. Stone-Romero, E.F., Alliger, G.M. and Aguinis, H. (1994) Type II error problems in the use of moderated multiple regression for the detection of moderating effects of dichotomous variables. Journal of Management, 20, 167-178.
53. Tracy, R.L. and Fiske, D.W. (1974) Does information and personality scales affect responses and attitudes? Educational and Psychological Measurement, 34, 617-629.
54. Truxillo, D.M., Bauer, T.N., Campion, M.A. and Paronto, M.E. (2002) Selection fairness information and applicant reactions: A longitudinal field study. Journal of Applied Psychology, 87(6), 1020-1031.
55. Vandenberg, R.J. and Lance, C.E. (2000) A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.