The comparative effect of subjective and objective after-action reviews on team performance on a complex task

Journal of Applied Psychology

Volumn 98, Issue 3, 2013, Pages 514-528

  Villado, Anton J ^a   Arthur, Winfred ^b  

^a RICE UNIVERSITY   (United States)

^b TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Adaptive performance; After action review; Complex skill acquisition; Team training; Training

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; COMPARATIVE STUDY; FEMALE; GROUP PROCESS; HUMAN; INTERPERSONAL COMMUNICATION; MALE; RESEARCH; STANDARD; TASK PERFORMANCE; YOUNG ADULT;

ADOLESCENT; ADULT; COMMUNICATION; FEMALE; GROUP PROCESSES; HUMANS; MALE; RESEARCH REPORT; TASK PERFORMANCE AND ANALYSIS; YOUNG ADULT;

EID: 84877828205     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0031510     Document Type: Article

References (57)

1. Alexander, L. T., Kepner, C. H., & Tregoe, B. B. (1962). The effectiveness of knowledge of results in a system-training program. Journal of Applied Psychology, 46, 202-211. doi:10.1037/h0042151
2. Arthur, W., Jr., Bell, S. T., & Edwards, B. D. (2007). A longitudinal examination of the comparative criterion-related validity of additive and referent-shift consensus operationalizations of team efficacy. Organizational Research Methods, 10, 35-58. doi:10.1177/1094428106287574
3. Arthur, W., Jr., Bennett, W., Jr., Stanush, P. L., & McNelly, T. L. (1998). Factors that influence skill decay and retention: A quantitative analysis. Human Performance, 11, 57-101. doi:10.1207/s15327043hup1101_3
4. Arthur, W., Jr., & Day, E. A. (2013). Introduction: Knowledge and skill decay in applied research. In W. Arthur, Jr., E. A. Day, W. Bennett, Jr., & A. Portrey (Eds.), Individual and team skill decay: The science and implications for practice (pp. 3-13). New York, NY: Taylor & Francis/ Psychology Press.
5. Arthur, W., Jr., Edwards, B. D., Bell, S. T., & Bennett, W., Jr. (2002). Identifying factors that influence the cognitive readiness and performance of teams: The development and evaluation of a high fidelity synthetic team task performance environment (STTPE) lab (Grant/ contract No. F41624-97-C-5000). Mesa, AZ: L-3 Communications Corporation, Link Simulation and Training Division.
6. Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs, NJ: Prentice Hall.
7. Bandura, A., & Cervone, D. (1983). Self-evaluative and self-efficacy mechanisms governing motivational effects of goal systems. Journal of Personality and Social Psychology, 45, 1017-1028. doi:10.1037/0022-3514.45.5.1017
8. Barry, B., & Stewart, G. L. (1997). Composition, process, and performance in self-managed groups: The role of personality. Journal of Applied Psychology, 82, 62-78. doi:10.1037/0021-9010.82.1.62
9. Beal, D. J., Cohen, R. R., Burke, M. J., & McLendon, C. L. (2003). Cohesion and performance in groups: A meta-analytic clarification of construct relations. Journal of Applied Psychology, 88, 989-1004. doi: 10.1037/0021-9010.88.6.989
10. Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, 232-245. doi:10.1037/0021-9010.71.2.232
11. Byrne, A. J., Sellen, A. J., Jones, J. G., Aitkenhead, A. R., Hussain, S., Gilder, F., Ribes, P. (2002). Effect of videotape feedback on anaesthetists' performance while managing simulated anaesthetic crises: A multicentre study. Anaesthesia, 57, 176-179. doi:10.1046/j.1365-2044.2002.02361.x
12. Custers, E. J. F. M., Regehr, G., McCulloch, W., Peniston, C., & Reznick, R. (1999). The effects of modeling on learning a simple surgical procedure: See one, do one or see many, do one? Advances in Health Sciences Education, 4, 123-143. doi:10.1023/A:1009763210212
13. DeNisi, A. S., Robbins, T., & Cafferty, T. P. (1989). Organization of information used for performance appraisals: Role of diary-keeping. Journal of Applied Psychology, 74, 124-129. doi:10.1037/0021-9010.74.1.124
14. Dunlap, W. P., Cortina, J. M., Vaslow, J. B., & Burke, M. J. (1996). Meta-analysis of experiments with matched groups of repeated measures designs. Psychological Methods, 1, 170-177. doi:10.1037/1082-989X.1.2.170
15. Dyer, J. L., Wampler, R. L., & Blankenbeckler, P. N. (2005). After action reviews with the ground soldier system (Research Rep. No. 1840). Arlington, VA: U. S. Army Research Institute for the Behavioral and Social Sciences.
16. Ellis, S., & Davidi, I. (2005). After-event reviews: Drawing lessons from successful and failed experience. Journal of Applied Psychology, 90, 857-871. doi:10.1037/0021-9010.90.5.857
17. Ellis, S., Ganzach, Y., Castle, E., & Sekely, G. (2010). The effect of filmed versus personal after-event reviews on task performance: The mediating and moderating role of self-efficacy. Journal of Applied Psychology, 95, 122-131. doi:10.1037/a0017867
18. Ellis, S., Mendel, R., & Nir, M. (2006). Learning from successful and failed experiences: The moderating role of kind of after-event review. Journal of Applied Psychology, 91, 669-680. doi:10.1037/0021-9010.91.3.669
19. eSim Games. (2007). Steel Beasts Pro PE ver. 2.370 [Computer software]. Mountain View, CA: Author.
20. Goldstein, I. L., Emanuel, J. T., & Howell, W. C. (1968). Effect of percentage and specificity of feedback on choice behavior in a probabilistic information-processing task. Journal of Applied Psychology, 52, 163-168. doi:10.1037/h0025525
21. Goldstein, I. L., & Ford, J. K. (2002). Training in organizations: Needs assessment, development and evaluation (4th ed.). Belmont, CA: Wadsworth.
22. Goodman, J. S., & Wood, R. E. (2004). Feedback specificity, learning opportunities, and learning. Journal of Applied Psychology, 89, 809-821. doi:10.1037/0021-9010.89.5.809
23. Guzzo, R. A., Jette, R. D., & Katzell, R. A. (1985). The effects of psychologically based intervention programs on worker productivity: A meta-analysis. Personnel Psychology, 38, 275-291. doi:10.1111/j.1744-6570.1985.tb00547.x
24. Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology, 64, 349-371. doi:10.1037/0021-9010.64.4.349
25. James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85-98. doi:10.1037/0021-9010.69.1.85
26. Kluger, A. N., & DeNisi, A. (1996). Effects of feedback intervention on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254-284. doi:10.1037/0033-2909.119.2.254
27. Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999). A revised index of interrater agreement for multi-item ratings of a single target. AppliedPsychological Measurement, 23, 127-135. doi:10.1177/ 01466219922031257
28. Locke, E. A. (1968). Toward a theory of work motivation and incentives: A conceptual analysis and integrative model. Organizational Behavior and Human Performance, 3, 157-189. doi:10.1016/0030-5073(68)90004-4
29. Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall.
30. Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57, 705-717. doi:10.1037/0003-066X.57.9.705
31. Meliza, L. L. (1996). Standardizing after action review systems (Research Rep. No. 1702). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
32. Meliza, L. L. (1998). A guide to standardizing after action review (AAR) aids (Research Product No. 99-01). Arlington, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
33. Meliza, L. L., Bessemer, D. W., & Hiller, J. H. (1994). Providing unit training feedback in the distributed interactive simulation environment. In R. F. Holz, J. H. Hiller, & H. H. McFann (Ed.), Determinants of effective unit performance: Research on measuring and managing unit training readiness (pp. 257-280). Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.
34. Mento, A. J., Steel, R. P., & Karen, R. J. (1987). A meta-analytic study of goal setting on task performance: 1966-1984. Organizational Behavior and Human Decision Processes, 39, 52-83. doi:10.1016/0749-5978(87)90045-8
35. Mesmer-Magnus, J. R., & DeChurch, L. A. (2009). Information sharing and team performance: A meta-analysis. Journal of Applied Psychology, 94, 535-546. doi:10.1037/a0013773
36. Middendorf, C. H., & Macan, T. H. (2002). Note-taking in the employment interview: Effects on recall and judgments. Journal of Applied Psychology, 87, 293-303. doi:10.1037/0021-9010.87.2.293
37. Mullen, B., & Copper, C. (1994). The relation between group cohesiveness and performance: An integration. Psychological Bulletin, 115, 210-227. doi:10.1037/0033-2909.115.2.210
38. Murphy, K. R. (1991). Criterion issues in performance appraisal research: Behavioral accuracy versus classification accuracy. Organizational Behavior and Human Decision Processes, 50, 45-50.
39. Murphy, K. R., & Balzer, W. K. (1986). Systematic distortions in memorybased ratings and performance evaluations: Consequences for rating accuracy. Journal of Applied Psychology, 71, 39-44. doi:10.1037/0021-9010.71.1.39
40. Neubert, M. J. (1998). The value of feedback and goal setting over goal setting alone and potential moderators of this effect: A meta-analysis. Human Performance, 11, 321-335. doi:10.1207/s15327043hup1104_2
41. Prince, C., Salas, E., Brannick, M., & Orasnau, J. (2005). Beyond facilitation: An improved CRM debrief for safety training. Human Factors and Aerospace Safety, 5, 1-22.
42. Rosenfeld, L. B., & Gilbert, J. R. (1989). The measurement of cohesion and its relationship to dimensions of self-disclosure in classroom settings. Small Group Behavior, 20, 291-301. doi:10.1177/ 104649648902000301
43. Ryan, A. M., Daum, D., Bauman, T., Grisez, M., Mattimore, K., Nalodka, T., & McCormick, S. (1995). Direct, indirect, and controlled observation and rating accuracy. Journal of Applied Psychology, 80, 664-670. doi:10.1037/0021-9010.80.6.664
44. Salas, E., Cooke, N. J., & Rosen, M. A. (2008). On teams, teamwork, and team performance: Discoveries and developments. Human Factors, 50, 540-547. doi:10.1518/001872008X288457
45. Sanchez, J. I., & De La Torre, P. (1996). A second look at the relationship between rating and behavioral accuracy in performance appraisal. Journal of Applied Psychology, 81, 3-10. doi:10.1037/0021-9010.81.1.3
46. Savoldelli, G. L., Naik, V. N., Park, J., Joo, H. S., Chow, R., & Hamstra, S. J. (2006). Value of debriefing during simulated crisis management: Oral versus video-assisted feedback. Anesthesiology, 105, 279-285. doi:10.1097/00000542-200608000-00010
47. Schmidt, R. A., & Bjork, R. A. (1992). New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. Psychological Science, 3, 207-217.
48. Shebilske, W. L., Jordan, J. A., Goettl, B. P., & Paulus, L. E. (1998). Observation versus hands-on practice of complex skills in dyadic, triadic, and tetradic training-teams. Human Factors, 40, 525-540. doi: 10.1518/001872098779649319
49. Smith-Jentsch, K. A., Canon-Bowers, J. A., Tannenbaum, S. I., & Salas, E. (2008). Guided team self-correction: Impacts on team mental models, processes, and effectiveness. Small Group Research, 39, 303-327. doi: 10.1177/1046496408317794
50. Sturman, M. C., Cheramie, R. A., & Cashen, L. H. (2005). The impact of job complexity and performance measurement on the temporal consistency, stability, and test-retest reliability of employee job performance ratings. Journal of Applied Psychology, 90, 269-283. doi:10.1037/0021-9010.90.2.269
51. Taylor, P. J., Russ-Eft, D. F., & Chan, D. W. L. (2005). A meta-analytic review of behavior modeling training. Journal of Applied Psychology, 90, 692-709. doi:10.1037/0021-9010.90.4.692
52. Tubbs, M. E. (1986). Goal setting: A meta-analytic examination of the empirical evidence. Journal of Applied Psychology, 71, 474-483. doi: 10.1037/0021-9010.71.3.474
53. United States Army. (1993). A leaders guide to after-action reviews (Training Circular 25-20). Washington, DC: Author.
54. Wentling, T. L. (1973). Mastery versus nonmastery instructions with varying test item feedback treatments. Journal of Educational Psychology, 65, 50-58. doi:10.1037/h0034820
55. Williams, K. J., DeNisi, A. S., Meglino, B. M., & Cafferty, T. P. (1986). Initial decisions and subsequent performance ratings. Journal of Applied Psychology, 71, 189-195. doi:10.1037/0021-9010.71.2.189
56. Woehr, D. J., & Feldman, J. (1993). Processing objective and question order effects on the causal relation between memory and judgment in performance appraisal: The tip of the iceberg. Journal of Applied Psychology, 78, 232-241. doi:10.1037/0021-9010.78.2.232
57. Zakay, D., Ellis, S., & Shevaliski, M. (2004). Outcomes' values and early warning indications as determinates of willingness to learn from experience. Experimental Psychology, 51, 150-157. doi:10.1027/1618-3169.51.2.150