Prospective, head-to-head study of three computerized neurocognitive assessment tools (CNTs): Reliability and validity for the assessment of sport-related concussion

Journal of the International Neuropsychological Society

Volumn 22, Issue 1, 2015, Pages 24-37

  Nelson, Lindsay D ^a   Laroche, Ashley A ^a   Pfaller, Adam Y ^a   Lerner, E Brooke ^a   Hammeke, Thomas A ^a,b   Randolph, Christopher ^c   Barr, William B ^d   Guskiewicz, Kevin ^e   McCrea, Michael A ^a,b  

^a MEDICAL COLLEGE OF WISCONSIN   (United States)

^b CLEMENT J ZABLOCKI VETERANS AFFAIRS MEDICAL CENTER   (United States)

^c LOYOLA UNIVERSITY CHICAGO   (United States)

^d NEW YORK UNIVERSITY SCHOOL OF MEDICINE   (United States)

^e UNIVERSITY OF NORTH CAROLINA   (United States)

Author keywords

ANAM; Axon Sports; Computerized testing; Concussion; ImPACT; Sport

Indexed keywords

ADOLESCENT; ANALYSIS OF VARIANCE; BRAIN CONCUSSION; COGNITION DISORDERS; COMPLICATION; COMPUTER ASSISTED DIAGNOSIS; FEMALE; HUMAN; INJURY SCALE; MALE; NEUROPSYCHOLOGICAL TEST; PHYSIOLOGY; PROCEDURES; REACTION TIME; RECEIVER OPERATING CHARACTERISTIC; REPRODUCIBILITY; SPORT INJURY; YOUNG ADULT;

ADOLESCENT; ANALYSIS OF VARIANCE; ATHLETIC INJURIES; BRAIN CONCUSSION; COGNITION DISORDERS; DIAGNOSIS, COMPUTER-ASSISTED; FEMALE; HUMANS; MALE; NEUROPSYCHOLOGICAL TESTS; REACTION TIME; REPRODUCIBILITY OF RESULTS; ROC CURVE; TRAUMA SEVERITY INDICES; YOUNG ADULT;

EID: 84953395195     PISSN: 13556177     EISSN: 14697661     Source Type: Journal    
DOI: 10.1017/S1355617715001101     Document Type: Article

References (70)

1. Belanger, H.G., and Vanderploeg, R.D. (2005). The neuropsychological impact of sports-related concussion: A meta-analysis. Journal of the International Neuropsychological Society, 11, 345-357.
2. Benjamini, Y., and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Statistical Mehodology), 57, 289-300.
3. Benjamini, Y., and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics, 29, 1165-1188.
4. Bleiberg, J., Cernich, A.N., Cameron, K., Sun, W., Peck, K., Ecklund, P.J., ⋯ Warden, D.L. (2004). Duration of cognitive impairment after sports concussion. Neurosurgery, 54, 1073-1078.
5. Broglio, S.P., Ferrara, M.S., Macciocchi, S.N., Baumgartner, T.A., and Elliott, R. (2007). Test-retest reliability of computerized concussion assessment programs. Journal of Athletic Training, 42, 509-514.
6. Broglio, S.P., Macciocchi, S.N., and Ferrara, M.S. (2007). Sensitivity of the concussion assessment battery. Neurosurgery, 60, 1050-1057. doi:10.1227/01.NEU.0000255479.90999.C0
7. Broglio, S.P., Pontifex, M.B., O'Connor, P., and Hillman, C.H. (2009). The persistent effects of concussion on neuroelectric indices of attention. Journal of Neurotrauma, 26, 1463-1470. doi:10.1089/neu.2008-0766
8. Broglio, S.P., and Puetz, T.W. (2008). The effect of sport concussion on neurocognitive function, self-report symptoms and postural control: A meta-analysis. Sports Medicine, 38, 53-67.
9. Bruce, J., Echemendia, R., Meeuwisse, W., Comper, P., and Sisco, A. (2014). 1 year test-retest reliability of ImPACT in professional ice hockey players. The Clinical Neuropsychologist, 28, 14-25. doi:10.1080/13854046.2013.866272
10. Cernich, A., Reeves, D., Sun, W., and Bleiberg, J. (2007). Automated Neuropsychological Assessment Metrics sports medicine battery. Archives of Clinical Neuropsychology, 22(Suppl. 1), S101-S114. doi:10.1016/j.acn.2006.10.008
11. Chelune, G.J., Naugle, R.I., Lders, H., Sedlak, J., and Awad, I.A. (1993). Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7, 41-52.
12. Cole, W.R., Arrieux, J.P., Schwab, K., Ivins, B.J., Qashu, F.M., and Lewis, S.C. (2013). Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population. Archives of Clinical Neuropsychology, 28, 732-742. doi:10.1093/arclin/act040
13. Collie, A., Darby, D., and Maruff, P. (2001). Computerised cognitive assessment of athletes with sports related head injury. British Journal of Sports Medicine, 35, 297-302.
14. Collie, A., Maruff, P., Makdissi, M., McCrory, P., McStephen, M., and Darby, D. (2003). CogSport: Reliability and correlation with conventional cognitive tests used in postconcussion medical evaluations. Clinical Journal of Sport Medicine, 13, 28-32.
15. Covassin, T., Elbin, R., III, and Stiller-Ostrowski, J.L. (2009). Current sport-related concussion teaching and clinical practices of sports medicine professionals. Journal of Athletic Training, 44, 400-404. doi:10.4085/1062-6050-44.4.400
16. Covassin, T., Elbin, R.J., III, Stiller-Ostrowski, J.L., and Kontos, A.P. (2009). Immediate post-concussion assessment and cognitive testing (ImPACT) practices of sports medicine professionals. Journal of Athletic Training, 44, 639-644. doi:10.4085/1062-6050-44.6.639
17. Crawford, J.R., Garthwaite, P.H., and Gault, C.B. (2007). Estimating the percentage of the population with abnormally low scores (or abnormally large score differences) on standardized neuropsychological test batteries: A generic method with applications. Neuropsychology, 21, 419-430. doi:10.1037/0894-4105.21.4.419
18. Derogatis, L.R. (2001). Brief Symptom Inventory 18 (BSI-18): Administration, scoring, and procedures manual. Bloomington, MN: Pearson.
19. Dettwiler, A., Murugavel, M., Putukian, M., Cubon, V., Furtado, J., and Osherson, D. (2014). Persistent differences in patterns of brain activation after sports-related concussion:A longitudinal functional magnetic resonance imaging study. Journal of Neurotrauma, 31, 180-188. doi:10.1089/neu.2013.2983
20. Diener, E., Emmons, R.A., Larsen, R.J., and Griffin, S. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71-75. doi:10.1207/s15327752jpa4901-13
21. Echemendia, R.J., Iverson, G.L., McCrea, M., Macciocchi, S.N., Gioia, G.A., Putukian, M., and Comper, P. (2013). Advances in neuropsychological assessment of sport-related concussion. British Journal of Sports Medicine, 47, 294-298. doi:10.1136/bjsports-2013-092186
22. Eckner, J.T., Kutcher, J.S., and Richardson, J.K. (2011). Betweenseasons test-retest reliability of clinically measured reaction time in National Collegiate Athletic Association Division I athletes. Journal of Athletic Training, 46, 409-414.
23. Elbin, R.J., Schatz, P., and Covassin, T. (2011). One-year test-retest reliability of the online version of ImPACT in high school athletes. American Journal of Sports Medicine, 39, 2319-2324. doi:10.1177/0363546511417173
24. Green, P. (2003). Greens Medical Symptom Validity Test for Windows. Edmonton, Alberta, Canada: Greens Publishing, Inc.
25. Guskiewicz, K.M., Ross, S.E., and Marshall, S.W. (2001). Postural stability and neuropsychological deficits after concussion in collegiate athletes. Journal of Athletic Training, 36, 263-273.
26. Helmick, K., Guskiewicz, K., Barth, J., Cantu, R., Kelly, J., McDonald, E., ⋯ Warden, D. (2006). Defense and Veterans Brain Injury Center Working Group on the acute management of mild traumatic brain injury in military operational settings: Clinical practice guideline and recommendations. Washington, DC: Defense and Veteran Brain Injury Center. Retrieved from http://www.pdhealth.mil/downloads/clinical-practice-guideline- recommendations.pdf
27. Iverson, G.L., Brooks, B.L., Collins, M.W., and Lovell, M.R. (2006). Tracking neuropsychological recovery following concussion in sport. Brain Injury, 20, 245-252. doi:10.1080/02699050500487910
28. Iverson, G.L., Lovell, M.R., and Collins, M.W. (2003). Interpreting change on ImPACT following sport concussion. Clinical Neuropsychologist, 17, 460-467. doi:10.1076/clin.17.4.460.27934
29. Jacobson, N.S., and Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
30. LaRoche, A.A., Nelson, L.D., Connelly, P.K., Walter, K.D., and McCrea, M.A. (2015). Sport-related concussion reporting and state legislative effects. Clinical Journal of Sport Medicine. doi:10.1097/JSM.0000000000000192
31. Lichtenstein, J.D., Moser, R.S., and Schatz, P. (2014). Age and test setting affect the prevalence of invalid baseline scores on neurocognitive tests. American Journal of Sports Medicine, 42, 479-484. doi:10.1177/0363546513509225
32. Louey, A.G., Cromer, J.A., Schembri, A.J., Darby, D.G., Maruff, P., Makdissi, M., and McCrory, P. (2014). Detecting cognitive impairment after concussion: Sensitivity of change from baseline and normative data methods using the CogSport/Axon cognitive test battery. Archives of Clinical Neuropsychology, 29, 432-441. doi:10.1093/arclin/acu020
33. MacDonald, J., and Duerson, D. (2015). Reliability of a computerized neurocognitive test in baseline concussion testing of high school athletes. Clinical Journal of Sport Medicine, 25, 367-372. doi:10.1097/JSM.0000000000000139
34. McClincy, M.P., Lovell, M.R., Pardini, J., Collins, M.W., and Spore, M.K. (2006). Recovery from sports concussion in high school and collegiate athletes. Brain Injury, 20, 33-39. doi:10.1080/02699050500309817
35. McCrea, M., Guskiewicz, K.M., Marshall, S.W., Barr, W., Randolph, C., Cantu, R.C., and Kelly, J.P. (2003). Acute effects and recovery time following concussion in collegiate football players: The NCAA Concussion Study. Journal of the American Medical Association, 290, 2556-2563. doi:10.1001/jama.290.19.2556
36. McCrea, M., Kelly, J.P., Randolph, C., Kluge, J., Bartolic, E., Finn, G., ⋯Baxter, B. (1998). Standardized assessment of concussion (SAC): On-site mental status evaluation of the athlete. Journal of Head Trauma Rehabilitation, 13, 27-35.
37. McCrory, P., Meeuwisse, W., Aubry, M., Cantu, B., Dvorak, J., Echemendia, R.J., ⋯ Tator, C.H. (2013). Consensus statement on concussion in sport-the 4th International Conference on Concussion in Sport held in Zurich, November 2012. Clinical Journal of Sport Medicine, 23, 89-117. doi:10.1097/JSM.0b013e31828b67cf
38. Meehan, W.P., III, d'Hemecourt, P., Collins, C.L., Taylor, A.M., and Comstock, R.D. (2012). Computerized neurocognitive testing for the management of sport-related concussions. Pediatrics, 129, 38-44. doi:10.1542/peds.2011-1972
39. Mondello, S., Schmid, K., Berger, R., Kobeissy, F., Italiono, D., Jeromin, A., ⋯Buki, A. (2014). The challenge of mild traumatic brain injury: Role of biochemical markers in diagnosis of brain damage. Medical Research Reviews, 34, 503-531.
40. Moser, R.S., Iverson, G.L., Echemendia, R.J., Lovell, M.R., Schatz, P., Webbe, F.M., ⋯Barth, J.T. (2007). Neuropsychological evaluation in the diagnosis and management of sports-related concussion. Archives of Clinical Neuropsychology, 22, 909-916. doi:10.1016/j. acn.2007.09.004
41. Moser, R.S., Schatz, P., and Lichtenstein, J.D. (2015). The importance of proper administration and interpretation of neuropsychological baseline and postconcussion computerized testing. Applied Neuropsychology. Child, 4, 41-48. doi:10.1080/21622965.2013.791825
42. Moser, R.S., Schatz, P., Neidzwski, K., and Ott, S.D. (2011). Group versus individual administration affects baseline neurocognitive test performance. American Journal of Sports Medicine, 39, 2325-2330. doi:10.1177/0363546511417114
43. Nakayama, Y., Covassin, T., Schatz, P., Nogle, S., and Kovan, J. (2014). Examination of the test-retest reliability of a computerized neurocognitive test battery. American Journal of Sports Medicine, 42, 2000-2005. doi:10.1177/0363546514535901
44. Nelson, L.D. (in press). False positive rates of reliable change indices for concussion test batteries: A Monte Carlo simulation. Journal of Athletic Training.
45. Nelson, L.D., Janecek, J.K., and McCrea, M.A. (2013). Acute clinical recovery from sport-related concussion. Neuropsychology Review, 23, 285-299. doi:10.1007/s11065-013-9240-7
46. Nelson, L.D., Pfaller, A.Y., Rein, L., and McCrea, M.A. (2015). Rates and predictors of invalid baseline test performance for three computerized neurocognitive tests (CNTs): ANAM, Axon, and ImPACT. American Journal of Sports Medicine, 43, 2018-2026. doi:10.1177/0363546515587714
47. Prichep, L.S., McCrea, M., Barr, W., Powell, M., and Chabot, R.J. (2013). Time course of clinical and electrophysiological recovery after sport-related concussion. Journal of Head Trauma Rehabilitation, 28, 266-273. doi:10.1097/HTR.0b013e318247b54e
48. Rahman-Filipiak, A.A.M., and Woodward, J.L. (2014). Administration and environment considerations in computer-based sports-concussion assessment. Neuropsychology Review, 23, 314-334.
49. Randolph, C., McCrea, M., and Barr, W.B. (2005). Is neuropsychological testing useful in the management of sport-related concussion? Journal of Athletic Training, 40, 139-152.
50. Register-Mihalik, J.K., Guskiewicz, K.M., Mihalik, J.P., Schmidt, J.D., Kerr, Z.Y., and McCrea, M.A. (2012). Reliable change, sensitivity, and specificity of amultidimensional concussion assessment battery: Implications for caution in clinical practice. Journal of Head Trauma Rehabilitation. doi:10.1097/HTR.0b013e3182585d37
51. Register-Mihalik, J.K., Guskiewicz, K.M., Mihalik, J.P., Schmidt, J.D., Kerr, Z.Y., and McCrea, M.A. (2013). Reliable change, sensitivity, and specificity of a multidimensional concussion assessment battery: Implications for caution in clinical practice. Journal of Head Trauma Rehabilitation, 28, 274-283. doi:10.1097/HTR.0b013e3182585d37
52. Register-Mihalik, J.K., Kontos, D.L., Guskiewicz, K.M., Mihalik, J.P., Conder, R., and Shields, E.W. (2012). Age-related differences and reliability on computerized and paper-and-pencil neurocognitive assessment batteries. Journal of Athletic Training, 47, 297-305. doi:10.4085/1062-6050-47.3.13
53. Resch, J.E., Driscoll, A., McCaffrey, N., Brown, C., Ferrara, M.S., Macciocchi, S., ⋯ Walpert, K. (2013). ImPact test-retest reliability: Reliably unreliable? Journal of Athletic Training, 48, 506-511. doi:10.4085/1062-6050-48.3.09
54. Resch, J.E., McCrea, M.A., and Cullum, C.M. (2013). Computerized neurocognitive testing in the management of sport-related concussion: An update. Neuropsychology Review, 23, 335-349. doi:10.1007/s11065-013-9242-5
55. Rousson, V., Gasser, T., and Seifert, B. (2002). Assessing intrarater, interrater and test-retest reliability of continuous measurements. Statistics in Medicine, 21, 3431-3446. doi:10.1002/sim.1253
56. Schatz, P. (2010). Long-term test-retest reliability of baseline cognitive assessments using ImPACT. American Journal of Sports Medicine, 38, 47-53. doi:10.1177/0363546509343805
57. Schatz, P., and Ferris, C.S. (2013). One-month test-retest reliability of the ImPACT test battery. Archives of Clinical Neuropsychology, 28, 499-504. doi:10.1093/arclin/act034
58. Schatz, P., Pardini, J.E., Lovell, M.R., Collins, M.W., and Podell, K. (2006). Sensitivity and specificity of the ImPACT Test Battery for concussion in athletes. Archives of Clinical Neuropsychology, 21, 91-99. doi:10.1016/j.acn.2005.08.001
59. Schatz, P., and Sandel, N. (2013). Sensitivity and specificity of the online version of ImPACT in high school and collegiate athletes. American Journal of Sports Medicine, 41, 321-326. doi:10.1177/0363546512466038
60. Schretlen, D.J., Testa, S.M., Winicki, J.M., Pearlson, G.D., and Gordon, B. (2008). Frequency and bases of abnormal performance by healthy adults on neuropsychological testing. Journal of the International Neuropsychological Society, 14, 436-445. doi:10.1017/S1355617708080387
61. Segalowitz, S.J., Mahaney, P., Santesso, D.L., MacGregor, L., Dywan, J., and Willer, B. (2007). Retest reliability in adolescents of a computerized neuropsychological battery used to assess recovery from concussion. NeuroRehabilitation, 22, 243-251.
62. Sim, A., Terryberry-Spohr, L., and Wilson, K.R. (2008). Prolonged recovery of memory functioning after mild traumatic brain injury in adolescent athletes. Journal of Neurosurgery, 108, 511-516. doi:10.3171/JNS/2008/108/3/0511
63. Straume-Naesheim, T.M., Andersen, T.E., and Bahr, R. (2005). Reproducibility of computer based neuropsychological testing among Norwegian elite football players. British Journal of Sports Medicine, 39(Suppl. 1), i64-i69. doi:10.1136/bjsm.2005.019620
64. Thomas, D.G., Apps, J.N., Hoffmann, R.G., McCrea, M., and Hammeke, T. (2015). Benefits of strict rest after acute concussion: A randomized controlled trial. Pediatrics, 135, 213-223. doi:10.1542/peds.2014-0966
65. Van Kampen, D.A., Lovell, M.R., Pardini, J.E., Collins, M.W., and Fu, F.H. (2006). The "value added" of neurocognitive testing after sports-related concussion. American Journal of Sports Medicine, 34, 1630-1635. doi:10.1177/0363546506288677
66. Vincent, A.S., Roebuck-Spencer, T., Lopez, M.S., Twillie, D.A., Logan, B.W., Grate, S.J., ⋯ Gilliland, K. (2012). Effects of military deployment on cognitive functioning. Military Medicine, 177, 248-255.
67. Wechsler, D. (2001). Wechsler Test of Adult Reading: WTAR. San Antonio, TX: The Psychological Corporation.
68. Weir, J.P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. Journal of Strength and Conditioning Research, 19, 231-240. doi:10.1519/15184.1
69. Yuh, E.L., Hawryluk, G.W., and Manley, G.T. (2014). Imaging concussion: A review. Neurosurgery, 75(Suppl. 4), S50-S63. doi:10.1227/NEU.0000000000000491
70. Zhu, D.C., Covassin, T., Nogle, S., Doyle, S., Russell, D., Pearson, R.L., ⋯ Kaufman, D.I. (2015). A potential biomarker in sports-related concussion: Brain functional connectivity alteration of the default-mode network measured with longitudinal resting-state fMRI over thirty days. Journal of Neurotrauma, 32, 327-341. doi:10.1089/neu.2014.3413