Reliability and validity of the NeuroCognitive Performance Test, A web-based neuropsychological assessment

Frontiers in Psychology

Volumn 6, Issue NOV, 2015, Pages

  Morrison, Glenn E ^a   Simone, Christa M ^a   Ng, Nicole F ^a   Hardy, Joseph L ^b  

^a Lumos Labs ^*  (United States)

^b Lumos Labs Inc   (United States)

Author keywords

Concurrent validity; Fluid reasoning; Memory; Neuropsychological assessment; Normative data; Psychomotor speed; Reliability; Web based

Indexed keywords

EID: 84949670107     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.01652     Document Type: Article

References (52)

1. American Educational Research Association. (1999). Standards for Educational and Psychological Testing. Washington, DC: AERA.
2. Anastasi, A., and Urbina, S. (1997). Psychological Testing, 7th Edn. Upper Saddle River, NJ: Prentice Hall.
3. Army Individual Test Battery. (1944). Army Individual Test Battery: Manual of directions and scoring. Washington, DC: War Department, Adjutant General's Office.
4. Baddeley, A. D. (1968). A 3 min reasoning test based on grammatical transformation. Psychon. Sci. 10, 341-342. doi: 10.3758/BF03331551.
5. Bauer, R. M., Iverson, G. L., Cernich, A. N., Binder, L. M., Ruff, R. M., and Naugle, R. I. (2012). Computerized neuropsychological assessment devices: joint position paper of the american academy of clinical neuropsychology and the national academy of neuropsychology. Arch. Clin. Neuropsychol. 27, 362-373. doi: 10.1093/arclin/acs027.
6. Bird, C. M., Papadopoulou, K., Ricciardelli, P., Rossor, M. N., and Cipolotti, L. (2003). Test-retest reliability, practice effects and reliable change indices for the recognition memory test. Br. J. Clin. Psychol. 42(Pt 4), 407-425. doi: 10.1348/014466503322528946.
7. Burke, H. R. (2010). Raven's progressive matrices: validity, reliability, and norms. J. Psychol. 82, 253-257. doi: 10.1080/00223980.1972.9923815.
8. Capitani, E., Barbarotto, R., and Laicana, M. (1996). Does education influence age-related cognitive decline? A further inquiry. Dev. Neuropsychol. 12, 231-240. doi: 10.1080/87565649609540648.
9. Cattell, R. B. (1987). Intelligence: Its Structure, Growth and Action: Its Structure, Growth and Action. Amsterdam: Elsevier Science Publishers B.V.
10. Collie, A., Maruff, P., Darby, D. G., and McStephen, M. (2003). The effects of practice on the cognitive test performance of neurologically normal individuals assessed at brief test-retest intervals. J. Int. Neuropsychol. Soc. 9, 419-428. doi: 10.1017/S1355617703930074.
11. Deloche, G., Seron, X., Larroque, C., Magnien, C., Metz-Lutz, M. N., Noel, M. N., et al. (1994). Calculation and number processing: assessment battery; role of demographic factors. J. Clin. Exp. Neuropsychol. 16, 195-208. doi: 10.1080/01688639408402631.
12. Dikmen, S. S., Heaton, R. K., GRANT, I., and Temkin, N. R. (1999). Test-retest reliability and practice effects of expanded Halstead-Reitan Neuropsychological Test Battery. J. Int. Neuropsychol. Soc. 5, 346-356. doi: 10.1017/S1355617799544056.
13. Feldstein, S. N., Keller, F. R., Portman, R. E., Durham, R. L., Klebe, K. J., and Davis, H. P. (1999). A comparison of computerized and standard versions of the Wisconsin Card Sorting Test. Clin. Neuropsychol. 13, 303-313. doi: 10.1076/clin.13.3.303.1744.
14. Ford, J. K., MacCallum, R. C., and Tail, M. (1986). The application of exploratory factor analysis in applied psychology: a critical review and analysis. Pers. Psychol. 39, 291-314. doi: 10.1111/j.1744-6570.1986.tb00583.x.
15. Gualtieri, C., and Johnson, L. (2006). Reliability and validity of a computerized neurocognitive test battery, CNS Vital Signs. Arch. Clin. Neuropsychol. 21, 623-643. doi: 10.1016/j.acn.2006.05.007.
16. Gulliksen, H. (1950). Effect of Group Heterogeneity on Test Reliability. Theory of Mental Tests. Hoboken: John Wiley & Sons Inc. 108-127.
17. Gur, R. C., Richard, J., Hughett, P., Calkins, M. E., Macy, L., Bilker, W. B., et al. (2010). A cognitive neuroscience-based computerized battery for efficient measurement of individual differences: standardization and initial construct validation. J. Neurosci. Methods 187, 254-262. doi: 10.1016/j.jneumeth.2009.11.017.
18. Hardy, J. L., Nelson, R. A., Thomason, M. E., Sternberg, D. A., Katovich, K., Farzin, F., et al. (2015). Enhancing cognitive abilities with comprehensive training: a large, online, randomized, active-controlled trial. PLoS ONE 10:e0134467. doi: 10.1371/journal.pone.0134467.
19. Hu, L., and Bentler, P. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equat. Model. Multidis. J. 6, 1-55. doi: 10.1080/10705519909540118.
20. Kane, R. L., and Kay, G. G. (1992). Computerized assessment in neuropsychology: a review of tests and test batteries. Neuropsychol. Rev. 3, 1-117. doi: 10.1007/BF01108787.
21. Keefe, R. S., Goldberg, T. E., Harvey, P. D., Gold, J. M., Poe, M. P., and Coughenour, L. (2004). The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery. Schizophr. Res. 68, 283-297. doi: 10.1016/j.schres.2003.09.011.
22. Kueider, A. M., Parisi, J. M., Gross, A. L., and Rebok, G. W. (2012). Computerized cognitive training with older adults: a systematic review. PLoS ONE 7:e40588. doi: 10.1371/journal.pone.0040588.
23. Lampit, A., Hallock, H., and Valenzuela, M. (2014). Computerized cognitive training in cognitively healthy older adults: a systematic review and meta-analysis of effect modifiers. PLoS Med. 11:e1001756. doi: 10.1371/journal.pmed.1001756.
24. Lemay, S., Bédard, M. A., and Rouleau, I. (2004). Practice effect and test-retest reliability of attentional and executive tests in middle-aged to elderly subjects. Clin. Neuropsychol. 18, 1-19. doi: 10.1080/13854040490501718.
25. Manly, J. (2008). "Race, culture, education, and cognitive test performance among older adults," in Handbook of Cognitive Aging: Interdisciplinary Perspectives, eds S. Hofer and D. Alwin (Thousand Oaks, CA: SAGE Publications, Inc), 398-418.
26. Maruff, P., Collie, A., Darby, D., Weaver-Cargin, J., Masters, C., and Currie, J. (2004). Subtle memory decline over 12 months in mild cognitive impairment. Dement. Geriatr. Cogn. Disord. 18, 342-348. doi: 10.1159/000080229.
27. McCrae, R. R., Kurtz, J. E., Yamagata, S., and Terracciano, A. (2011). Internal consistency, retest reliability, and their implications for personality scale validity. Pers. Soc. Psychol. Rev. 15, 28-50. doi: 10.1177/1088868310366253.
28. Milner, B. (1971). Interhemispheric differences in the localization of psychological processes in man. Br. Med. Bull. 27, 272-277.
29. Nakayama, Y., Covassin, T., Schatz, P., Nogle, S., and Kovan, J. (2014). Examination of the Test-Retest Reliability of a Computerized Neurocognitive Test Battery. Am. J. Sports Med. 42, 2000-2005. doi: 10.1177/0363546514535901.
30. Nuechterlein, K. H., Green, M. F., Kern, R. S., Baade, L. E., Barch, D. M., Cohen, J. D., et al. (2008). The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. Am. J. Psychiatry 165, 203-213. doi: 10.1176/appi.ajp.2007.07010042.
31. Prabhakaran, V., Rypma, B., and Gabrieli, J. D. E. (2001). Neural substrates of mathematical reasoning: a functional magnetic resonance imaging study of neocortical activation during performance of the necessary arithmetic operations test. Neuropsychology 15, 115-127. doi: 10.1037/0894-4105.15.1.115.
32. Prabhakaran, V., Smith, J. A., Desmond, J. E., Glover, G. H., and Gabrieli, J. D. (1997). Neural substrates of fluid reasoning: an fMRI study of neocortical activation during performance of the Raven's Progressive Matrices Test. Cogn. Psychol. 33, 43-63.
33. Rabbitt, P., Diggle, P., Smith, D., Holland, F., and Mc Innes, L. (2001). Identifying and separating the effects of practice and of cognitive ageing during a large longitudinal study of elderly community residents. Neuropsychologia 39, 532-543. doi: 10.1016/S0028-3932(00)00099-3.
34. Raven, J. (2000). The Raven's progressive matrices: change and stability over culture and time. Cogn Psychol. 41, 1-48. doi: 10.1006/cogp.1999.0735.
35. R Core Team (2015). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing. Available online at: http://www.R-project.org/.
36. Reitan, R. M., and Wolfson, D. (1995). The Halstead-Reitan Neuropsycholgical Test Battery: Therapy and Clinical Interpretation. Tucson, AZ: Neuropsychological Press.
37. Revelle, W. (2015). Psych: Procedures for Personality and Psychological Research. Evanston, IL: Northwestern University. Available online at: http://CRAN.R-project.org/package=psychVersion=1.5.8.
38. Richardson, J. T. E. (2007). Measures of short-term memory: a historical review. Cortex 43, 635-650. doi: 10.1016/S0010-9452(08)70493-3.
39. Robbins, T. W., James, L. F., Owen, A. M., Sahakian, B. J., Lawrence, A. D., McInnes, L., et al. (1998). A study of performance on tests from the CANTAB battery sensitive to frontal lobe dysfunction in a large sample of normal volunteers: implications for theories of executive functioning and cognitive aging. Cambridge Neuropsychological Test Automated Battery. J. Int. Neuropsychol. Soc. 4, 474-490. doi: 10.1017/s1355617798455073.
40. Rossetti, H. C., Lacritz, L. H., Cullum, C. M., and Weiner, M. F. (2011). Normative data for the Montreal Cognitive Assessment (MoCA) in a population-based sample. Neurology 77, 1272-1275. doi: 10.1212/WNL.0b013e318230208a.
41. Royer, F. L. (1971). Spatial orientational and figural information in free recall of visual figures. J. Exp. Psychol. 91, 326-332. doi: 10.1037/h0031846.
42. Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychol. Rev. 103, 403-428. doi: 10.1037/0033-295X.103.3.403.
43. Schlegel, R., and Gilliland, K. (2007). Development and quality assurance of computer-based assessment batteries. Arch. Clin. Neuropsychol. 22, 49-61. doi: 10.1016/j.acn.2006.10.005.
44. Sekhon, J. S. (2011). Multivariate and propensity score matching software with automated balance optimization: the matching package for R. J. Statist. Softw. 42, 1-52.
45. Wechsler, D. (1945). Wechsler Memory Scale. San Antonio, TX: Psychological Corporation.
46. Wechsler, D. (1955). Manual for the Wechsler Adult Intelligence Scale. Oxford, England: Psychological Corp. 110.
47. Wechsler, D. (1981). Wechsler Adult Intelligence Scale-Revised. San Antonio, TX: Psychological Corporation.
48. Wild, K., Howieson, D., Webbe, F., Seelye, A., and Kaye, J. (2008). Status of computerized cognitive testing in aging: a systematic review. Alzheim. Demen. 4, 428-437. doi: 10.1016/j.jalz.2008.07.003.
49. Williams, J. E., and McCord, D. M. (2006). Equivalence of standard and computerized versions of the Raven Progressive Matrices Test. Comput. Hum. Behav. 22, 791-800. doi: 10.1016/j.chb.2004.03.005.
50. Williams, L. M., Simms, E., Clark, C. R., Paul, R. H., Rowe, D., and Gordon, E. (2005). The test-retest reliability of a standardized neurocognitive and neurophysiological test battery: "neuromarker". Int. J. Neurosci. 115, 1605-1630. doi: 10.1080/00207450590958475.
51. Wood, S. N. (2011). Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. J. R. Statist. Soc. B 73, 3-36. doi: 10.1111/j.1467-9868.2010.00749.x.
52. Zygouris, S., and Tsolaki, M. (2014). Computerized cognitive testing for older adults: a review. Am. J. Alzheimers. Dis. Other Demen. 30, 13-28. doi: 10.1177/1533317514522852.