Reliability and Validity of Ambulatory Cognitive Assessments

Assessment

Volumn 25, Issue 1, 2018, Pages 14-30

  Sliwinski, Martin J ^a   Mogle, Jacqueline A ^a   Hyun, Jinshil ^a   Munoz, Elizabeth ^a   Smyth, Joshua M ^a   Lipton, Richard B ^b  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

^b ALBERT EINSTEIN COLLEGE OF MEDICINE   (United States)

Author keywords

ambulatory assessment; cognition; ecological validity; smartphone

Indexed keywords

ADULT; AGED; AGING; CLINICAL PSYCHOLOGY; COGNITION; DEVICES; ECOLOGY; FEMALE; HUMAN; MALE; MEMORY; MENTAL STRESS; MIDDLE AGED; NEUROPSYCHOLOGICAL TEST; NEW YORK; PROSPECTIVE STUDY; REACTION TIME; RECALL; REPRODUCIBILITY; SMARTPHONE; STANDARDS; TASK PERFORMANCE; VALIDATION STUDY;

ADULT; AGED; AGING; COGNITION; ECOLOGY; FEMALE; HUMANS; MALE; MEMORY; MENTAL RECALL; MIDDLE AGED; NEUROPSYCHOLOGICAL TESTS; NEW YORK CITY; PROSPECTIVE STUDIES; PSYCHOLOGY, CLINICAL; REACTION TIME; REPRODUCIBILITY OF RESULTS; SMARTPHONE; STRESS, PSYCHOLOGICAL; TASK PERFORMANCE AND ANALYSIS;

EID: 85035046781     PISSN: 10731911     EISSN: 15523489     Source Type: Journal    
DOI: 10.1177/1073191116643164     Document Type: Article

References (53)

1. Ackerman P. L., Beier M. E., (2007). Further explorations of perceptual speed abilities in the context of assessment methods, cognitive abilities, and individual differences during skill acquisition. Journal of Experimental Psychology: Applied, 13, 249-272. doi:10.1037/1076-898X.13.4.249
2. Allard M., Husky M., Catheline G., Pelletier A., Dilharreguy B., Amieva H... Swendsen J., (2014). Mobile technologies in the early detection of cognitive decline. PLoS ONE, 9(12), e112197. doi:10.1371/journal.pone.0112197
3. Alloway T. P., Alloway R. G., (2010). Investigating the predictive roles of working memory and IQ in academic attainment. Journal of Experimental Child Psychology, 106, 20-29. doi:10.1016/j.jecp.2009.11.003
4. Brose A., Lövdén M., Schmiedek F., (2014). Daily fluctuations in positive affect positively co-vary with working memory performance. Emotion, 14, 1-6. doi:10.1037/a0035210
5. Brose A., Schmiedek F., Lövdén M., Lindenberger U., (2012). Daily variability in working memory is coupled with negative affect: The role of attention and motivation. Emotion, 12, 605-617. doi:10.1037/a0024436
6. Brunswik E., (1943). Organismic achievement and environmental probability. Psychological Review, 50, 255-272. doi:10.1037/h0060889
7. Charles S. T., Piazza J. R., Mogle J., Sliwinski M. J., Almeida D. M., (2013). The wear and tear of daily stressors on mental health. Psychological Science, 24, 733-741. doi:10.1177/0956797612462222
8. Conway A. R. A., Cowan N., Bunting M. F., Therriault D. J., Minkoff S. R. B., (2002). A latent variable analysis of working memory capacity, short-term memory capacity, processing speed, and general fluid intelligence. Intelligence, 30, 163-183. doi:10.1016/S0160-2896(01)00096-4
9. Cranford J. A., Shrout P. E., Iida M., Rafaeli E., Yip T., Bolger N., (2006). A procedure for evaluating sensitivity to within-person change: Can mood measures in diary studies detect change reliably? Personality & Social Psychology Bulletin, 32, 917-929. doi:10.1177/0146167206287721
10. Dirk J., Schmiedek F., (2015). Fluctuations in elementary school children’s working memory performance in the school context. Journal of Educational Psychology. Advance online publication. doi:10.1037/edu0000076
11. Engle R. W., Tuholski S. W., Laughlin J. E., Conway A., (1999). Working memory, short-term memory, and general fluid intelligence: A latent-variable approach. Journal of Experimental Psychology: General, 128, 309-331. doi:10.1037/0096-3445.128.3.309
12. Fahrenberg J., Myrtek M., Pawlik K., Perrez M., (2007). Ambulatory assessment—Monitoring behavior in daily life settings: A behavioral-scientific challenge for psychology. European Journal of Psychological Assessment, 23, 206-213. doi:10.1027/1015-5759.23.4.206
13. Heron K. E., Smyth J. M., (2010). Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. British Journal of Health Psychology, 15(Pt. 1), 1-39. doi:10.1348/135910709X466063
14. Hultsch D. F., MacDonald S. W., Hunter M. A., Levy-Bencheton J., Strauss E., (2000). Intraindividual variability in cognitive performance in older adults: Comparison of adults with mild dementia, adults with arthritis, and healthy adults. Neuropsychology, 14, 588-598.
15. Just M. A., Carpenter P. A., (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122-149.
16. MacDonald S. W. S., Li S.-C., Bäckman L., (2009). Neural underpinnings of within-person variability in cognitive functioning. Psychology and Aging, 24, 792-808. doi:10.1037/a0017798
17. Miyake A., Friedman N. P., Rettinger D. A., Shah P., Hegarty M., (2001). How are visuospatial working memory, executive functioning, and spatial abilities related? A latent-variable analysis. Journal of Experimental Psychology: General, 130, 621-640.
18. Nesselroade J. R., (1991). The warp and the woof of the developmental fabric. In Downs R. M., Liben L. S., Palermo D. S., (Eds.), Visions of aesthetics, the environment & development: The legacy of Joachim F. Wohlwill (pp. 213-240). Hillsdale, NJ: Lawrence Erlbaum.
19. Owsley C., Ball K., McGwin G., Sloane M. E., Roenker D. L., White M. F., Overley E. T., (1998). Visual processing impairment and risk of motor vehicle crash among older adults. Journal of the American Medical Association, 279, 1083-1088.
20. Ram N., Gerstorf D., (2009). Time-structured and net intraindividual variability: Tools for examining the development of dynamic characteristics and processes. Psychology and Aging, 24, 778-791. doi:10.1037/a0017915
21. Raven J., (2000). The Raven’s Progressive Matrices: Change and stability over culture and time. Cognitive Psychology, 41, 1-48. doi:10.1006/cogp.1999.0735
22. Raykov T., Marcoulides G. A., (2006). On multilevel model reliability estimation from the perspective of structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 13, 130-141. doi:10.1207/s15328007sem1301_7
23. Riediger M., Wrzus C., Klipker K., Müller V., Schmiedek F., Wagner G. G., (2014). Outside of the laboratory: Associations of working-memory performance with psychological and physiological arousal vary with age. Psychology and Aging, 29, 103-114. doi:10.1037/a0035766
24. Rucklidge J. J., Tannock R., (2002). Neuropsychological profiles of adolescents with ADHD: Effects of reading difficulties and gender. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 43, 988-1003.
25. Salthouse T. A., (2000). Aging and measures of processing speed. Biological Psychology, 54, 35-54.
26. Schmader T., Johns M., Forbes C., (2008). An integrated process model of stereotype threat effects on performance. Psychological Review, 115, 336-356. doi:10.1037/0033-295X.115.2.336
27. Schmiedek F., Li S.-C., Lindenberger U., (2009). Interference and facilitation in spatial working memory: Age-associated differences in lure effects in the n-back paradigm. Psychology and Aging, 24, 203-210. doi:10.1037/a0014685
28. Schmiedek F., Lövdén M., Lindenberger U., (2013). Keeping it steady older adults perform more consistently on cognitive tasks than younger adults. Psychological Science, 24, 1747-1754. doi:10.1177/0956797613479611
29. Schmiedek F., Lövdén M., Lindenberger U., (2014). A task is a task is a task: Putting complex span, n-back, and other working memory indicators in psychometric context. Frontiers in Psychology, 5, 1475. doi:10.3389/fpsyg.2014.01475
30. Schneider-Garces N. J., Gordon B. A., Brumback-Peltz C. R., Shin E., Lee Y., Sutton B. P... Fabiani M., (2010). Span, CRUNCH, and beyond: Working memory capacity and the aging brain. Journal of Cognitive Neuroscience, 22, 655-669. doi:10.1162/jocn.2009.21230
31. Scott S. B., Graham-Engeland J. E., Engeland C. G., Smyth J. M., Almeida D. M., Katz M. J... Sliwinski M. J., (2015). The Effects of Stress on Cognitive Aging, Physiology and Emotion (ESCAPE) Project. BMC Psychiatry, 15, 146. doi:10.1186/s12888-015-0497-7
32. Shiffman S., Patten C., Gwaltney C., Paty J., Gnys M., Kassel J... Balabanis M., (2006). Natural history of nicotine withdrawal. Addiction (Abingdon, England), 101, 1822-1832. doi:10.1111/j.1360-0443.2006.01635.x
33. Shiffman S., Paty J. A., Gnys M., Kassel J. D., Elash C., (1995). Nicotine withdrawal in chippers and regular smokers: Subjective and cognitive effects. Health Psychology, 14, 301-309. doi:10.1037/0278-6133.14.4.301
34. Shiffman S., Stone A. A., Hufford M. R., (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1-32. doi:10.1146/annurev.clinpsy.3.022806.091415
35. Siedlecki K. L., (2007). Investigating the structure and age invariance of episodic memory across the adult lifespan. Psychology and Aging, 22, 251-268. doi:10.1037/0882-7974.22.2.251
36. Sliwinski M. J., (2008). Measurement-burst designs for social health research. Social & Personality Psychology Compass, 2, 245-261.
37. Sliwinski M. J., (2011). Approaches to modeling intraindividual and interindividual facets of change for developmental research. In Fingerman K. L., Berg C. A., Smith J., Antonucci T. C., (Eds.), Handbook of life-span development (pp. 1-25). New York, NY: Springer.
38. Sliwinski M. J., Almeida D. M., Smyth J., Stawski R. S., (2009). Intraindividual change and variability in daily stress processes: Findings from two measurement-burst diary studies. Psychology and Aging, 24, 828-840. doi:10.1037/a0017925
39. Sliwinski M. J., Smyth J. M., Hofer S. M., Stawski R. S., (2006). Intraindividual coupling of daily stress and cognition. Psychology and Aging, 21, 545-557. doi:10.1037/0882-7974.21.3.545
40. Smyth J. M., Stone A. A., (2003). Ecological momentary assessment research in behavioral medicine. Journal of Happiness Studies, 4, 35-52. doi:10.1023/A:1023657221954
41. Spooner D., Pachana N., (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology, 21, 327-337. doi:10.1016/j.acn.2006.04.004
42. Stenneken P., Egetemeir J., Schulte-Körne G., Müller H. J., Schneider W. X., Finke K., (2011). Slow perceptual processing at the core of developmental dyslexia: A parameter-based assessment of visual attention. Neuropsychologia, 49, 3454-3465. doi:10.1016/j.neuropsychologia.2011.08.021
43. Strauss B., (2002). Social facilitation in motor tasks: A review of research and theory. Psychology of Sport and Exercise, 3, 237-256. doi:10.1016/S1469-0292(01)00019-X
44. Timmers C., Maeghs A., Vestjens M., Bonnemayer C., Hamers H., Blokland A., (2014). Ambulant cognitive assessment using a smartphone. Applied Neuropsychology: Adult, 21, 136-142. doi:10.1080/09084282.2013.778261
45. Tiplady B., Oshinowo B., Thomson J., Drummond G. B., (2009). Alcohol and cognitive function: Assessment in everyday life and laboratory settings using mobile phones. Alcoholism, Clinical and Experimental Research, 33, 2094-2102. doi:10.1111/j.1530-0277.2009.01049.x
46. Unsworth N., Engle R. W., (2007). The nature of individual differences in working memory capacity: Active maintenance in primary memory and controlled search from secondary memory. Psychological Review, 114, 104-132. doi:10.1037/0033-295X.114.1.104
47. Unsworth N., Spillers G. J., (2010). Working memory capacity: Attention control, secondary memory, or both? A direct test of the dual-component model. Journal of Memory and Language, 62, 392-406. doi:10.1016/j.jml.2010.02.001
48. Verhaeghen P., Salthouse T. A., (1997). Meta-analyses of age–cognition relations in adulthood: Estimates of linear and nonlinear age effects and structural models. Psychological Bulletin, 122, 231-249. doi:10.1037/0033-2909.122.3.231
49. Waters A. J., Li Y., (2008). Evaluating the utility of administering a reaction time task in an ecological momentary assessment study. Psychopharmacology, 197, 25-35. doi:10.1007/s00213-007-1006-6
50. Waters A. J., Marhe R., Franken I. H. A., (2012). Attentional bias to drug cues is elevated before and during temptations to use heroin and cocaine. Psychopharmacology, 219, 909-921. doi:10.1007/s00213-011-2424-z
51. Waters G. S., Caplan D., (2003). The reliability and stability of verbal working memory measures. Behavior Research Methods, Instruments, & Computers, 35, 550-564. doi:10.3758/BF03195534
52. Wild-Wall N., Falkenstein M., Gajewski P. D., (2011). Age-related differences in working memory performance in a 2-back task. Frontiers in Psychology, 2, 186. doi:10.3389/fpsyg.2011.00186
53. Yoon M., Millsap R. E., (2007). Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling: A Multidisciplinary Journal, 14, 435-463. doi:10.1080/10705510701301677