The Questionable Ecological Validity of Ecological Momentary Assessment: Considerations for Design and Analysis

Research in Human Development

Volumn 14, Issue 3, 2017, Pages 253-270

  Ram, Nilam ^a,b   Brinberg, Miriam ^a   Pincus, Aaron L ^a   Conroy, David E ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

^b DIW BERLIN   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85028443310     PISSN: 15427609     EISSN: 15427617     Source Type: Journal    
DOI: 10.1080/15427609.2017.1340052     Document Type: Article

References (58)

1. Abbott, A., (1995). Sequence analysis:New methods for old ideas. Annual Review of Sociology, 21, 93–113. doi:10.1146/annurev.so.21.080195.000521
2. Almeida, D. M., (2005). Resilience and vulnerability to daily stressors assessed via diary methods. Current Directions in Psychological Science, 14, 64–68. doi:10.1111/j.0963-7214.2005.00336.x
3. Baltes, P. B., & Nesselroade, J. R., (1979). History and rationale of longitudinal research. In J. R., Nesselroade & P. B., Baltes (Eds.), Longitudinal research in the study of behavior and development (pp. 1–39). New York, NY:Academic Press.
4. Barta, W. D., Tennen, H., & Litt, M. D., (2012). Measurement reactivity in diary research. In M. R., Mehl, & T. S., Conner (Eds.), Handbook of research methods for studying daily life (pp. 108–123). New York, NY:Guilford Press.
5. Bolger, N., Davis, A., & Rafaeli, E., (2003). Diary methods:Capturing life as it is lived. Annual Review of Psychology, 54, 579–616. doi:10.1146/annurev.psych.54.101601.145030
6. Bolger, N., & Laurenceau, J. P., (2013). Intensive longitudinal methods:An introduction to diary and experience sampling research. New York, NY:Guilford Press.
7. Brinberg, M., Ram, N., Hülür, G., Brick, T. R., & Gerstorf, D., (in press). Analyzing dyadic data using grid-sequence analysis:Inter-dyad differences in intra-dyad dynamics. Journal of Gerontology:Psychological Sciences.
8. Bronfenbrenner, U., (1977). Toward an experimental ecology of human development. American Psychologist, 32, 513–531. doi:10.1037/0003-066X.32.7.513
9. Brunswik, E., (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62, 193–217. doi:10.1037/h0047470
10. Campbell, D. T., & Stanley, J. C., (1963). Experimental and quasi-experimental designs for research. Chicago, IL:RandMcNally.
11. Corbin, J., & Strauss, A., (2008). Basics of qualitative research:Techniques and procedures for developing grounded theory (3rd ed.). Thousand Oaks, CA:Sage.
12. Csikszentmihalyi, M., & Larson, R., (1984). Being adolescent:Conflict and growth in the teenage years. New York, NY:Basic Books.
13. Elzinga, C. H., & Liefbroer, A. C., (2007). De-standardization of family-life trajectories of young adults:A cross-national comparison using sequence analysis. European Journal of Population, 23, 225–250. doi:10.1007/s10680-007-9133-7
14. Fiske, D., (1971). Measuring the concept of personality. Chicago, IL:Aldine.
15. Gabadinho, A., Ritschard, G., Muller, N. S., & Studer, M., (2011). Analyzing and visualizing state sequences in R with TraMineR. Journal of Statistical Software, 40(4), 1–37. doi:10.18637/jss.v040.i04
16. Hamaker, E. L., (2012). Why researchers should think “within-person”:A paradigmatic rationale. In M. R., Mehl & T. S., Conner (Eds.), Handbook of research methods for studying daily life (pp. 43–61). New York, NY:Guilford Press.
17. Harari, G. M., Lane, N. D., Wang, R., Crosier, B. S., Campbell, A. T., & Gosling, S. D., (2016). Using smartphones to collect behavioral data in psychological Science:Opportunities, practical considerations, and challenges. Perspectives on Psychological Science, 11, 838–854. doi:10.1177/1745691616650285
18. Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M., (2007). Epistemological foundations for the measurement of experience. In J. M., Hektner, J. A., Schmidt, & M., Csikszentmihalyi (Eds.), Experience sampling method:Measuring the quality of everyday life (pp. 3–15). Thousand Oaks, CA:Sage.
19. Hormuth, S. E., (1986). The sampling of experiences in situ. Journal of Personality, 54, 262–293. doi:10.1111/jopy.1986.54.issue-1
20. Hovsepian, K., Al’absi, M., Ertin, E., Kamarck, T., Nakajima, M., & Kumar, S., (2015, September). cStress:Towards a gold standard for continuous stress assessment in the mobile environment. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing (pp. 493–504). New York, NY:ACM.
21. Hufford, M. R., & Shields, A. L., (2002). Electronic diaries:An examination of applications and what works in the field. Applied Clinical Trials, 11, 46–56.
22. Hufford, M. R., (2007). Special methodological challenges and opportunities in ecological momentary assessment. In Stone, S. Shiffman, A. A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture:Self-reports in health research (pp. 54–75). Oxford University Press.
23. Kearns, K., Edwards, R., & Tingstrom, D. H., (1990). Accuracy of long momentary time-sampling intervals:Implications for classroom data collection. Journal of Psychoeducational Assessment, 8, 74–85. doi:10.1177/073428299000800109
24. Kim, B.-R., & Teti, D. M., (2014). Maternal emotional availability during infant bedtime:An ecological framework. Journal of Family Psychology, 28, 1–11. doi:10.1037/a0035157
25. Koffer, R. E., Ram, N., Conroy, D. E., Pincus, A. L., & Almeida, D., (2016). Stressor diversity:Introduction and empirical integration into the daily stress model. Psychology and Aging, 31, 301–320. doi:10.1037/pag0000095
26. Lu, H., Frauendorfer, D., Rabbi, M., Mast, M. S., Chittaranjan, G. T., Campbell, A. T., … Choudhury, T., (2012). Stresssense:Detecting stress in unconstrained acoustic environments using smartphones. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing (pp. 351–360). New York, NY:ACM.
27. Mark, G., Gonzalez, V. M., & Harris, J., (2005, April). No task left behind?:Examining the nature of fragmented work. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 321–330). Portland, OR:ACM.
28. McLachlan, G. J., & Peel, D., (2000). Finite mixture models. New York, NY:Wiley.
29. Meany‐Daboul, M. G., Roscoe, E. M., Bourret, J. C., & Ahearn, W. H., (2007). A comparison of momentary time sampling and partial‐interval recording for evaluating functional relations. Journal of Applied Behavior Analysis, 40, 501–514. doi:10.1901/jaba.2007.40-501
30. Mehl, M. R., (2012). Naturalistic observation sampling:The electronically activated recorder (EAR). In M. R., Mehl & T. S., Conner (Eds.), Handbook of research methods for studying daily life (pp. 176–192). New York, NY:Guilford Press.
31. Mehl, M. R., & Conner, T. S., (Eds.). (2012). Handbook of research methods for studying daily life. New York, NY:Guilford Press.
32. Molenaar, P. C., (2004). A manifesto on psychology as idiographic science:Bringing the person back into scientific psychology, this time forever. Measurement, 2, 201–218.
33. Needleman, S. B., & Wunsch, C. D., (1970). A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 48, 443–453. doi:10.1016/0022-2836(70)90057-4
34. Nesselroade, J. R., & Molenaar, P. C., (2016). Some behavioral science measurement concerns and proposals. Multivariate Behavioral Research, 51, 396–412. doi:10.1080/00273171.2015.1050481
35. Newcomb, M. E., Swann, G., Estabrook, R., Corden, M., Begale, M., Ashbeck, A., … Mustanski, B., (in press). Patterns and predictors of compliance in a prospective diary study of substance use and sexual behavior in a sample of young men who have sex with men. Assessment.
36. Nyquist, H., (2002). Certain topics in telegraphic transmission theory. Proceedings of the IEEE, 86, 280–305. (Reprinted from Transactions of the AIEE, pp. 617–644, 1928)
37. Powell, J., Martindale, A., & Kulp, S., (1975). An evaluation of time-sample measures of behavior. Journal of Applied Behavior Analysis, 8, 463–469. doi:10.1901/jaba.1975.8-463
38. Powell, J., Martindale, B., Kulp, S., Martindale, A., & Bauman, R., (1977). Taking a closer look:Time sampling and measurement error. Journal of Applied Behavior Analysis, 10, 325–332. doi:10.1901/jaba.1977.10-325
39. Radley, K. C., O’Handley, R. D., & Labrot, Z. C., (2015). A comparison of momentary time sampling and partial-interval recording for assessment of effects of social skills training. Psychology in the Schools, 52, 363–378. doi:10.1002/pits.21829
40. Ram, N., Conroy, D., Pincus, A. L., Lorek, A., Rebar, A. H., Roche, M. J., & Gerstorf, D., (2014). Examining the interplay of processes across multiple time-scales:Illustration with the Intraindividual Study of Affect, Health, and Interpersonal Behavior (iSAHIB). Research in Human Development, 11, 142–160. doi:10.1080/15427609.2014.906739
41. Ram, N., & Gerstorf, D., (2009). Time structured and net intraindividual variability:Tools for examining the development of dynamic characteristics and processes. Psychology and Aging, 24, 778–791. doi:10.1037/a0017915
42. Ram, N., & Reeves, B., (in press). Time sampling. In M., Bornstein (Ed.), The SAGE encyclopedia of lifespan human development. New York, NY:Sage.
43. Reis, H. T., (2012). Why researchers should think “real-world”:A conceptual rationale. In M. R., Mehl & T. S., Conner (Eds.), Handbook of research methods for studying daily life (pp. 3–21). New York, NY:Guilford Press.
44. Reynolds, B. M., Robles, T. F., & Repetti, R. L., (2016). Measurement reactivity and fatigue effects in daily diary research with families. Developmental Psychology, 52, 442–456. doi:10.1037/dev0000081
45. Saeb, S., Zhang, M., Karr, C. J., Schueller, S. M., Corden, M. E., Kording, K. P., & Mohr, D. C., (2015). Mobile phone sensor correlates of depressive symptom severity in daily-life behavior:An exploratory study. Journal of Medical Internet Research, 17, e175. doi:10.2196/jmir.4273
46. Salathé, M., Kazandjieva, M., Lee, J. W., Levis, P., Feldman, M. W., & Jones, J. H., (2010). A high-resolution human contact network for infectious disease transmission. Proceedings of the National Academy of Sciences, 107, 22020–22025.
47. Sankoff, D., & Kruskal, J. B., (Eds.). (1983). Time warps, string edits, and macromolecules:The theory and practice of sequence comparison. Reading, MA:Addison Wesley.
48. Schwartz, N., (2007). Retrospective and concurrent self-reports:The rationale for real-time data capture. In A., Stone, S., Shiffman, A. A., Atienza, & L., Nebeling (Eds.), The science of real-time data capture:Self-reports in health research (pp. 11–26). New York, NY:Oxford University Press.
49. Shiffman, S., Stone, A. A., & Hufford, M. R., (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32. doi:10.1146/annurev.clinpsy.3.022806.091415
50. Stone, A. A., Shiffman, S., Schwartz, J. E., Broderick, J. E., & Hufford, M. R., (2002). Patient non-compliance with paper diaries. British Medical Journal, 324(7347), 1193–1194. doi:10.1136/bmj.324.7347.1193
51. Stone, A. A., Shiffman, S., Atienza, A. A., & Nebeling, L., (2007). Historical roots and rationale of ecological momentary assessment (EMA). In A. A. Stone, S. Shiffman, A. A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture:Self-reports in health research (pp. 3–10). Oxford:Oxford University Press.
52. Stone, A. A., & Shiffman, S. S., (2011). Ecological validity for patient reported outcomes. In A., Steptoe, K., Freedland, J. R., Jennings, M. M., Llabre, S. B., Manuck, & E. J., Susman (Eds.), Handbook of behavioral medicine:Methods and applications (pp. 99–112). New York, NY:Springer.
53. Studer, M., & Ritschard, G., (2015). What matters in differences between life trajectories:A comparative review of sequence dissimilarity measures. Journal of the Royal Statistical Society, Series A, 179, 481–511. doi:10.1111/rssa.12125.
54. Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., … Campbell, A. T., (2014). StudentLife:Assessing mental health, academic performance and behavioral trends of college students using smartphones. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 3–14). New York, NY:ACM.
55. Ward, J. H., Jr. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58, 236–244. doi:10.1080/01621459.1963.10500845
56. Wegener, D. T., & Blankenship, K. L., (2007). Ecological validity. In R. F., Baumeister & K. D., Vohs (Eds.), Encyclopedia of social psychology (pp. 275–276). Thousand Oaks, CA:Sage.
57. Wilhelm, P., Perrez, M., & Pawlik, K., (2012). Conducting research in daily life:A historical review. In M. R., Mehl & T. S., Conner (Eds.), Handbook of research methods for studying daily life (pp. 62–86). New York, NY:Guilford Press.
58. Yeykelis, L., Cummings, J. J., & Reeves, B., (2014). Multitasking on a single device:Arousal and the frequency, anticipation, and prediction of switching between media content on a computer. Journal of Communication, 64, 167–192. doi:10.1111/jcom.12070