Blending Qualitative and Computational Linguistics Methods for Fidelity Assessment: Experience with the Familias Unidas Preventive Intervention

Administration and Policy in Mental Health and Mental Health Services Research

Volumn 42, Issue 5, 2015, Pages 574-585

  Gallo, Carlos ^a   Pantin, Hilda ^b   Villamar, Juan ^a   Prado, Guillermo ^b   Tapia, Maria ^b   Ogihara, Mitsunori ^b   Cruden, Gracelyn ^a   Brown, C Hendricks ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

^b UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE   (United States)

Author keywords

Automated fidelity ratings; Computational linguistics; Computational methods; Fidelity assessment; Implementation research; Preventive intervention

Indexed keywords

ADOLESCENT; CONDUCT DISORDER; EVIDENCE BASED PRACTICE; FAMILY HEALTH; FEMALE; HEALTH CARE QUALITY; HEALTH SERVICES RESEARCH; HIGH RISK BEHAVIOR; HISPANIC; HIV INFECTIONS; HUMAN; HUMAN RELATION; LINGUISTICS; MACHINE LEARNING; MALE; PREVENTION AND CONTROL; PREVENTIVE MEDICINE; QUALITATIVE RESEARCH; RANDOMIZED CONTROLLED TRIAL (TOPIC); REPRODUCIBILITY; SCHOOL HEALTH SERVICE; SEXUAL BEHAVIOR; STATISTICS; UNSAFE SEX; VIDEORECORDING;

ADOLESCENT; CONDUCT DISORDER; EVIDENCE-BASED PRACTICE; FAMILY HEALTH; FEMALE; HEALTH SERVICES RESEARCH; HISPANIC AMERICANS; HIV INFECTIONS; HUMANS; LINGUISTICS; MACHINE LEARNING; MALE; PREVENTIVE MEDICINE; PROCESS ASSESSMENT (HEALTH CARE); PROFESSIONAL-FAMILY RELATIONS; QUALITATIVE RESEARCH; RANDOMIZED CONTROLLED TRIALS AS TOPIC; REPRODUCIBILITY OF RESULTS; RISK-TAKING; SCHOOL HEALTH SERVICES; SEXUAL BEHAVIOR; STATISTICS AS TOPIC; UNSAFE SEX; VIDEO RECORDING;

EID: 84939563091     PISSN: 0894587X     EISSN: 15733289     Source Type: Journal    
DOI: 10.1007/s10488-014-0538-4     Document Type: Article

References (57)

1. Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health,38(1), 4–23. doi:10.1007/s10488-010-0327-7.
2. Alexander, J., Pugh, C., Parsons, B., & Sexton, T. (2000). Family functional therapy. In D. S. Elliot (Ed.), Blueprints for violence prevention: Book three (pp. 445–453). Boulder: Center for the Study and Prevention of Violence.
3. Allen, J. D., Linnan, L. A., & Emmons, K. M. (2012). Fidelity and its relationship to impementation effectiveness, adaptation, and dissemination. In R. C. Brownson, G. A. Colditz, & E. K. Proctor (Eds.), Dissemination and implementation in health: Translating science to practice (pp. 281–304). New York: Oxford University Press.
4. Bahrammirzaee, A. (2010). A comparative survey of artificial intelligence applications in finance: Artificial neural networks, expert system and hybrid intelligent systems. Neural Computing and Applications,19(8), 1165–1195. doi:10.1007/s00521-010-0362-z.
5. Barber, J. P., Gallop, R., Crits-Christoph, P., Frank, A., Thase, M. E., Weiss, R. D., et al. (2006). The role of therapist adherence, therapist competence, and the alliance in predicting outcome of individual drug counseling: Results from the NIDA collaborative cocaine treatment study. Psychotherapy Research,16, 240–299.
6. Brown, C. H., Kellam, S. G., Kaupert, S., Muthén, B. O., Wang, W., Muthén, L., et al. (2012). Partnerships for the design, conduct, and analysis of effectiveness, and implementation research: Experiences of the prevention science and methodology group. Administration and Policy in Mental Health,39(4), 301–316. doi:10.1007/s10488-011-0387-3.
7. Brown, C. H., Mohr, D. C., Gallo, C. G., Mader, C., Palinkas, L. A., Wingood, G., et al. (2013). A computational future for preventing HIV in minority communities: How advanced technology can improve implementation of effective programs. Journal of Acquired Immune Deficiency Syndromes,63(1), S72–S84.
8. Busch, A. M., Kanter, J. W., Callaghan, G. M., Baruch, D. E., Weeks, C. E., & Berlin, K. S. (2009). A micro-process analysis of functional analytic psychotherapy’s mechanism of change. Behavior Therapy,40(3), 280–290. doi:10.1016/j.beth.2008.07.003.
9. Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics,22(2), 249–254.
10. Chambers, D. A. (2008). Advancing the science of implementation: A workshop summary. Administration and Policy in Mental Health and Mental Health Services Research,35(1–2), 3–10. doi:10.1007/s10488-007-0146-7.
11. Chambers, D. (2012). The interactive systems framework for dissemination and implementation: Enhancing the opportunity for implementation science. American Journal of Community Psychology,50(3–4), 282–284. doi:10.1007/s10464-012-9528-4.
12. Chinman, M., Hunter, S., Ebener, P., Paddock, S., Stillman, L., Imm, P., et al. (2008). The getting to outcomes demonstration and evaluation: An illustration of the prevention support system. American Journal of Community Psychology,41(3–4), 206–224. doi:10.1007/s10464-008-9163-2.
13. De Cooman, G., & Zaffalon, M. (2004). Updating beliefs with incomplete observations. Artificial Intelligence,159(1–2), 75–125.
14. Durlak, J., & DuPre, E. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology,41(3), 327–350. doi:10.1007/s10464-008-9165-0.
15. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. (No. FMHI Publication #231). Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute.
16. Gallo, C. G., Aist, G., Allen, J. F., de Beaumont, W., Coria, S., Gegg-Harrison, W., & Swift, M. (2007). Continuous understanding in a multimodal dialogue corpus. Proceedings of DECALOG: The 2007 Workshop on the Semantics and Pragmatics of Dialogue, Trento (pp. 75–82).
17. Gallo, C. G., Jaeger, T. F., & Furth, K. (2010). A database for the exploration of Spanish planning. The Seventh International Conference on Language Resources and Evaluation (LREC). Valletta, Malta.
18. Hanson, R., Gros, K., Davidson, T., Barr, S., Cohen, J., Deblinger, E., & Ruggiero, K. (2013). National trainers’ perspectives on challenges to implementation of an empirically-supported mental health treatment. Administration and Policy in Mental Health and Mental Health Services Research, Online First, 1–13. doi:10.1007/s10488-013-0492-6.
19. Henggeler, S. W., & Borduin, C. M. (1990). Family therapy and beyond: A multisystemic approach to treating the behavior problems of children and adolescents. Pacific Grove: Brooks/Cole.
20. Henggeler, S. W., Schoenwald, S. K., Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (1998). Multisystemic treatment of antisocial behavior in children and adolescents. Treatment manuals for practitioners. New York: Guilford Press.
21. Henggeler, S. W., Schoenwald, S. K., Liao, J. G., Letourneau, E. J., & Edwards, D. L. (2002). Transporting efficacious treatments to field settings: The link between supervisory practices and therapist fidelity in MST programs. Journal of Clinical Child and Adolescent Psychology: The Official Journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53,31(2), 155–167.
22. Herschell, A. D. (2010). Fidelity in the field: Developing infrastructure and fine-tuning measurement. Clinical Psychology: Science and Practice,17(3), 253–257. doi:10.1111/j.1468-2850.2010.01216.x.
23. Hogue, A., Dauber, S., Chinchilla, P., Fried, A., Henderson, C., Inclan, J., et al. (2008). Assessing fidelity in individual and family therapy for adolescent substance abuse. Journal of Substance Abuse Treatment,35(2), 137–147.
24. Holmes, J. N., & Holmes, W. J. (2002). Speech synthesis and recognition 2e (HBK) (2nd ed.). New York: Taylor & Francis.
25. Huth, M., & Ryan, M. (2004). Logic in computer science: Modelling and reasoning about systems. Cambridge: Cambridge University Press.
26. Inoue, M., Ogihara, M., Hanada, R., & Furuyama, N. (2010). Utility of gestural cues in indexing semantic miscommunication. Future Information Technology (FutureTech), 2010 5th International Conference on Future Information Technology (pp. 1–6).
27. Kellam, S. G., Mackenzie, A. C. L., Brown, C. H., Poduska, J. M., Petras, H., & Wilcox, H. C. (2011). The good behavior game and the future of prevention and treatment. Addiction Science and Clinical Practice,6(1), 73–84.
28. Landsverk, J., Brown, C. H., Chamberlain, P., Palinkas, L., Rolls Reutz, J., & Horwitz, S. M. (2012). Design and analysis in dissemination and implementation research. In R. C. Brownson, G. A. Colditz, & E. K. Proctor (Eds.), Dissemination and implementation research in health: Translating science to practice (pp. 225–260). London: Oxford University Press.
29. Liddle, H. (2002). Multidimensional family therapy (MDFT) for adolescent cannabis users. Rockville: Center for Substance Abuse Treatment, Substance Abuse and Mental Health Services Administration.
30. Liddle, H. A., Rowe, C. L., Gonzalez, A., Henderson, C. E., Dakof, G. A., & Greenbaum, P. E. (2006). Changing provider practices, program environment, and improving outcomes by transporting multidimensional family therapy to an adolescent drug treatment setting. American Journal on Addictions,15(Suppl), 102–112.
31. Lopez, A. (2008). Statistical machine translation. ACM Computing Surveys (CSUR),40(3), 8.
32. Minuchin, S. (1974). Families and family therapy. Cambridge: Harvard University Press.
33. Minuchin, S., & Fishman, C. H. (1981). Family therapy techniques. Cambridge: Harvard University Press.
34. Orlinsky, D. E., Ronnestad, M. H., & Willutski, U. (2004). In Lambert M. J. (Ed.), Fifty years of psychotherapy process-outcome research: Continuity and change [Handbook of psychotherapy and behaviour change] (5th ed.). New York: John Wiley & Sons.
35. Overholser, J. C. (1995). Elements of the socratic method: IV. Disavowal of knowledge. Psychotherapy: Theory, research, practice. Training,32(2), 283–292.
36. Palinkas, L. A., Aarons, G. A., Horwitz, S. M., Chamberlain, P., Hulburt, M., & Landsverk, J. (2011). Mixed method designs in implementation research. Administration & Policy in Mental Health and Mental Health Services,38(1), 44–53.
37. Pantin, H., Coatsworth, J. D., Feaster, D. J., Newman, F. L., Briones, E., Prado, G., et al. (2003). Familias unidas: The efficacy of an intervention to promote parental investment in Hispanic immigrant families. Prevention Science,4(3), 189–201.
38. Pantin, H., Prado, G., Lopez, B., Huang, S., Tapia, M. I., Schwartz, S. J., et al. (2009). A randomized controlled trial of familias unidas for Hispanic adolescents with behavior problems. Psychosomatic Medicine,71(9), 987–995. doi:10.1097/PSY.0b013e3181bb2913.
39. Pantin, H., Schwartz, S. J., Sullivan, S., Prado, G., & Szapocznik, J. (2004). Ecodevelopmental HIV prevention programs for Hispanic adolescents. American Journal of Orthopsychiatry,74(4), 545–558.
40. Poduska, J., Kellam, S. G., Brown, C. H., Ford, C., Windham, A., Keegan, N., et al. (2009). Study protocol for a group randomized controlled trial of a classroom-based intervention aimed at preventing early risk factors for drug abuse: Integrating effectiveness and implementation research. Implementation Science,4, 56. doi:10.1186/1748-5908-4-56.
41. Popescu, A., Etzioni, O., & Kautz, H. (2003). Towards a theory of natural language interfaces to databases. Proceedings of the 8th International Conference on Intelligent User Interfaces, Miami (pp. 149–157).
42. Prado, G., Huang, S., Cordova, D., Malcolm, S., Estrada, Y., Cano, N., et al. (2013). Ecodevelopmental and intrapersonal moderators of a family based preventive intervention for hispanic youth: A latent profile analysis. Prevention Science: The Official Journal of the Society for Prevention Research,14(3), 290–299. doi:10.1007/s11121-012-0326-x.
43. Prado, G., & Pantin, H. (2011). Reducing substance use and HIV health disparities among Hispanic youth in the USA: The Familias Unidas program of research. Intervencion Psicosocial,20(1), 63–73. doi:10.5093/in2011v20n1a6.
44. Prado, G., Pantin, H., Briones, E., Schwartz, S. J., Feaster, D., Huang, S., et al. (2007). A randomized controlled trial of a parent-centered intervention in preventing substance use and HIV risk behaviors in Hispanic adolescents. Journal of Consulting and Clinical Psychology,75(6), 914–926. doi:10.1037/0022-006X.75.6.914.
45. Prado, G., Pantin, H., Huang, S., Cordova, D., Tapia, M. I., Velazquez, M. R., et al. (2012). Effects of a family intervention in reducing HIV risk behaviors among high-risk Hispanic adolescents: A randomized controlled trial. Archives of Pediatric and Adolescent Medicine,166(2), 127–133. doi:10.1001/archpediatrics.2011.189.
46. Prado, G., Pantin, H., Schwartz, S. J., Schwartz, S. J., Lupei, N. S., & Szapocznik, J. (2006). Predictors of engagement and retention into a parent-centered, ecodevelopmental HIV preventive intervention for Hispanic adolescents and their families. Journal of Pediatric Psychology,31(9), 874–890. doi:10.1093/jpepsy/jsj046.
47. Prinz, R. J., Sanders, M. R., Shapiro, C. J., Whitaker, D. J., & Lutzker, J. R. (2009). Population-based prevention of child maltreatment: The US triple p system population trial. Prevention Science,10(1), 1–12. doi:10.1007/s11121-009-0123-3.
48. Real, K., & Poole, M. S. (2005). Innovation implementation: Conceptualization and measurement in organizational research. In R. Woodman, W. Pasmore & A. B. Shani (Eds.), Research in organizational change and development (15th ed., pp. 63–134). Online: Emerald Group Publishing Limited. doi:10.1016/S0897-3016(04)15003-9.
49. Robbins, M. S., Feaster, D. J., Horigian, V. E., Puccinelli, M. J., Henderson, C., & Szapocznik, J. (2011). Therapist adherence in brief strategic family therapy for adolescent drug abusers. Journal of Consulting and Clinical Psychology,79(1), 43–53. doi:10.1037/a0022146.
50. Rowe, C., Rigter, H., Henderson, C., Gantner, A., Mos, K., Nielsen, P., et al. (2013). Implementation fidelity of multidimensional family therapy in an international trial. Journal of Substance Abuse Treatment,44(4), 391–399.
51. Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., et al. (2008). A survey of the infrastructure for children’s mental health services: Implications for the implementation of empirically supported treatments (ESTs). Administration and Policy in Mental Health and Mental Health Services Research,35(1–2), 84–97.
52. Schoenwald, S. K., & Garland, A. F. (2013). A review of treatment adherence measurement methods. Psychological Assessment,25, 146–156. doi:10.1037/a0029715.
53. Schoenwald, S. K., Garland, A. F., Chapman, J. E., Frazier, S. L., Sheidow, A. J., & Southam-Gerow, M. A. (2011). Toward the effective and efficient measurement of implementation fidelity. Administration and Policy in Mental Health,38(1), 32–43. doi:10.1007/s10488-010-0321-0.
54. Suarez-Morales, L., Matthews, J., Martino, S., Ball, S. A., Rosa, C., Farentinos, C., et al. (2007). Issues in designing and implementing a Spanish-language multi-site clinical trial. American Journal on Addictions,16(3), 206–215.
55. Szapocznik, J., & Coatsworth, J. D. (1999). An ecodevelopmental framework for organizing the influences on drug abuse: A developmental model of risk and protection. Drug abuse: Origins & interventions. Washington: American Psychological Association. doi:10.1037/10341-014.
56. Szapocznik, J., Hervis, O. E., & Schwartz, S. (2003). Brief strategic family therapy for adolescent drug abuse. NIH publication no. 03-4751; NIDA Therapy Manuals for Drug Addiction Series ed. Rockville: National Institute of Drug Abuse.
57. Wall, L., Christiansen, T., & Orwant, J. (2000). Programming perl (3rd ed.). Sebastopol: O’Reilly Media.