Evolutionary Evaluation: Implications for evaluators, researchers, practitioners, funders and the evidence-based program mandate

Evaluation and Program Planning

Volumn 45, Issue , 2014, Pages 127-139

  Urban, Jennifer Brown ^a   Hargraves, Monica ^b   Trochim, William M ^b  

^a MONTCLAIR STATE UNIVERSITY   (United States)

^b Cornell University ^*  (United States)

Author keywords

Developmental systems theory; Evaluation design; Evidence based program (EBP); Evolutionary epistemology; Evolutionary Evaluation; Evolutionary theory; Experimental design; Lifecycles; Program evolution; Randomized controlled trial (RCT); Systems evaluation; Validity

Indexed keywords

BOTTOM-UP APPROACH; CONCEPTUAL FRAMEWORK; EVOLUTIONARY THEORY; EXPERIMENTAL DESIGN; THEORETICAL STUDY; TOP-DOWN APPROACH;

HUMAN; METHODOLOGY; PROCEDURES; PROGRAM DEVELOPMENT; PROGRAM EVALUATION; RANDOMIZED CONTROLLED TRIAL (TOPIC); SYSTEMS THEORY;

HUMANS; PROGRAM DEVELOPMENT; PROGRAM EVALUATION; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN; SYSTEMS THEORY;

EID: 84899847165     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.evalprogplan.2014.03.011     Document Type: Article

References (76)

1. Bannan-Ritlans The role of design in research: The integrative learning design framework. Educational Researcher 2003, 32(1):21-24.
2. Bertalanffy L.V. The history and status of general systems theory. Trends in general systems theory 1972, Wiley-Interscience, New York. G.J. Klir (Ed.).
3. Bradie M., Harms W. Evolutionary epistemology. The stanford encyclopedia of philosophy 2006, Stanford University, Stanford, CA. E.N. Zalta (Ed.).
4. Brooks A. Commentary on "Expanding evaluative thinking: Evaluation through the program life cycle". American Journal of Evaluation 2012, 33(2):280-282.
5. Campbell D.T. Reforms as experiments. American Psychologist 1969, 24(4):409-429.
6. Campbell D.T. Evolutionary epistemology. The philosophy of Karl Popper 1974, Open Court Publishing Co., LaSalle, IL. P.A. Schilpp (Ed.).
7. Campbell D.T. Evolutionary epistemology. Methodology and epistemology for social science: Selected papers of Donald T. Campbell. 1988, University of Chicago Press, Chicago. E.S. Overman (Ed.).
8. Campbell D.T., Stanley J.C. Experimental and quasi-experimental designs for research on teaching. Handbook of research on teaching 1963, Rand McNally, Chicago. N.L. Gage (Ed.).
9. Caracelli V. Structured conceptualization: A framework for interpreting evaluation results. Evaluation and Program Planning 1989, 12(1):45-52.
10. Chapel T.J. Evaluation purpose and use: The core of the CDC program evaluation framework. American Journal of Evaluation 2012, 33(2):286-289.
11. Chen H. Practical program evaluation: Assessing and improving planning, implementation, and effectiveness 2005, Sage, Thousand Oaks, CA.
12. Chen H. The bottom-up approach to integrative validity: A new perspective for program evaluation. Evaluation and Program Planning 2010, 33:205-214.
13. Chen H., Rossi P. Theory-driven evaluations 1990, Sage, Thousand Oaks, CA.
14. Colby A., Kohlberg L., Lieberman M. A longitudinal study of moral judgement. Monographs of the Society for Research in Child Development 1983, 48(1-2).
15. Colosi L., Brown J.S. Towards a systems evaluation protocol. Paper presented at the American Evaluation Association 2006.
16. Community Preventive Services Task Force The Community Guide 2012, Retrieved from http://www.thecommunityguide.org/index.html.
17. Cook T.D., Campbell D.T. Quasi-experimentation: Design and analysis for field settings 1979, Houghton Mifflin Company, Boston.
18. Cornell Office for Research on Evaluation (CORE) The evaluation facilitator's guide to: Systems evaluation protocol 2009, Cornell Digital Print Services, Ithaca, NY.
19. Cronbach L.J., Ambron S.R., Dornbusch S.M., Hess R., Hornik R.C., Phillips D.C., et al. Toward reform of program evaluation 1980, Jossey-Bass, San Francisco.
20. Cziko G.A., Campbell D.T. Comprehensive evolutionary epistemology bibliography. Journal of Social and Biological Structures 1990, 13(1):41-82.
21. Darwin C. On the origin of species by means of natural selection, or the preservation of favoured races in the struggle for life 1859, John Murray, London.
22. Durlak J.A., DuPre E.P. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology 2008, 41(3-4):327-350. 10.1007/s10464-008-9165-0.
23. Green L.W. Making research relevant: If it is an evidence-based practice, where's the practice-based evidence?. Family Practice 2008, 25(Suppl. 1):i20-i24.
24. Grob G.F. A lifer's perspective on life cycle evaluations. American Journal of Evaluation 2012, 33(2):282-285.
25. Grol R., Grimshaw J. Evidence-based implementation of evidence-based medicine. The Joint Commission Journal on Quality Improvement 1999, 25(10):503-513.
26. Hebbard C., Trochim W., Urban J.B., Casillas W., Cathles A., Hargraves M., et al. Alignment of program lifecycles and evaluation lifecycles. Paper presented at the American Evaluation Association 2009, https://core.human.cornell.edu/documents/lifecycleposterAEA2009.pdf.
27. Herbert J.D. The science and practice of empirically supported treatments. Behavior Modification 2003, 27(3):412-430.
28. Institute of Education Sciences What Works Clearinghouse 2012, Retrieved from http://ies.ed.gov/ncee/wwc/.
29. Kane M., Trochim W. Concept mapping for planning and evaluation 2006, Sage Publications, Thousand Oaks, CA.
30. Kazdin A.E. Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist 2008, 63(3):146-159.
31. Kerner J.F., Guirguis-Blake J., Hennessy K.D., Brounstein P.J., Vinson C., Schwartz R.H., et al. Translating research into improved outcomes in comprehensive cancer control. Cancer Causes & Control 2005, 16:27-40.
32. Khoury M.J., Gwinn M., Yoon P.W., Dowling N., Moore C.A., Bradley L. The continuum of translation research in genomic medicine: How can we accelerate the appropriate integration of human genome discoveries into health care and disease prevention. Genetics in Medicine 2007, 9(10):665-674.
33. Kohlberg L. The development of children's orientations toward a moral order: I. Sequence in the development of moral thought. Vita Humana 1963, 6:11-33.
34. Kohlberg L. Essays on moral development. The pyschology of moral development 1984, Vol. 2. Harper & Row, San Francisco.
35. Laszlo E. The Systems View of the World: A Holistic Vision for Our Time (Advances in Systems Theory, Complexity, and the Human Sciences) 1996, Hampton Press, Creskill, NJ.
36. Lerner R.M. Concepts and theories of human development 2002, Lawrence Earlbaum Associates, Mahwah, NJ.
37. Lerner R.M. Developmental science, developmental systems, and contemporary theories of human development. Theoretical models of human development 2006, Vol. 1:1-17. Wiley, Hoboken, NJ. 6th ed.
38. Linstone H.A., Turoff M. The Delphi method: Techniques and applications 1975, Addison-Wesley Publishing Company, Reading, MA.
39. Mark M.M. Program life cycle stage as a guide to evaluation decision making: Benefits, limits, alternatives, and future directions. American Journal of Evaluation 2012, 33(2):277-280.
40. Mayo Clinic Clinical trials: A chance to try evolving therapies. 2007, Retrieved from http://www.mayoclinic.com/health/clinical-trials/DI00033.
41. Mayr U. Age differences in the selection of mental sets: The role of inhibition, stimulus ambiguity, and response-set overlap. Psychology and Aging 2001, 16(1):96-109. 10.1037//0882-7974.16.1.96.
42. Meyer J.R. Symbiotic relationships 2012, From http://www.cals.ncsu.edu/course/ent591k/symbiosis.html.
43. Midgley G. Systems thinking 2003, Sage, Thousand Oaks, CA.
44. Molles M.C. Ecology: Concepts and applications 2001, McGraw-Hill, Boston. 2nd ed.
45. National Institutes of Health Commonfund 2012, Retrieved from http://commonfund.nih.gov/.
46. National Library of Medicine What are clinical trial phases? 2008, Retrieved from http://www.nlm.nih.gov/services/ctphases.html.
47. Nutley S., Davies H.T.O. Making a reality of evidence-based practice: Some lessons from the diffusion of innovations. Public Money & Management 2000, 20(4):35-42.
48. Overton W.F. Developmental psychology: Philosophy, concepts, methodology. Theoretical models of human development 2006, Vol. 1:18-88. Wiley, Hoboken, NJ. 6th ed.
49. Overton W.F. Life-span development: Concepts and issues. Handbook of life-span development 2010, Vol. 1. Wiley, Hoboken, NJ.
50. Patton M.Q. Developmental evaluation: Applying complexity concepts to enhance innovation and use 2011, Guilford Press, New York.
51. Pickett S.T., Kolasa J., Jones C.G. Ecological understanding 1994, Academic Press, San Diego, CA.
52. Pollan M. The botany of desire: A plant's-eye view of the world 2002, Random House, New York.
53. Popper K. Evolutionary epistemology. Paper presented at the Sections I-VI of "The Rationality of Scientific Revolutions" at the Herbert Spencer Lecture 1973.
54. Popper K. Evolutionary epistemology. Problems of scientific revolution 1975, Oxford University Press, Oxford. R. Harre (Ed.).
55. Popper K. Evolutionary epistemology. Popper selections 1985, 78-86. Princeton University Press, Princeton, NJ. D.M. Miller (Ed.).
56. Ragsdell G., West D., Wilby J. Systems theory and practice in the knowledge age 2002, Kluwer Academic/Plenum Publishers, New York.
57. Richerson P.J., Mulder M.B., Vila B.J. Principles of human ecology 1996, Simon & Schuster Custom Pub, Needham Heights, MA.
58. Rossi P., Lipsey M., Freeman H. Evaluation: A systematic approach 2003, Sage Publications, Thousand Oaks, CA.
59. Ruegg R., Jordan G. Overview of evaluation methods for R&D programs: A directory of evaluation methods relevant to technology development programs 2007, U.S. Department of Energy, Office of Energy Efficiency and Renewable Energy.
60. Scheirer M.A. Expanding evaluative thinking: Evaluation through the program life cycle. American Journal of Evaluation 2012, 33(2):264-277.
61. Scheirer M.A., Scheirer M.A., Mark M.M., Brooks A., Grob G.F., Chapel T.J., et al. Planning evaluation through the program life cycle. American Journal of Evaluation 2012, 33(2):263-294.
62. Scriven M. The methodology of evaluation. Perspectives on curriculum evaluation, AERA monograph series on curriculum evaluation 1967, Vol. 1:38-83. Rand McNally, Skokie, IL.
63. Shadish W., Cook T.D., Campbell D.T. Experimental and quasi-experimental designs for generalized causal inference 2002, Houghton Mifflin, Boston.
64. Toulmin S. The evolutionary development of natural science. American Scientist 1967, 55(4):456-471.
65. Trochim W. Research methods: The concise knowledge base 2005, Cengage Learning, Mason, OH.
66. Trochim W. An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning 1989, 12(1):1-16.
67. Trochim W., Kane M. Concept mapping: An introduction to structured conceptualization in health care. International Journal for Quality in Health Care 2005, 17(3):187-191.
68. Trochim W., Linton R. Conceptualization for planning and evaluation. Evaluation and Program Planning 1986, 9(4):289-308.
69. Trochim W., Urban J.B., Hargraves M., Hebbard C., Buckley J., Archibald T., et al. The guide to the systems evaluation protocol 2012, Cornell Digital Print Services, Ithaca, NY.
70. Trochim W.M. Evolutionary perspectives in evaluation: Theoretical and practical implications. Paper presented at the Eastern Evaluation Research Society 2007, http://www.socialresearchmethods.net/research/EERS2007/Evolutionary%20Perspectives%20in%20Evaluation%20Theoretical%20and%20Practical%20Implications.pdf.
71. Trochim W.M., Hertzog S., Kane C., Duttweiller M. Building evaluation capacity in extension systems. Paper presented at the American Evaluation Association 2007.
72. UNEP/CMS Secretariat Migratory species and climate change: Impacts of a changing environment on wild animals 2006, UNEP/CMS Convention on Migratory Species and DEFRA, Bonn, Germany.
73. Urban J.B., Hargraves M., Hebbard C., Burgermaster M., Trochim W. Evaluation in the context of lifecycles: A place for everything and everything in its place. Paper presented at the American Evaluation Association 2011.
74. Urban J.B., Trochim W.M. The role of evaluation in research-practice integration: Working toward the "golden spike". American Journal of Evaluation 2009, 30(4):538-553.
75. Walker L., Gustafson P., Hennig K.H. The consolidation/transition model in moral reasoning development. Developmental Psychology 2001, 37:187-197.
76. Wandersman A.H., Lesesne C.A. If translational research is the answer, what's the question? Who gets to ask it?. Research for the public good: Applying the methods of translational research to improve human health and well-being 2012, 33-51. American Psychological Association, Washington, DC. E. Wethington, R.E. Dunifon (Eds.).