Evaluation Theory and Practice: Comparing Program Evaluation and Evaluation in Planning

Journal of Planning Education and Research

Volumn 38, Issue 1, 2018, Pages 98-110

  Guyadeen, Dave ^a   Seasons, Mark ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

Author keywords

evaluation challenges and benefits; plan evaluation; planning evaluation; program evaluation

Indexed keywords

EID: 85041005755     PISSN: 0739456X     EISSN: 15526577     Source Type: Journal    
DOI: 10.1177/0739456X16675930     Document Type: Review

References (90)

1. Agol Dorice, Lataweic Agnieszka E., Strassburg Bernardo B. N., 2014. “Evaluating Impacts of Development and Conservation Projects Using Sustainability Indicators: Opportunities and Challenges.” Environmental Impact Assessment Review 48:1–9.
2. Alexander Ernest R., 2006a. “Evolution and Status: Where Is Planning-Evaluation Today and How Did It Get Here?” in Evaluation in Planning: Evolution and Prospects, edited by Alexander Ernest R., 3–18. Burlington, VT: Ashgate.
3. Alexander Ernest R., 2006b. “Evaluations and Rationalities: Reasoning with Values in Planning.” In Evaluation in Planning: Evolution and Prospects, edited by Alexander Ernest R., 3–18. Burlington, VT: Ashgate.
4. Alexander Ernest R., 2011. “Evaluating Planning: What Is Successful Planning and (How) Can We Measure It?” In Evaluation for Participation and Sustainability in planning, edited by Hull Angela, Alexander Ernest R., Khakee Abdul, Woltjer Johan, 32–46. New York: Routledge.
5. Alkin Marvin C., 2013. Evaluation Roots: A Wider Perspective of Theorists’ Views and Influences. Thousand Oaks, CA: Sage.
6. Allred Dustin, Chakraborty Arnab. 2015. “Do Local Development Outcomes Follow Voluntary Regional Plans?” Journal of the American Planning Association 81:104–20.
7. Baer William C., 1997. “General Plan Evaluation Criteria: An Approach to Making Better Plans.” Journal of the American Planning Association 63:329–44.
8. Baker Mark, Wong Cecilia. 2005. “Indicators and Strategy Monitoring: The Case of the English Regions.” Environment and Planning B: Planning and Design 33:661–83.
9. Balsas Carlos. 2012. “What about Plan Evaluation? Integrating Evaluation in Urban Planning Studios’ Pedagogy.” Planning Practice & Research 27:475–94.
10. Barbanente Angela, Khakee Abdul. 2005. “Scenarios as an Exploratory Evaluation Approach—Some Experiences from Southern Mediterranean.” In Beyond Benefit-Cost Analysis: Accounting for Non-market Values in Planning Evaluation, edited by Miller Donald, Patassini Domenico, 225–47. Burlington, VT: Ashgate.
11. Baynham Maggie, Stevens Mark. 2014. “Are We Planning Effectively for Climate Change? An Evaluation of Official Community Plans in British Columbia.” Journal of Environmental Planning and Management 57:557–87.
12. Berke Philip, Backhurst Michael, Day Maxine, Ericksen Neil, Laurian Lucie, Crawford Jan, Dixon Jennifer. 2006. “What Makes Plan Implementation Successful? An Evaluation of Local Plans and Implementation Practices in New Zealand.” Environment and Planning B: Planning and Design 33:581–600.
13. Berke Philip, Godschalk David. 2009. “Searching for the Good Plan: A Meta-analysis of Plan Quality Studies.” Journal of Planning Literature 23:227–40.
14. Berke Philip, Smith Gavin, Lyles Ward. 2012. “Planning for Resiliency: Evaluation of State Hazard Mitigation Plans under the Disaster Mitigation Act.” Natural Hazards Review 13:139–49.
15. Bracken Ian. 1981. Urban Planning Methods: Research and Policy Analysis. New York: Methuen.
16. Briassoulis Helen. 2001. “Sustainable Development and Its Indicators: Through a (Planner’s) Glass Darkly.” Journal of Environmental Planning and Management 44:409–27.
17. Brody Samuel D., 2003. “Are We Learning to Make Better Plans? A Longitudinal Analysis of Plan Quality Associated with Natural Hazards.” Journal of Planning Education and Research 23:191–201.
18. Brody Samuel, Godschalk David R., Burby Raymond J., 2003. “Mandating Citizen Participation in Plan Making.” Journal of the American Planning Association 69:245–64.
19. Brody Samuel, Highfield Wesley E., 2005. “Does Planning Work? Testing the Implementation of Local Environmental Planning in Florida.” Journal of the American Planning Association 71:159–75.
20. Brody Samuel, Highfield Wesley E., Thornton Sara. 2006. “Planning at the Urban Fringe: An Examination of the Factors Influencing Nonconforming Development Patterns in Southern Florida.” Environment and Planning B: Planning and Design 33:75–96.
21. Burby Raymond J., 2003. “Making Plans That Matter: Citizen Involvement and Government Action.” Journal of the American Planning Association 69:33–49.
22. Burchfield Marcy. 2014. “Growth Plan’s Performance Indicators: An Opportunity to Be More Than Just a Numbers Exercise.” http://www.neptis.org/latest/news/growth-plans-performance-indicators-opportunity-be-more-just-numbers-exercise (accessed December 31, 2015).
23. Carmona Mathew, Sieh Louie. 2005. “Performance Measurement Innovation in English Planning Authorities.” Planning Theory & Practice 6 (2005): 303–333.
24. Carmona Mathew, Sieh Louie. 2008. “Performance Measurement in Planning—Towards a Holistic View.” Environment and Planning C: Government and Policy 26:428–54.
25. Chapin Timothy, Deyle Robert E., Baker Earl J., 2008. “A Parcel-Based GIS Method for Evaluating Conformance of Local Land-Use Planning with a State Mandate to Reduce Exposure to Hurricane Flooding.” Environment and Planning B: Planning and Design 35:261–79.
26. Christensen Karen S., 2015. “Both Process and Outcome Essential to Planning.” Journal of Planning Education and Research 35:188–98.
27. Chouinard Jill A., 2013. “The Practice of Evaluation in Public Sector Contexts: A Response.” American Journal of Evaluation 34:266–69.
28. Cousins J. Bradley. 2004. “Crossing the Bridge: Toward Understanding through Systematic Inquiry.” In Evaluation Roots, edited by Alkin Marvin C., 293–318. Thousand Oaks, CA: Sage.
29. Cousins J. Bradley, Goh Swee C., Elliott Catherine, Aubry Tim, Gilbert Nathalie. 2014. “Government and Voluntary Sector Differences in Organizational Capacity to Do and Use Evaluation.” Evaluation and Program Planning 44:1–13.
30. Cronbach Lee J., 1991. “Functional Evaluation Design.” In Foundations of Program Evaluation: Theories of Practice, edited by Shadish Willam R., Cook Thomas D., Leviton Laura C., 323–76. Newbury Park, CA: Sage.
31. David Nina Palmy. 2015. “Factors Affecting Planned Unit Development Implementation.” Planning Practice and Research 30:393–409.
32. Faehnle Maija, Tyrvainen Liisa. 2013. “A Framework for Evaluating and Designing Collaborative Planning.” Land Use Policy 34:332–41.
33. Faludi Andreas. 2000. “The Performance of Spatial Planning.” Planning Practice and Research 15:299–318.
34. Faludi Andreas. 2006. “Evaluating Plans: The Application of the European Spatial Development Perspective.” In Evaluation in Planning: Evolution and Prospect, edited by Alexander Ernest A., 3–18. Burlington, VT: Ashgate.
35. Faludi Andreas, Altes Willem K., 1997. “Evaluating Communicative Planning.” In Evaluating Theory-Practice and Urban-Rural Interplay in Planning, edited by Borri Dino, Khakee Abdul, Lacirignola Cosimo, 3–22. Dordrecht: Kluwer.
36. Fetterman David M., 2004. “Branching Out or Standing on a Limb: Looking to Our Roots for Insight.” In Evaluation Roots, edited by Altes Marvin C., 304–18. Thousand Oaks, CA: Sage.
37. Gennaio Maria-Pia, Hersperger Anna M., Burgi Mattias. 2009. “Containing Urban Sprawl—Evaluating Effectiveness of Urban Growth Boundaries Set by the Swiss Land Use Plan.” Land Use Policy 26:224–32.
38. Greene Jennifer C., 1994. “The Art of Interpretation, Evaluation, and Presentation.” In Qualitative Program Evaluation: Practice and Promise, edited by Denzin Norman K., Lincoln Yvoona S., Thousand Oaks, CA: Sage.
39. Guba Egon G., Lincoln Yvonna S., 1989. Fourth Generation Evaluation. Newbury Park, CA: Sage.
40. Hambleton Robin, Thomas Huw. 1995. Urban Policy Evaluation—Challenge and Change. London: Paul Chapman.
41. Hoch Charles. 2002. “Evaluating Plans Pragmatically.” Planning Theory 1:53–75.
42. Hoernig Heidi, Seasons Mark. 2004. “Monitoring Indicators in Local and Regional Planning Practice: Concepts and Issues.” Planning Practice and Research 19:81–99.
43. Howlett Michael, Ramesh M., Perl Anthony. 2009. Studying Public Policy: Policy Cycles and Policy Subsystems. Toronto: Oxford University Press.
44. Kernaghan Kenneth, Marson Brian, Borins Sandford. 2005. The New Public Organization. Toronto: IPAC.
45. Khakee Abdul, Hull Angela, Miller Donald, Woltjer Johan. 2008. “New Principles in Planning Evaluation.” In New Principles in Planning Evaluation, edited by Khakee Abdul, Hull Angela, Miller Donald, Woltjer Johan, 1–13. Burlington, VT: Ashgate.
46. Khakee Abdul. 2003. “The Emerging Gap between Evaluation Research and Practice.” Evaluation 9:340–52.
47. Khakee Abdul. 2006. “Reading Plans as an Exercise in Evaluation.” Evaluation 6:119–36.
48. Laurian Lucie, Crawford Jan, Day Maxine, Kouwenhoven Peter, Mason Greg, Ericksen Neil, Beattie Lee. 2010. “Evaluating the Outcomes of Plans: Theory, Practice and Methodology.” Environment and Planning B: Planning and Design 37:740–57.
49. Laurian Lucie, Day Maxine, Berke Philip, Ericksen Neil, Backhurst Michael, Crawford Jan, Dixon Jenny. 2004. “Evaluating Plan Implementation: A Conformance-Based Methodology.” Journal of the American Planning Association 70:471–80.
50. Laurian Lucie, Shaw Mary M., 2008. “Evaluation of Public Participation: The Practices of Certified Planners.” Journal of Planning Education and Research 28:293–309.
51. Lichfield Nathaniel. 2000. “Where Do We Go from Here?” Evaluation 6:351–59.
52. Lichfield Nathaniel. 2005. “Towards Social Value for Money.” In Beyond Benefit-Cost Analysis: Accounting for Non-market Values in Planning Evaluation, edited by Miller Donald, Patsassini Domenico, 117–26. Burlington, VT: Ashgate.
53. Loh Carolyn G., 2011. “Assessing and Interpreting Non-conformance in Land-Use Planning Implementation.” Planning Practice & Research 26:271–87.
54. Mascarenhas André, Nunes Luís, Ramos Tomás. 2015. “Selection of Sustainability Indicators for Planning: Combining Stakeholders’ Participation and Data Reduction Techniques.” Journal of Cleaner Production 92:295–307.
55. Mastop H., Faludi Andreas. 1997. “Evaluation of Strategic Plans: The Performance Principle.” Environment and Planning B: Planning & Design 24:815–32.
56. McDavid James C., Hawthorn Laura L., 2006. Performance Evaluation & Performance Measurement: An Introduction to Practice. Thousand Oaks, CA: Sage.
57. Millard-Ball Adam. 2012. “The Limits of Planning: Casual Impacts of City Climate Action Plans.” Journal of Planning Education and Research 33:5–19.
58. Miller Donald, Patassini Domenico. 2005. “Introduction—Accounting for Non-market Values in Planning Evaluation.” In Beyond Benefit-Cost Analysis: Accounting for Non-market Values in Planning Evaluation, edited by Miller Donald, Patassini Domenico, 1–12. Burlington, VT: Ashgate.
59. Mills Albert J., Mills Jean C. Helms, Forshaw Carolyn, Bratton John. 2007. Organizational Behaviour in a Global Context. Peterborough, Ontario, Canada: Bradshaw.
60. Morckel Victoria C., 2010. “Commentary.” Environment and Planning B: Planning and Design 37:769–74.
61. Morrison Nicola, Pearce Barry. 2000. “Developing Indicators for Evaluating the Effectiveness of the UK Land Use Planning System.” Town Planning Review 71:191–211.
62. Oliveira Vitor, Pinho Paulo. 2009. “Evaluating Plans, Processes and Results.” Planning Theory & Practice 10:35–63.
63. Oliveira Vitor, Pinho Paulo. 2010a. “Evaluation in Urban Planning: Advances and Prospects.” Journal of Planning Literature 24:343–61.
64. Oliveira Vitor, Pinho Paulo. 2010b. “Measuring Success in Planning.” Town Planning Review 81:307–32.
65. Oliveira Vitor, Pinho Paulo. 2011. “Bridging the Gap between Planning Evaluation and Programme Evaluation: The Contribution of the PPR Methodology.” Evaluation 17:293–307.
66. Pal Leslie A., 2014. Beyond Policy Analysis: Public Issue Management in Turbulent Times. Toronto: Nelson.
67. Patton Michael Q., 2002. Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage.
68. Patton Michael Q., 2011. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford.
69. Perdicoúlis Anastássios, Glasson John. 2011. “The Use of Indicators in Planning: Effectiveness and Risks.” Planning Practice & Research 26:349–67.
70. Poister Theodore H., 2003. Measuring Performance in Public and Nonprofit Organizations. San Francisco, CA: Jossey-Bass.
71. Posavac Emil J., Carey Raymond G., 2007. Program evaluation: Methods and case studies. Upper Saddle River, NJ: Pearson.
72. Rae Alasdair, Wong Cecilia. 2012. “Monitoring Spatial Planning Policies: Towards an Analytical, Adaptive, and Spatial Approach to a ‘Wicked Problem.’” Environment and Planning B: Planning and Design 39:880–96.
73. Roberts Peter. 2006. “Evaluating Regional Sustainable Development: Approaches, Methods and the Politics of Analysis.” Journal of Environmental Planning and Management 49:515–32.
74. Rodriguez-Campos Liliana. 2012. “Advances in Collaborative Evaluation.” Evaluation and Program Planning 35:523–28.
75. Rossi Peter H., Lipsey Mark W., Freeman Howard E., Evaluation: A Systematic Approach, 7th ed. Thousand Oaks, CA: Sage.
76. Ryan Brent D., 2011. “Reading through a Plan: A Visual Interpretation of What Plans Mean and How They Innovate.” Journal of the American Planning Association 77:309–26.
77. Seasons Mark. 2003. “Monitoring and Evaluation in Urban Planning: Considering the Realities.” Journal of the American Planning Association 69:430–40.
78. Shadish Willam R., Thomas Cook D., Leviton Laura C., 1991. “Good Theory for Social Program Evaluation.” In Foundations of Program Evaluation: Theories of Practice, edited by Shadish Willam R., Cook Thomas D., Leviton Laura C., 36–67. Newbury Park, CA: Sage.
79. Stevens Mark. 2013. “Evaluating the Quality of Official Community Plans in Southern British Columbia.” Journal of Planning Education and Research 33:471–90.
80. Talen Emily. 1996a. “After the Plans: Methods to Evaluate the Implementation Success of Plans.” Journal of Planning Education and Research 16:79–91.
81. Talen Emily. 1996b. “Do Plans Get Implemented? A Review of Evaluation in Planning.” Journal of Planning Literature 10:248–59.
82. Talen Emily. 1997. “Success, Failure, and Conformance: An Alternative Approach to Planning Evaluation.” Environment and Planning B: Planning and Design 24:573–87.
83. Tang Zhenghong. 2008. “Evaluating Local Coastal Zone Land Use Planning Capacities in California.” Ocean and Coastal Management 51:544–55.
84. Tang Zhenghong, Lindell Michael K., Prater Carla, Wei Ting, Hussey Christopher M., 2011. “Examining Local Coastal Zone Management Capacity in U.S. Pacific Coastal Counties.” Coastal Management 39:105–32.
85. UNDP (United Nations Development Programme). “UNDP Evaluation Policy.” http://web.undp.org/evaluation/policy.shtml (accessed December 31, 2015).
86. Vedung Evert. 2010. “Four Waves of Evaluation Diffusion.” Evaluation 16:263–77.
87. Von Winterfeldt Detlof, Fisher Gregory W., 1973. “Multi-attribute Utility Theory: Models and Assessment Procedure.” In Utility, Probability and Human Decision Making, edited by Wendt Dirk, Vlek Charles, 47–83. Boston: D. Reidel Publishing.
88. Waldner Leora S., 2004. “Planning to Perform: Evaluation Models for City Planners.” Berkley Planning Journal 17:1–28.
89. Weiss Carol. 1998. Evaluation. Upper Saddle River, NJ: Prentice Hall.
90. Wong Cecilia, Baker Mark, Kidd Sue. 2006. “Monitoring of Spatial Strategies: The Case of Local Development Documents in England.” Environment and Planning C: Government and Policy 24:533–52.