The social impact of advice during disability welfare reform: from social return on investment to evidencing public value through realism and complexity

Public Management Review

Volumn 21, Issue 2, 2019, Pages 238-263

  Farr, Michelle ^a,b,c   Cressey, Peter ^c  

^a The National Institute for Health Research Health Protection Research Unit in Blood Borne and Sexually Transmitted Infections at University College London in partnership with UK Health Security Agency   (United Kingdom)

^b UNIVERSITY OF BRISTOL   (United Kingdom)

^c University of Bath   (United Kingdom)

Author keywords

Advice services; complexity; disability; mental health; public value; realist evaluation; social impact; social return on investment; welfare reform

Indexed keywords

EID: 85047105902     PISSN: 14719037     EISSN: 14719045     Source Type: Journal    
DOI: 10.1080/14719037.2018.1473474     Document Type: Article

References (100)

1. Alford, J., 2016. “Co-Production, Interdependence and Publicness: Extending Public Service-Dominant Logic.” Public Management Review 18 (5): 673–691. doi:10.1080/14719037.2015.1111659
2. Alford, J., S., John, K., Douglas, Geuijen, K., and P., ‘T Hart. 2017. “Ventures in Public Value Management: Introduction to the Symposium.” Public Management Review 19 (5): 589–604. doi:10.1080/14719037.2016.1192160
3. Alford, J., and S., Yates. 2014. “Mapping Public Value Processes.” International Journal of Public Sector Management 27 (4): 334–352. doi:10.1108/IJPSM-04-2013-0054
4. Anderson, S., 2017. “Citizens Advice Services in Wymondham and Attleborough Cut as District Council Slashes Budget.” Eastern Daily Press, May 7. http://www.edp24.co.uk/news/citizens-advice-services-in-wymondham-and-attleborough-cut-as-district-council-slashes-budget-1-5006654
5. Arvidson, M., and F., Lyon. 2014. “Social Impact Measurement and Non-Profit Organisations: Compliance, Resistance, and Promotion.” Voluntas 25 (4): 869–886. doi:10.1007/s11266-013-9373-6
6. Arvidson, M., F., Lyon, S., McKay, and D., Moro. 2010. “The Ambitions and Challenges of SROI”. Working paper. TSRC Third Sector Research Centre, 49
7. Arvidson, M., F., Lyon, S., McKay, and D., Moro. 2013. “Valuing the Social? the Nature and Controversies of Measuring Social Return on Investment (SROI).” Voluntary Sector Review 4 (1): 3–18. doi:10.1332/204080513X661554
8. Ashdown, K., 2017. “Reduction of Citizens Advice Services Announced following Staffordshire County Council Funding Cuts.” Staffordshire Newsletter, February 24. http://www.staffordshirenewsletter.co.uk/reduction-of-citizens-advice-services-announced-following-staffordshire-county-council-funding-cuts/story-30159302-detail/story.html
9. Banke-Thomas, A., B. M., Oluwabusayo, A., Charles, and N., Van Den Broek. 2015. “Social Return on Investment (SROI) Methodology to Account for Value for Money of Public Health Interventions: A Systematic Review.” BMC Public Health 15 (1): 1–14. doi:10.1186/s12889-015-1935-7
10. Barnes, M. C., D., Gunnell, R., Davies, K., Hawton, N., Kapur, J., Potokar, and J. L., Donovan. 2016. “Understanding Vulnerability to Self-Harm in Times of Economic Hardship and Austerity: A Qualitative Study.” BMJ Open 6: 2. doi:10.1136/bmjopen-2015-010131
11. Barnes, M. C., J. L., Donovan, C., Wilson, J., Chatwin, R., Davies, J., Potokar, N., Kapur, K., Hawton, R., O’Connor, and D., Gunnell. 2017. “Seeking Help in Times of Economic Hardship: Access, Experiences of Services and Unmet Need.” BMC Psychiatry 17 (1): 84. doi:10.1186/s12888-017-1235-0
12. Barr, B., D., Taylor-Robinson, D., Stuckler, R., Loopstra, A., Reeves, and M., Whitehead. 2016a. “‘First, Do No Harm’: Are Disability Assessments Associated with Adverse Trends in Mental Health? A Longitudinal Ecological Study.” Journal of Epidemiology and Community Health 70 (4): 339–345. doi:10.1136/jech-2015-206209
13. Barr, B., D., Taylor-Robinson, D., Stuckler, R., Loopstra, A., Reeves, S., Wickham, and M., Whitehead. 2016b. “Fit-For-Work or Fit-For-Unemployment? Does the Reassessment of Disability Benefit Claimants Using a Tougher Work Capability Assessment Help People into Work?” Journal of Epidemiology and Community Health 70 (5): 452–458. doi:10.1136/jech-2015-206333
14. Bassi, A., 2012. “How to Measure the Intangibles? Towards a System of Indicators (S.A.V.E.) For the Measurement of the Performance of Social Enterprises.” In Challenge Social Innovation, edited by H. W., Franz, J., Hochgerner, and J., Howaldt, 325–350. Berlin, Heidelberg: Springer
15. Bath and North East Somerset Council. 2013. Petition for Debate - Citizens Advice Bureau. https://democracy.bathnes.gov.uk/mgAi.aspx?ID=10907
16. BBC. 2016. “Belfast Benefits Appeals Service Loses Funding.” British Broadcasting Corporation, December 29. http://www.bbc.co.uk/news/uk-northern-ireland-38457271
17. Bozeman, B., 2007. Public Values and Public Interest: Counter-Balancing Economic Individualism. Washington, DC: Georgetown University Press
18. Bozeman, B., and D., Sarewitz. 2011. “Public Value Mapping and Science Policy Evaluation.” Minerva 49 (1): 1–23. doi:10.1007/s11024-011-9161-7
19. Bryson, J., A., Sancino, J., Benington, and S., Eva. 2017. “Towards a Multi-Actor Theory of Public Value Co-Creation.” Public Management Review 19 (5): 640–654. doi:10.1080/14719037.2016.1192164
20. Butler, J., and D., Redding. 2017. Healthy Commissioning: How the Social Value Act Is Being Used by Clinical Commissioning Groups. London: National Voices and Social Enterprise UK. http://www.nationalvoices.org.uk/publications/our-publications/healthy-commissioning-how-social-value-act-being-used-clinical
21. Byrne, D., 2013. “Evaluating Complex Social Interventions in a Complex World.” Evaluation 19 (3): 217–228. doi:10.1177/1356389013495617
22. Citizens Advice Camden. 2016. “Written Evidence from Citizens Advice Camden.” http://data.parliament.uk/WrittenEvidence/CommitteeEvidence.svc/EvidenceDocument/Public%20Accounts/Contracted%20out%20health%20and%20disability%20assessments/written/28241.html
23. Committee of Public Accounts. 2016. Contracted Out Health and Disability Assessments: Thirty-Third Report of Session 2015–16. London: House of Commons
24. Cooney, K., 2017. “Legitimation Dynamics: How SROI Could Mobilize Resources for New Constituencies.” Evaluation and Program Planning 64: 110–115. doi:10.1016/j.evalprogplan.2016.11.010
25. Corden, A., R., Sainsbury, A., Irvine, and S., Clarke. 2010. “The Impact of Disability Living Allowance and Attendance Allowance: Findings from Exploratory Qualitative Research.” Research Report no. 649. London: Department of Work and Pensions
26. Cressey, P., S., Milner, M., Farr, N., Abercrombie, and B., Jaynes. 2014. Proving the Value of Advice: a Study of the Impact of Citizens’ Advice Bureau Services. Bath: Institute of Policy Research, University of Bath. http://www.bath.ac.uk/ipr/policy-briefs/cab.html
27. Dahl, A., and J., Soss. 2014. “Neoliberalism for the Common Good? Public Value Governance and the Downsizing of Democracy.” Public Administration Review 74 (4): 496–504. doi:10.1111/puar.12191
28. De Jong, J., S., Douglas, M., Sicilia, Z., Radnor, M., Noordegraaf, and P., Debus. 2017. “Instruments of Value: Using the Analytic Tools of Public Value Theory in Teaching and Practice.” Public Management Review 19 (5): 605–620. doi:10.1080/14719037.2016.1192162
29. Department for Work and Pensions. 2015. Government’s Response to the Independent Review of the Work Capability Assessment—Year Five. London: Stationery Office
30. Donati, P., and M., Archer. 2015. The Relational Subject. Cambridge: Cambridge University Press
31. Duffy, S., 2016. Basic Income Plus. Sheffield: Centre for Welfare Reform. http://www.centreforwelfarereform.org/library/by-date/basic-income-plus.html
32. Emerson, J., 2003. “The Blended Value Proposition: Integrating Social and Financial Returns.” California Management Review 45 (4): 35–51. doi:10.2307/41166187
33. Eppel, E. A., and M. L., Rhodes. 2017. “Complexity Theory and Public Management: A ‘Becoming’ Field.” Public Management Review 20 (7): 949–959. doi:10.1080/14719037.2017.1364414
34. Farr, M., P., Cressey, S., Milner, N., Abercrombie, and B., Jaynes. 2014. Proving the Value of Advice: a Study of the Impact of Citizens’ Advice Bureau Services. Exeter: South West Forum. http://opus.bath.ac.uk/39052/5/SWF_Proving_our_Value_CAB_Bath_April_2014_1.pdf
35. Financial Conduct Authority. 2014. FCA Confirms Price Cap Rules for Payday Lenders. London: Financial Conduct Authority. https://www.fca.org.uk/news/press-releases/fca-confirms-price-cap-rules-payday-lenders
36. Forster, N., S. M., Dalkin, M., Lhussier, P., Hodgson, and S. M., Carr. 2016. “Exposing the Impact of Citizens Advice Bureau Services on Health: A Realist Evaluation Protocol.” BMJ Open 6: 1. doi:10.1136/bmjopen-2015-009887
37. Freud, D., 2007. Reducing Dependency, Increasing Opportunity: Options for the Future of Welfare to Work. An Independent Report to the Department for Work and Pensions. Leeds: Corporate Document Services
38. Fujiwara, D., and R., Campbell. 2011. Valuation Techniques for Social Cost–Benefit Analysis: Stated Preference, Revealed Preference and Subjective Well-Being Approaches. London: HM Treasury and Department for Work and Pensions
39. Gentleman, A., 2013a. “Atos Comes under Attack in Emotional Commons Debate”. The Guardian, January 17. https://www.theguardian.com/society/2013/jan/17/atos-attack-emotional-commons-debate
40. Gentleman, A., 2013b. “Fitness-for-Work Tests Unfair on People with Mental Health Problems, Court Says”. The Guardian, May 22. https://www.theguardian.com/society/2013/may/22/fitness-work-tests-mental-health-unfair
41. Geraghty, L., 2017. “More Backing as Creditworthiness Assessment Bill Gathers Pace”. The Big Issue, December 8. https://www.bigissue.com/news/backing-creditworthiness-assessment-bill-gathers-pace/
42. Gibbon, J., and C., Dey. 2011. “Developments in Social Impact Measurement in the Third Sector: Scaling up or Dumbing Down?” Social and Environmental Accountability Journal 31 (1): 63–72. doi:10.1080/0969160X.2011.556399
43. Gray, P., 2014. An Independent Review of the Personal Independence Payment Assessment. London: Her Majesty’s Stationery Office
44. Gray, P., 2017. The Second Independent Review of the Personal Independence Payment Assessment. London: Her Majesty’s Stationery Office
45. Greig, G., K., Garthwaite, and C., Bambra. 2014. “Addressing Health Inequalities: Five Practical Approaches for Local Authorities.” Perspectives in Public Health 134 (3): 132–134. doi:10.1177/1757913914530654
46. Grieco, C., 2015. Assessing Social Impact of Social Enterprises. Cham: Springer International Publishing. doi: 10.1007/978-3-319-15314-8
47. Haighton, C., S., Moffatt, D., Howel, E., McColl, E., Milne, M., Deverill, G., Rubin, T., Aspray, and M., White. 2012. “The Do-Well Study: Protocol for a Randomised Controlled Trial, Economic and Qualitative Process Evaluations of Domiciliary Welfare Rights Advice for Socio-Economically Disadvantaged Older People Recruited via Primary Health Care.” BMC Public Health 12 (1): 382. doi:10.1186/1471-2458-12-382
48. Handley, S., F., Sabri, and A., Kazimirski. 2016. Shared Measurement: Greater than the Sum of Its Parts. London: New Philanthropy Capital
49. Harrington, M., 2010. An Independent Review of the Work Capability Assessment. London: Stationery Office
50. Harrington, M., 2011. An Independent Review of the Work Capability Assessment-Year Two. London: Stationery Office
51. Harrington, M., 2012. An Independent Review of the Work Capability Assessment-Year Three. London: Stationery Office
52. Hartley, J., J., Alford, E., Knies, and S., Douglas. 2017. “Towards an Empirical Research Agenda for Public Value Theory.” Public Management Review 19 (5): 670–685. doi:10.1080/14719037.2016.1192166
53. Haynes, P., 2017. “Understanding the Influence of Values in Complex Systems-Based Approaches to Public Policy and Management.” Public Management Review 20 (7): 980–996. doi:10.1080/14719037.2017.1364411
54. HM Government. 2012. Public Services (Social Value) Act. London: Stationary Office
55. Hood, C., 1991. “A Public Management for All Seasons?” Public Administration 69 (1): 3–19. doi:10.1111/j.1467-9299.1991.tb00779.x
56. Iacobucci, G., 2014. “GPs’ Workload Climbs as Government Austerity Agenda Bites.” British Medical Journal 349. doi:10.1136/bmj.g4300
57. Khazbak, R., 2017. Social Value in Procurement: Roundtable Write-Up. London: New Local Government Network. http://www.nlgn.org.uk/public/2017/social-value-in-procurement/
58. Litchfield, P., 2013. An Independent Review of the Work Capability Assessment–Year Four. London: Stationery Office
59. Litchfield, P., 2014. An Independent Review of the Work Capability Assessment–Year Five. London: Her Majesty’s Stationery Office
60. Machin, R., 2017. ““Made to Measure? an Analysis of the Transition from Disability Living Allowance to Personal Independence Payment.” “ Journal of Social Welfare and Family Law 39 (4): 435–453. doi:10.1080/09649069.2017.1390291
61. Maier, F., C., Schober, R., Simsa, and R., Millner. 2015. “SROI as a Method for Evaluation Research: Understanding Merits and Limitations.” Voluntas 26 (5): 1805–1830. doi:10.1007/s11266-014-9490-x
62. Martinelli, L., 2017. The Fiscal and Distributional Implications of Alternative Universal Basic Income Schemes in the UK: IPR Working Paper. Bath: Institute for Policy Research, University of Bath
63. Millar, R., and K., Hall. 2012. “Social Return on Investment (SROI) and Performance Measurement.” Public Management Review 15 (6): 923–941. doi:10.1080/14719037.2012.698857
64. Mills, C., 2017. “‘Dead People Don’t Claim’: A Psychopolitical Autopsy of UK Austerity Suicides.” Critical Social Policy. doi:10.1177/0261018317726263
65. Ministry of Justice. 2017a. Tribunals and Gender Recognitions Certificates Statistics Quarterly: January to March 2017 and 2016 to 2017. Main Tables. London: Ministry of Justice. https://www.gov.uk/government/statistics/tribunals-and-gender-recognitions-certificates-statistics-quarterly-january-to-march-2017-and-2016-to-2017
66. Ministry of Justice. 2017b. Tribunals and Gender Recognition Statistics Quarterly, January to March 2017 (Provisional). London: Ministry of Justice. https://www.gov.uk/government/statistics/tribunals-and-gender-recognitions-certificates-statistics-quarterly-january-to-march-2017-and-2016-to-2017
67. Moffatt, S., E., Noble, and C., Exley. 2010. “‘Done More for Me in a Fortnight than Anybody Done in All Me Life.’ How Welfare Rights Advice Can Help People with Cancer.” BMC Health Services Research 10 (1): 259. doi:10.1186/1472-6963-10-259
68. Moody, M., L., Littlepage, and N., Paydar. 2015. “Measuring Social Return on Investment.” Nonprofit Management and Leadership 26 (1): 19–37. doi:10.1002/nml.21145
69. Mook, L., J., Maiorano, S., Ryan, A., Armstrong, and J., Quarter. 2015. “Turning Social Return on Investment on Its Head.” Nonprofit Management and Leadership 26 (2): 229–246. doi:10.1002/nml.21184
70. Moore, M., 1995. Creating Public Value. Cambridge: Harvard University Press
71. Moore, M., 2014. “Public Value Accounting: Establishing the Philosophical Basis.” Public Administration Review 74 (4): 465–477. doi:10.1111/puar.12198
72. Morris, D., and W., Barr. 2013. “The Impact of Cuts in Legal Aid Funding on Charities.” Journal of Social Welfare and Family Law 35 (1): 79–94. doi:10.1080/09649069.2013.774737
73. Mueller, C., H., Gaus, and J., Rech. 2014. “The Counterfactual Self-Estimation of Program Participants.” American Journal of Evaluation 35 (1): 8–25. doi:10.1177/1098214013503182
74. NAO. 2016a. Contracted-Out Health and Disability Assessments. London: National Audit Office
75. NAO. 2016b. Departmental Overview 2015-16. Department for Work & Pensions. London: National Audit Office
76. NAO (National Audit Office). 2014. The Impact of Funding Reductions on Local Authorities. London: NAO
77. Neville, S., 2013. “DWP Tells ATOS Healthcare to Raise Standards.” Financial Times, July 22
78. Nicholls, A., and J., Emerson. 2015. “Social Finance: Capitalizing Social Impact.” In Social Finance, edited by A., Nicholls, R., Paton, and J., Emerson, 1–41. Oxford: Oxford University Press
79. Nicholls, J., 2017. “Social Return on investment—Development and Convergence.” Evaluation and Program Planning 64: 127–135. doi:10.1016/j.evalprogplan.2016.11.011
80. Nicholls, J., E., Lawlor, E., Neitzert, and T., Goodspeed. 2012. A Guide to Social Return on Investment. London: Cabinet Office
81. O’Hara, M., 2016. “It Will Hurt like Hell if Newcastle’s Citizens Advice Bureau Shuts Down.” The Guardian, January 20. https://www.theguardian.com/public-leaders-network/2016/jan/20/newcastle-citizens-advice-bureau-north-east-council-cuts
82. OECD. 2010. Sickness, Disability and Work: Breaking the Barriers: A Synthesis of Findings across OECD Countries. Paris: OECD Publishing. doi:10.1787/9789264088856-en
83. Office for Budget Responsibility. 2016. Welfare Trends Report. Presented to Parliament by the Economic Secretary to the Treasury by Command of Her Majesty. London: Crown Copyright
84. Osborne, S. P., 2010. The New Public Governance? London: Routledge
85. Osborne, S. P., 2017. “From Public Service-Dominant Logic to Public Service Logic: Are Public Service Organizations Capable of Co-Production and Value Co-Creation?” Public Management Review 20 (2): 225–231. doi:10.1080/14719037.2017.1350461
86. Pawson, R., 2013. The Science of Evaluation. London: Sage
87. Pearce, D., G., Atkinson, and S., Mourato. 2006. Cost-Benefit Analysis and the Environment: Recent Developments. Paris: Organisation for Economic Co-operation and Development
88. Raffass, T., 2017. “Demanding Activation.” Journal of Social Policy 46 (2): 349–365. doi:10.1017/S004727941600057X
89. Reeder, N., C., O’ Sullivan, S., Tucker, P., Ramsden, and G., Mulgan. 2012. Strengthening Social Innovation in Europe: Journey to Effective Assessment and Metrics. Brussels. Enterprise and Industry: European Commission
90. Roberts, A., L., Marshall, and A., Charlesworth. 2012. A Decade of Austerity? the Funding Pressures Facing the NHS from 2010/11 to 2021/22. Research Report. London: Nuffield Trust
91. Room, G., 2013. “Evidence for Agile Policy Makers: The Contribution of Transformative Realism.” Evidence & Policy 9 (2): 225–244. doi:10.1332/174426413X662653
92. Rutherford, J., 2007. “New Labour, the Market State, and the End of Welfare.” Soundings 36: 40–54. doi:10.3898/136266207820465660
93. Sancino, A., 2016. “The Meta Co-Production of Community Outcomes: Towards a Citizens’ Capabilities Approach.” Voluntas 27 (1): 409–424. doi:10.1007/s11266-015-9596-9
94. Shefer, G., C., Henderson, M., Frost-Gaskin, and R., Pacitti. 2016. “Only Making Things Worse: A Qualitative Study of the Impact of Wrongly Removing Disability Benefits from People with Mental Illness.” Community Mental Health Journal 52 (7): 834–841. doi:10.1007/s10597-016-0012-8
95. Spurr, H., 2014. “Advice Service Funding Slashed by 11.7%.” Inside Housing, May 23. https://www.insidehousing.co.uk/news/news/advice-service-funding-slashed-by-117-39969
96. United Nations. 2016. “Convention on the Rights of Persons with Disabilities: Inquiry Concerning the United Kingdom of Great Britain and Northern Ireland Carried Out by the Committee under Article 6 of the Optional Protocol to the Convention.” Report of the Committee. www.ohchr.org/Documents/HRBodies/CRPD/CRPD.C.15.R.2.Rev.1-ENG.doc
97. Walton, M., 2016. “Setting the Context for Using Complexity Theory in Evaluation: Boundaries, Governance and Utilisation.” Evidence & Policy 12 (1): 73–89. doi:10.1332/174426415X14298726247211
98. Westhorp, G., 2012. “Using Complexity-Consistent Theory for Evaluating Complex Systems.” Evaluation 18 (4): 405–420. doi:10.1177/1356389012460963
99. Westhorp, G., 2013. “Developing Complexity-Consistent Theory in a Realist Investigation.” Evaluation 19 (4): 364–382. doi:10.1177/1356389013505042
100. Whitworth, A., 2017. “Budget Cut Threat to Law Centre Advice Services.” Gloucester Review, February 2. http://www.gloucesterreview.co.uk/article.cfm?id=102315&headline=Budget%20cut%20threat%20to%20Law%20Centre%20advice%20services§ionIs=news&searchyear=2017