Assessing the accuracy of ex ante evaluation through feedback research: A case study

The Impact of Legislation: A Critical Analysis of Ex Ante Evaluation

Volumn , Issue , 2009, Pages 199-228

  van Gestel, Rob ^a,b,c   Vranken, Jan ^d  

^a TILBURG UNIVERSITY   (Netherlands)

^b Dutch Association of Legislation   (Netherlands)

^c European Academy of Legislation   (Netherlands)

^d NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77950371109     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1163/ej.9789004170209.i-280     Document Type: Chapter

References (72)

1. R.W. Bauman and T. Kahana, Th e Least Examined Branch: Th e Role of Legislatures in the Constitutional State (Cambridge University Press, Cambridge 2006).
2. J.Th .J. van den Berg, J.L.W. Broeksteeg and L.F.M. Verhey (eds.), Het parlement, Staatsrechtconferentie 2006 (Wolf Legal Publishers, Nijmegen 2007).
3. Th ese words are taken from one of the few Dutch scholars who have done research on the pre-Parliamentary phase of the legislative process
4. L.J.M. d'Anjou, Actoren en factoren in het wetgevingsproces (W.E.J. Tjeenk Willink, Zwolle 1986), p. 274
5. Not much has been written on the methodology of feedback research. An exception is: E.H. Mory, 'Feedback Research Revisited' in H. Jonassen (eds.), Handbook of Research on Educational Communications and Technology (Lawrence Earlbaum Associates, Mahwah 2004), pp. 745-784.
6. where it is stated that feedback research allows the comparison of actual performance with some set standard of performance. Mory argues that information presented via feedback in instruction might include not only answer correctness, but other information such as precision, timeliness, learning guidance, motivational messages, lesson sequence advisement, critical comparisons, and learning focus
7. A detailed overview of the advisory tasks of the Council of State can be found in T. Borman, Wetgevingsadvisering door de Raad van State in Nederland (W.E.J. Tjeenk Willink, Deventer 2000).
8. C. Kortmann, J. Fleuren and M. Kindlova, Constitutional Law of 10 EU Member States: Th e 2004 Enlargement (Kluwer Law International, Deventer 2006).
9. www.conseil-etat.fr/ce/home/index.shtml. More information on the competences of different Councils of State in Europe can be found through the European association of Councils of State and Supreme Administrative Jurisdictions of which the secretariat is established at the Belgium Council of State www.juradmin.eu/en/home_en.html
10. Th e text of the Dutch Constitution can be found at the website of the Ministry of the Interior and Kingdom Relations:www.minbzk.nl/contents/pages/6156/grondwet_UK_6-02.pdf
11. www.raadvanstate.nl
12. www.actal.nl
13. More information about the Social and Economic Council of the Netherlands can be found in English at www.ser.nl/en.aspx Th e same goes for the National Health Council at www.gr.nl/index.php
14. Kamerstukken II 1990-1991, 22008, nos. 1-2 (available in English)) and in the so-called Aanwijzingen voor de regelgeving, which are available in English as "Directives on legislation" at www.kc-wetgeving.nl/ (the website has restricted access but a guest account can be obtained through the Ministry of Justice). Th e Directives on legislation are also on archive with the authors (r.a.j.vanGestel@uvt.nl and j.b.m.vranken@uvt.nl).
15. Published in the Council of State's Annual Report 2003, Annex 4, pp. 151-152, and also available at www.raadvanstate.nl (English).
16. See Article 25a of the Council of State Act 1962 (Wet op de Raad van State van 9 maart 1962).
17. Th is information is drawn from previous empirical research concerning the way the Council of State uses its advisory competence. In general, all these reports conclude that the Council relies heavily on intuition, experience, and practical wisdom
18. Ph. Eijlander and W. Voermans, Evaluatie regeerakkoord 1998 uit wetgevingsperspectief (Tilburg 2002) till .uvt.nl/articles/voermans/internet.pdf
19. B.R. Dorbeck-Jung, Beelden over de wetgevingsadvisering van de Raad van State. Hoe wetgevingskwaliteit gemaakt wordt (BJU, Den Haag 2003)
20. J.L.W. Broeksteeg, E.M.J. Hardy, S. Klosse, M.G.W.M. Peeters and L.F.M. Verhey, Zicht op wetgevingskwaliteit: een onderzoek naar de wetgevingsadvisering van de Raad van State (WODC, Den Haag 2005 www.wodc.nl
21. See the article written by the Vice-President of the Council of State on the policy-analytical review: H.D. Tjeenk Willink, 'De beleidsanalytische toetsing door de Raad van State' (2005) 2 RegelMaat, pp. 51-59.
22. About ex post evaluation of legislation: U. Karpen (ed.), Evaluation of Legislation, Proceedings of the Fourth Congress of the European Association of Legislationx (Nomos, Den Haag 2002).
23. H.B Winter, Evaluatie van wetgeving: Structurering en institutionalisering van wetsevaluatie in Nederland (Kluwer, Deventer 2002)
24. K. Van Aeken, Proeven van wetsevaluatie (dissertation) (Universiteit van Antwerpen, Antwerpen 2002)
25. L. Mader, L'evaluation legislative: pour une analyse empirique des eff ets de la legislation (Payot, Lausanne 1985).
26. See Chapter 7 of the General Administrative Law Act concerning objections and administrative appeal. Th e Act can be found in English at www.justitie.nl/onderwerpen/wetgeving/awb/Wettekst_awb/index.aspx
27. Wet van 22 april 2004, houdende regeling van de aanspraak op, de toegang tot en de bekostiging van jeugdzorg (Wet op de jeugdzorg), Stb.(Netherlands Bulletin of Acts and Decrees) 2004, 306 (for a number of corrections, see Stb. 2004, 700).
28. Het kind en de rekening. Eindrapportage van de werkgroep IBO fi nanciering jeugdbeleid, 2005-2006, no. 2.
29. Kamerstukken II 28 168, F (Senate).
30. Kamerstukken II 28 168, A (the Council's advice and the 'Nader Rapport'). Items 1-5 and 8 concern the political-analytical aspects.
31. Item 1 of the Council's advice (Kamerstukken II 28 168, A).
32. Item 2 of the Council's advice (Kamerstukken II 28 168, A).
33. Kamerstukken II 28168, A ('Nader Rapport', answers to items 1-5 and 8).
34. Bestuur, Management Consultants, Evaluatieonderzoek Wet op de Jeugdzorg. Eindrapport (Evaluation Youth Assistance Act. Final report), November 2006.
35. In a letter to the Parliament (DJB/JZ-2721942).
36. Wet kosten bestuurlijke voorprocedures van 24 januari 2002, Stb. 2002, 55.
37. Verslag Van De Commissie evaluatie algemene wet bestuursrecht, Toepassing en eff ecten van de Algemene wet bestuursrecht 1994-1996, Den Haag 18 december 1996
38. Kamerstukken II 1997-1998, 25 600 VI, no. 46, p. 40.
39. H.B. Winter, P.O. de Jong and R. van Dijk, Evaluatie Wet kosten bestuurlijke voorprocedures (Groningen 2004). Th e full report is available at rechten.eldoc.ub.rug.nl/FILES/departments/Algemeen/Recht1/2005/evaluatieWkbv/EvaluatieWkbv.pdf
40. R. Ortlep and R.J.G.M. Widdershoven, 'De Wet kosten bestuurlijke voorprocedures in de rechtspraak' (2005) JB plus, pp. 147-167.
41. Th ey draw attention to publications such as J.J. Simon, 'Kosten van voorprocedures' (1998) JB, pp. 68 et seq
42. R.J. Tjittes, 'Bestuurlijke proeftuin' (1998) RM Th emis, pp. 1-2
43. N.S.J. Koeman, 'Proceskosten voorprocedures: voorontwerp gepresenteerd' (1998/9) NTB, pp. 290-293
44. Van Ravels (1998) Bouwrecht, p. 522.
45. Th is remark refers to the case law of the ECtHR, Procola vs. Luxembourg judgment of 28 September 1995, Series A, no. 326 (14570/89), concerning the double mandate of the (Luxembourg) Council of State in relation to the requirement of judicial independency as laid down in Article 6 of the ECHR.
46. Kamerstukken II 2000-2001, 27 024, nr. 14.
47. Kamerstukken II 2001-2002, 27 024, nr. 17.
48. In that fi le, the Council of State missed the openness to critically assess its own case law. For the methodological pitfalls concerning ex ante evaluation
49. F.L. Leeuw, 'Reconstructing Program Th eories: Methods Available and Problems to be Solved' (2003) 24:1 American Journal of Evaluation, pp. 5-20.
50. Th e idea that ex ante assessments of legislative drafts can simply serve as a way of speaking truth to power is refuted by, for instance, A. Meuwese, Impact Assessment in EU Lawmaking (dissertation, Leiden 2008).
51. T.C. Borman, 'Zeven stellingen over de (herstructurering van de) Raad van State en de wetgevingskwaliteit' in T. Barkhuysen, T. Borman and R. van Gestel, De wetgevingsadvisering door de Raad van State (BJU, Den Haag 2008), p. 7.
52. C.H. Weiss, 'Th e Interface between Evaluation and Public Policy' (1999) Evaluation, pp. 468-486.
53. P. Eijlander, 'Het wetgevingsbeleid na de bruikbare rechtsorde', in: P. Popelier and Ph. Eijlander, Preadviezen Vereniging voor wetgeving en wetgevingsbeleid 2007 (BJU, Den Haag 2008), p. 52
54. See also the call from the Senate in its latest annual report to make more use of experiments and sunset clauses in the process of drafting new legislation.
55. http://www.eerstekamer.nl/
56. R. Baldwin, 'Is Better Regulation Smarter Regulation?' Public Law (Autumn 2005), pp. 485-511
57. S. Jacobs, Current Trends in Regulatory Impact Analysis: Th e Challenges of Mainstreaming RIA in to Policymaking (Jacob & Associates Inc., Washington DC 2006
58. For an overview of the international literature on this and other common problems in ex ante evaluation
59. SERV, Reguleringsimpactanalyse in Vlaanderen, Evaluatie en aanbevelingen (Brussel, 22 November 2006).
60. J. Hertin, J. Turnpenny, A. Jordan, M. Nilsson, D. Russel, and B. Nykvist, Rationalising the Policy Mess? Ex Ante Policy Assessment and the Utilisation of Knowledge in the Policy Process, FFU report (Freie Universitat Berlin, Fachbereich Politik- und Socialwissenschaften, Berlin 2007)
61. With diff erent types of laws, we mean temporary versus permanent legislation, politically sensitive laws versus more technical ones, strictly national laws and regulations versus bills that are used to implement European directives at a member state level, etcetera
62. See also, in this volume, chapters 2, 3, 4 and 5.
63. R. Pawson, Evidence-Based Policy: A Realist Perspective (SAGE, London 2006), p. 8.
64. R.A.J. Van Gestel, Evidence-based Lawmaking and the Quality of Legislation: Regulatory Impact Assessments in the European Union and the Netherlands, in: H. Schaff er and J. Iliopoulos-Strangas (eds.), State Modernization in Europe (Ant. N. Sakkoulas-Berliner Wissenschafts-Verlag-Bruylant 2007), pp. 139-165.
65. C. Kirkpatrick and D. Parker (eds.), Regulatory Impact Assessment. Towards Better Regulation? (Edward Elgar, Cheltenham 2007).
66. F. Leeuw, 'Trends and Developments in Program Evaluation in General and Criminal Justice Programs in Particular' (2005) 11 European Journal on Criminal Policy and Research, pp. 18-35.
67. See also, in this volume, chapters 2, 3, 4 and 5.
68. R. Pawson, Evidence-Based Policy: A Realist Perspective (SAGE, London 2006), p. 8.
69. R.A.J. van Gestel, Evidence-based Lawmaking and the Quality of Legislation: Regulatory Impact Assessments in the European Union and the Netherlands, in: H. Schaff er and J. Iliopoulos-Strangas (eds.), State Modernization in Europe (Ant. N. Sakkoulas-Berliner Wissenschafts-Verlag-Bruylant 2007), pp. 139-165.
70. C. Kirkpatrick and D. Parker (eds.), Regulatory Impact Assessment. Towards Better Regulation? (Edward Elgar, Cheltenham 2007).
71. F. Leeuw, 'Trends and Developments in Program Evaluation in General and Criminal Justice Programs in Particular' (2005) 11 European Journal on Criminal Policy and Research, pp. 18-35.
72. The case of the compensation of costs in the administrative objection procedure can serve as an example here. In retrospect both the commission who prepared the working draft as the Council of State itself focused to much on the case law and overlooked signs indicting that juridifi cation of the objection procedure might not be a realistic threat.