Research note. The (non-)use of ex-post legislative evaluations by the European Commission

by Stijn van Voorst and Pieter Zwaan

Since the early 2000s the European Commission has started to systematically produce ex-post legislative (EPL) evaluations: reports that assess the functioning of existing EU legislation. During the last decade these evaluations have increasingly grown in importance. For example, they have been linked to the Commission’s smart regulation agenda created in 2010 and its REFIT programme created in 2012, both of which attempt to simplify existing legislation and to cut back on unnecessary rules. Furthermore, in 2015 the Commission published new better regulation guidelines that include an extensive section about EPL evaluations.

Despite this growing importance of EPL evaluations in the day-to-day work and the rhetoric of the European Commission, they have received much less academic scrutiny than other types of evaluations of EU policies (like impact assessments). Our work seeks to change this by describing and explaining the variance in the use of these EPL evaluations.

In theory, EPL evaluations can help to improve policies by enhancing learning. Based on input from stakeholders and other sources, EPL evaluations aim to produce data about the way in which EU legislation is implemented at the national level, achieves its intended goals and/or has unintended side-effects. Such information can form the foundation for decisions that the EU institutions take about if and how the content of legislation should be amended.

However, for EPL evaluations to fulfil this crucial function of learning, their results must be used. As the extensive literature about such use shows, there are many evaluations of which the findings remain ignored by policy makers, either because of political interests or because the reports are lacking in quality. Political constraints are especially likely to be an issue for legislative evaluations, as legislation is more likely to affect a large share of the European public and to be discussed at the political level than other types of EU policies. Therefore, our paper focuses on the question to what extent and how the inherently political conditions in which the European Commission operates affect its use of EPL evaluations.

Our study is an in-depth analysis of the Commission’s use of three specific EPL evaluations: the evaluation of the EU’s seed and plant propagating material legislation (S&PM legislation) from 2008, the evaluation of the EU’s consumer protection cooperation (CPC) regulation from 2012 and the evaluation of the EU policy on animal welfare from 2010. The first report was used extensively by the Commission, the second report was partly used and the third report was not used at all, with use being defined as the extent to which the evaluation’s recommendations were followed-up by legislative proposals that were in line with these recommendations.

The selected evaluations are all of high methodological quality and all concern legislation which was under the responsibility of the same directorate-general (SANCO/SANTE), as controlling for these variables allowed us to focus on the specific impact of political conditions.

We collected our data via document analysis and semi-structured interviews. Commission proposals for legislative amendments and the documents leading up to them were studied to identify which of the evaluations’ suggestions had been followed-up and which suggestions had been dropped at what moment in time. Detailed explanations for these decisions were subsequently gathered via nineteen in-depth interviews with a broad variety of actors, including the civil servants from the Commission who managed the evaluations, the consultants who  conducted the evaluations and representatives of the European Parliament and interest groups involved in the policy file.

Based on existing literature about politics and evaluation use, we expected that the absence of opposition from major political actors to an evaluation’s findings would be a necessary condition for their use in subsequent policy processes. In other words, we predicted that the Commission would not follow-up on recommendations from EPL evaluations when the content of these recommendations would go against its own preferences or the preferences of actors that it depends on (like the European Parliament, the Council and major interest groups).

However, our findings contradicted this hypothesis. In the first two cases that we studied (the S&PM legislation evaluation and the CPC regulation evaluation) we found that the recommendations from the evaluations had been followed-up by the Commission despite significant opposition to them from respectively the European Parliament plus various NGOs and the Council. In our third case (the animal welfare evaluation) the results from the evaluation had not been followed-up by the Commission, but we did found no evidence that this was caused by opposition from influential actors to its specific findings.

Instead of the absence of opposition from major political actors being a necessary condition for use, we found a lack of salience of the policy field to which an EPL evaluation belongs to be a sufficient condition for non-use. This means that the Commission is unlikely to propose legislation that is not linked to its political priorities (which are currently economic growth, human rights and migration), even when an evaluation recommends to do so.

The cause of this is that the Juncker Commission that currently holds office has promised to reduce regulatory burdens for European citizens and companies and therefore tries to limit its legislative output. In policy areas that the Commission has selected as its priorities there appears to be room for the results of EPL evaluations to affect the content of legislative proposals, but in other areas EPL evaluations cannot change the fact that the Commission is unwilling to propose new legislation to begin with. This reveals a potential conflict between two aims of the Commission’s better regulation agenda: the goal to reduce regulatory burdens on the one hand and the goal of evidence-based policy on the other hand. When evidence from EPL evaluations or other sources recommends new legislation to be produced, these two aims may clash. In the current situation, the goal to limit legislative output seems to be prioritized.

  • Voorst, S. van & Zwaan, P. (2018). The (non-)use of ex-post legislative evaluations by the European Commission. Journal of Europese Public Policy. Epub ahead of print, 20 March 2018. DOI: 10.1080/13501763.2018.1449235

 

Stijn van Voorst is a PhD candidate at Tilburg Law School at Tilburg University, the Netherlands. He works on a four-year PhD project about ex-post legislative evaluations produced by the European Commission.

 

 

 

Pieter Zwaan is an assistant professor at the Institute for Management Research at Radboud University, Nijmegen, the Netherlands.