OECD iREG indicators: a brief note

by Fabrizio De Francesco and Moritz Wassum, University of Strathclyde, Glasgow


Image credits: OECD, 2015 Indicators of Regulatory Policy and Governance: Design, Methodology and Key Results

(Arndt, C. et al. (2015), 2015 Indicators of Regulatory Policy and Governance: Design, Methodology and Key Results, OECD Regulatory Policy Working Papers, No. 1, OECD Publishing, Paris.). The acronym iREG denotes a new generation of indicators for comparatively assessing on a regular basis the quality of regulatory governance among OECD member states. Built on the 2012 OECD recommendations on regulatory policy and governance, iREG indicators are composed of three sub-indices capturing the three main principles of regulatory governance: regulatory impact analysis (RIA), stakeholder engagement and ex post evaluation. Complementing the broader OECD’s assessment of member states’ regulatory policy summarised in the annual publication  “Regulatory Policy Outlook”, the data present the situation at the end of 2014 of organisational and institutional features “to design, enforce and revise regulations”. The innovativeness of these indicators is about the careful consideration of conceptual and methodological issues in their design and use as described in an OECD working paper. Christiane Arndt and her colleagues at the OECD were able to address two key issues in the construction of indicators: the double counting of items and weighting attribution.

The remainder of this note is structured as follows. The next section illustrates the conceptual background and the methodological advancement of this new generation of indicators of regulatory management. Then we compare these indicators to previous OECD indicators of regulatory management, as well as other OECD indicators measuring the extent of regulatory reform. This comparative assessment of way of measuring regulatory governance is complemented by an overview to the recent World Bank’s “Global indicators of regulatory governance”. Finally, we conclude with a comparative assessment of this new generation of indicators from the perspective of researchers and policymakers.

Conceptual background and methodological advancements of iREG

The OECD’s iREGs are based on a questionnaire which has been send to national public officials as well as the European Commission. The 2015 iREGs saw some major methodological advancements. Based on the 2012 Recommendation of the Council the OECD structured the questions around three clear categories (stakeholder engagement, Regulatory Impact Assessment (RIA), and ex post evaluation) which again were classified into four sub-categories: systematic adoption, methodology, oversight and quality control, and transparency. Taking the example of stakeholder engagement, the OECD not only gathered information on whether countries engage in consultation with the public but also on the frequency of public consultations, the forms (internet, fora, closed or open meetings) of consultation practices, the transparency of information about those consultations, and at which stages of the legislative process the public is consulted. In order to test the robustness of the methodology used for the indicators the OECD conducted different sensitivity analyses, including principal components analysis, Monte Carlo analysis, and weight-sensitivity analysis to avoid unintended double-counting and improve the weighting-schemes.

Comparing iREG with previous OECD indicators of regulatory governance

Since the OECD initiated its work on the regulatory policy of its member states as well as some key partner countries in the mid-1990s it has collected five rounds of surveys (1998, 2005, 2008, 2009, and 2014). The most recent survey of 2014 was used for constructing the 2015 Indicators of Regulatory Policy and Governance (iREG). Compared to previous indicators, the OECD improved some methodological issues (robustness tests) and refined the questionnaire, especially the parts shedding light on the extent to which governments engage in cost-benefit analysis of regulations, conduct ex-post evaluations, and ensure stakeholder engagement within the whole policy process. Overall the iREGs became more structured and focused. The OECD attempts to conduct the iREGs in their form of 2015 on a regular basis which would enable longitudinal time-series analysis in the future. On the political level the OECD moved away from attempting to reduce the scale of government to a focus on improving the regulatory policy process. This may include a reduction in regulations that impose an unnecessary burden on business and civil society but not necessarily has to. The difference is that the new approach focuses on improving the mechanisms and techniques used for cost-benefit calculations of existing and future regulations (concerning their efficiency and effectivity).

Comparing iREG with other OECD indicators of regulatory reform

The OECD has also produced several other indicators for assessing the extent of government regulation for controlling product market, professional and retail services, utilities and for ensuring employment protection. All these indicators have a common methodology: Composite indicators are calculated as weighted averages of sub-indexes and vary between 0 and 6. Such a methodology increases the legibility of the OECD indicators and allows a consistent data collection across time. These elements are extremely useful in comparative policy analysis.

Comparing iREG with the World Bank’s Global indicators of regulatory governance

As discussed in more detail elsewhere, international organizations can be seen as competing for the market on governance indicators. The World Bank has now published its Global indicators of regulatory governance that similarly to the OECD’s IReg assess the institutional quality of a given country’s rulemaking. The World Bank focuses on five institutional dimensions: transparency, public consultation, impact assessment, access to law and regulation, and administrative remedies and judicial review. Relying on expertise from public and private sector, the survey items concern both legal requirements and administrative practices. Researchers may appreciate the reference to administrative legislations that is useful for finding the primary sources and tracing the change in regulatory governance in a given country.

In line with rational theories of democratic decision-making both the OECD’s IReg and the World Bank’s Global indicators of regulatory governance are based on the perspective that the regulatory quality can be improved if the participation of potentially affected social groups in the decision-making process is enhanced and impacts of regulations are assessed more carefully. While the OECD focuses on a detailed assessment of best practices concerning consultation, impact assessment and ex post evaluation, the World Bank revolves around a more comprehensive institutional quality of rulemaking that includes administrative remedies and judicial review.

Concluding remarks

This brief note has compared the new OECD IReg with other global regulatory governance indicators in term of conceptual approach and methodology. Both researchers and policymakers can benefit from the variety of supply in the “market” of regulatory governance indicators. After producing measures for assessing the extent of regulation, both the OECD and the World Bank are paying increasingly attention to the institutional quality of rulemaking. But differently from indicators of the extent of regulation, consistency in the aggregation methodology and longitudinal data collection is still need to be achieved.

(Fabrizio De Francesco and Moritz Wassum)