Evaluation in the service of public policies

The faster a society is changing, the more important it is to evaluate the effectiveness of public policies over time. On the occasion of the conference “Conoscere per intervenire: dati, analisi e politiche per il welfare locale” (Know and act: Data, Analysis and Policies for Local Welfare), FBK-IRVAPP Director Mirco Tonin was invited to Modena to talk about the Trento experience. We asked him for an insight into the Institute’s research activities and how the culture of evaluation can benefit administrations and citizens.

Director Tonin, if you had to briefly explain what public policy evaluation is to those outside the field, what would you say?

Shakespeare has Hamlet say, “There are more things in heaven and earth, Horatio, than you can dream of in your philosophy.” Reality is often more complex than we can imagine, especially in times of constant innovation. Therefore, it becomes essential to go to the field to see whether policies that are designed to achieve a certain goal actually succeed in doing so. This is what evaluation is all about.

What benefits come from a culture of evaluation, for administrations and for citizens?

First, since resources are scarce, it is important to use them efficiently. Understanding what works and what doesn’t allows for better focus of interventions. It then triggers a learning process, whereby past evaluations allow me to better understand the social reality in which I operate, and thus to design more effective interventions in the future.

How does FBK’s IRVAPP in particular operate?

IRVAPP does exactly that, conducting both ex ante assessments, i.e. before the implementation of a given measure, and ex post, i.e. afterwards. Here, we use various methodologies to understand what would have happened in the absence of a given policy, the so-called counterfactual, to determine its actual impact through comparison with the situation observed following implementation.

At a recent conference in Modena, “Know and act: Data, Analysis and Policies for Local Welfare,” organized by the “Welfare Datalab” observatory, you were invited to present the Trento experience. In what areas can what is done in Trentino act as a model?

The city of Modena, in collaboration with various institutions in the area, has set up a laboratory aimed at sharing data and information that will help develop and improve local public welfare policies, an activity that IRVAPP has been doing for some time and on which it has accumulated experience and reputation. In addition, thanks to the increased competencies of the Autonomous Province, our evaluation activities extend beyond the field of welfare.

Can you give us some examples of IRVAPP’s activities both in welfare and in other fields?

Over the years, IRVAPP has evaluated the effects of income support, scholarships for university students, subsidies to bridge the digital divide, interventions to raise awareness about visual impairment, incentives to save for educational purposes, and the European Disability Charter, to name a few.

Are there other examples where the IRVAPP experience in Trentino is acting as a pioneer?

Yes, actually. This month we are holding a training course on policy impact evaluation for public administrators, where we discuss what questions can be asked of evaluators and how to interpret their answers to improve administrative action. This is a project funded by PNRR, within the MUSA ecosystem involving the main institutions in Lombardy and of which IRVAPP is also a part.

What do you hope for in the near future in the area of public policy evaluation and what will IRVAPP’s work focus on?

I think there is an increasing need for evaluation. While in the past an experienced official could have a reasonable idea of the effects of a given measure, this becomes progressively more difficult in societies that are increasingly complex and changing. IRVAPP is committed to continuing to provide society with the best methodologies available to carry out this work.