Evaluating the effects of labour policies, incentives to firms, and welfare policies: methodological issues and case-studies


Research Project supported by the Italian Ministry of Education and Scientific Research, 2003

[Home] [Versione italiana] [Printer-friendly version]

Projects | 1997 | 1999 | 2001 | 2003 | 2005 



This research project fits the strand in the literature placing an increasing relevance on micro-level analyses of the effects of public policies.
It builds on previous research supported by grants from MIUR (a project financed in 1997, on "Labour supply and unemployment: issues of measurement and analysis"; a second one financed in 1999 on "Employment and unemployment in Italy: measurement issues and behavioural analyses"; finally, an on going one financed in 2001 on “Dynamics and inertia in the Italian labour market and policies evaluation”), with U. Trivellato as project leader.

The distinct feature of the present project is

  1. its focus on the methodological framework of programme evaluation and

  2. the analysis of selected policies in three fields: labour market, incentives to firms, welfare.

In the following we place the programme in the relevant scientific background, mainly as regards the previous experience of the members of the research team. A brief link to the literature on programme evaluation follows, while references to the literature relevant for the case studies we plan to work on are kept to the bare essentials.

U. Trivellato, project leader, has been working on issues of measurement and analysis of the Italian labour market since the mid-'80s. Jointly with other researchers – some of them now in the team of this project –, beginning of the 1990s he went on with research on issues focusing on unemployment and its dynamics (flows and durations). Various contributions, both methodological and empirical, on the labour market attachment of not-employed persons and on unemployment dynamics been published in several well-known journals ("Economia & Lavoro", "European Economic Review", "Journal of Econometrics", "Labour", "Quality & Quantity", "Statistica", "Survey Methodology").
Starting with the research project co-financed by MIUR in 1997, the research interests of the team widened somewhat, chiefly in three directions: (i) exploring the potential for labour market analysis of Netlabor, a database resulting from the administrative process in the State-driven Labour Exchanges; (ii) extending the dynamic analyses to the entire set of labour force gross flows and to the relationships between (un)employment and living conditions (chiefly poverty); (ii) issues in monitoring and evaluating labour market programmes, with attention both to methodological issues and applications. On this last research theme, along with contributions mainly of a methodological flavour (Martini and Trivellato, 1997; Rettore and Trivellato, 1999; Rettore, Trivellato e Martini, 2002) there are several exercises of policy evaluation on selected labour market programmes implemented in Italy (Battistin, Gavosto and Rettore, 2001; Battistin and Rettore, 2001; Contini et al. 2002; Paggiaro and Trivellato, 2002).

B. Contini, directing the RU in Turin, has worked since the ’80s on the development and scientific utilisation of large longitudinal panels originating from administrative sources. The applications have ranged on numerous issues related to the dynamics of Italian labour market. He and the late R.Revelli developed the design and prototype of the INPS (the Italian Social Security institute) Observatory on Firms, Employment and Wages. He has co-authored (with R.Revelli) the first studies on industrial demography in Italy and many theoretical and empirical papers on labour market flows. See chiefly Contini and Revelli (1992) and Contini (2002). During the three just mentioned programs co-financed by MIUR, Contini has directed the development of a linked employer-employee longitudinal data-set based on SS data.

G. Tattara and the RU of Venezia have done substantial research on the characteristics of the system of industrial districts in the Veneto region and, more recently, on worker mobility in the same region using SS data (see Tattara, 2001, and Tattara and Valentini, 2003).

The members of the RU in Siena, directed by A. Lemmi, in the recent past and more intensively in the last two years have focused their scientific interests on the issues of individual behaviour in the labour market, moving from studies mainly devoted to the measurement of poverty and living conditions (see the papers by Betti , Cheli and Lemmi in the reference list).

The RU of Piemonte Orientale is directed by A. Martini, who holds a remarkable research experience in the area of social policy evaluation. From 1988 to 1998 he has been Research Economist at Mathematica Policy Research and then Senior Research Associate at the Urban Institute, Washington, D.C., where he has been responsible for various research contracts on US social policies, especially on impact evaluation of social and labour market policies (see Martini, 1993, and Martini and Trivellato, 1997). A high academic training on the evaluation of industrial policies, documented by a Ph.D., characterises the curriculum and the research interests of the other member of the RU, D. Bondonio.

The RU in Salerno is directed by Sergio Destefanis. His research interests include wage and price determination, growth and development in dualistic economies and the quantitative analysis of productive processes. Other recent work focuses on the efficiency analysis of non-conventional producers (municipalities, non-profit organisations), on the determination of the efficiency of manufacturing firms and on the quantitative analysis of production across countries and regions.
Coming to the four broad themes the project focuses on (see sec. 2.1), here we give a brief link to the literature on impact evaluation. Only a short list of references relevant for the case studies is given here (see the projects of the six RUs for more details).


How to evaluate the impact of public policies is a special case of the more general issue of how to measure causal links in an observational study: see Holland (1986) and Rosenbaum and Rubin (1983). The crucial point is how to control for confounding factors in the (likely) presence of major across individuals heterogeneity possibly related to the rule according to which some of the individuals become treatments while others become controls.
Over the last twenty years the statistical and econometric literature on the issue experienced a major growth. The state of the art is properly made in Manski (1995), Mohr (1995), Rosenbaum (2002), Angrist and Krueger (1999), Heckman, Lalonde e Smith (1999) Blundell and Costa Dias (2000).
Recent developments stress the need for nonparametric methods, namely methods avoiding to impose strong and often unwarranted restrictions on the functional form of the models specified to represent the phenomena under analysis this, way providing more robust inferences. Such methods call for rich longitudinal data set at the micro level providing information on both treatments and controls (a recent contribution on the links between evaluation methods and data information with some examples from the Italian case is Rettore, Trivellato and Martini, 2002).
On looking at the Italian experience, some recent studies did provide results in line with the international literature but overall the state of the art is not satisfactory (see the list of references herein and in the local RU projects). Most times even the public policies funded by the EU did not go through a proper impact evaluation despite the effort spent to produce what has been called an evaluation. In fact, a closer look reveals that the practice of evaluating UE funded policies amounts to accounting for how the money have been spent in line with an accountability principle but useless as a tool to produce knowledge on the policy efficacy. The lack of knowledge on the impact of public policies is even deeper when the policies are funded, designed and run at the local level. Most times even elementary information are missing (see Rettore and Trivellato, 1999 and Rettore, Trivellato and Martini, 2002).
Placing the current proposal in this context, the value emerges of one of its goals: showing how the data needed to properly monitoring and evaluating public policies could be obtained and based on those data providing sound empirical evidence on the impact of selected policies. Ultimately, the research project can turn out useful to: (i) (re-)designing policies taking into account what happened in the past experience; (ii) designing from the beginning the collection of the data needed for the evaluation of the policies; (iii) enhance behaviours from the main agents – policy makers, policy managers, analysts in charge of evaluating the policies – in the sense of improving the design and the evaluation of policies.


Basic references to the specific case studies the group will deal with are: Snower (1994), Calmfors and Lang (1995), Heckman, LaLonde and Smith (1999). Previous research work on these case studies are documented in Brunello and Miniaci (1997), Rettore and Trivellato (1999), Contini et al. (2002), Paggiaro and Trivellato (2001) and (2002), Rettore, Trivellato and Martini (2002), Ciollone and Guelfi (2003).


Business incentive programs for SMEs in the EU Ob.2 areas are similar to the US “Enterprise Zone” and “Empowerment Zone “ (Greenbaum and Bondonio 2003). Rigorous impact evaluations of US programs have been carried out with methods that mimic experimental conditions by offering some exogenous variation in the exposure to the program intervention (e.g. Papke 1994, Boarnet and Bogart 1996, Greenbaum and Engberg 2000, Bondonio and Engberg 2000).
As for policies providing incentives to investments, a few studies have referred to state support for firms in the Mezzogiorno and other depressed areas (e.g. Pellegrini, 1999; Ministero dell’Industria, 2000a; Bondonio, 2002), other studies have concerned themselves with the effects of R&D incentives (Ministero dell’Industria, 2000b), or with the effects of various types of incentives on the real and financial characteristics of firms (Lodde et al., 1993; Bagella and Becchetti, 1998).


The welfare policies the group will deal with did not go through any impact evalution so far. An assessment of the institutional context in which such policies take place and of their broad feature are presented and put into perspective in Saraceno (2002), Gauthier (2001), Kohler, Billari e Ortega (2002), Ministero del Lavoro e delle politiche Sociali 2002). A recent analysis of the impact of social policies on employment and welfare dependence is in Bolvig, Jensen e Rosholm (2003).



Info: anna.giraldo@unipd.it