This post was written by Ansa's Benedict Rickey who co-authored the report « L’expérimentation sociale à l’épreuve du terrain : un bilan d’une décennie d’expérimentations sociales en France ». He is part of Ansa’s homelessness team and leads Ansa’s work on evaluation. Prior to joining Ansa he worked in the UK for over five years as a researcher and consultant for New Philanthropy Capital and Shared Intelligence.
The words “French public services” are rarely associated with words like “innovation”, “impact” and “efficiency” in the minds of most foreign observers. However, there has been a quiet revolution happening in France.
Since 2005, social policy experimentation has established itself at the heart of the public policy landscape. Martin Hirsch has played a key role in this process. Working from the heart of Nicolas Sarkozy’s government from 2007 as Commissioner for Active Solidarity, he acted as an agitator for the introduction of this approach. From 2008, he led the team that experimented the new French minimum income scheme - Revenu de solidarité active (RSA). He then went on to kick-start a € 300 M youth experimentation fund aimed at pilot testing new approaches to addressing exclusion amongst young people. This wave of activity built support for the idea of social policy experimentation, whilst also building the capacity within France to put the method into practice.
However, how successful has this approach really been in France?
In theory, the approach is simple: test an innovation at small-scale, evaluate its effectiveness, and then scale it up if it works. In practice, it is incredibly difficult to follow this process from end to end, as Ansa found in its recent review of social policy experimentation in France.
Founded in 2006, Ansa is a think and do tank that tests innovative approaches to addressing poverty and social exclusion. Alongside organisations like J-Pal Europe, Ansa has been a key advocate of social policy experimentation. We have also been directly involved in setting up a number of social policy experiments, including the RSA. We therefore felt well placed to conduct this review of social policy experimentation in France. Ansa conducted a rapid literature review, prepared five case studies and ran an interactive workshop with people who had been directly involved in policy experimentation - mainly civil servants, evaluators, and local service managers.
The findings were mixed. On the one hand, the period saw a huge amount of activity – with several hundred social policy experiments launched over an eight year period. On the other hand, the approach met with serious challenges as it was put to the test on the ground; two of which stand out. Firstly, many social policy experiments met with serious obstacles in their set up and implementation. These included: difficulties in justifying high costs; the complexities of local project partnerships; the divergent timetables of service deliverers and evaluators; and evaluation methods that seemed onerous for certain projects. Secondly, few social policy experimentations were scaled up, even those that seemed to have worked. Clearly it is difficult to know whether the results of one pilot test could be reproduced in other areas or at national level. However, not enough effort was put into working out how to share findings and support the adoption of effective approaches amongst commissioners and professionals.
The report made recommendations for how to improve the use of social policy experimentation in future:
- Open the definition of experimentation to include pilot tests of varying sizes and levels of ambition.
- Support service delivery organisations running a social policy experimentation to set up and manage the complex partnership around the pilot, between the funder, service deliverer and evaluator.
- Find ways to better coordinate the project and evaluation calendars to ensure effective pilot projects don’t shut down before evaluation results are published.
- Diversify the evaluation methods used for social policy experimentation, including a wide range of experimental, quasi-experimental and non-experimental methods.
- Produce an “Evaluation charter” setting out principles regarding the role of evaluators and of evaluation in social policy experimentation, in particular to address ethical questions.
- Improve the transmission of findings from experiments – not just of the results achieved, but also the good practice identified and key lessons for policy makers. We feel much could be learned in France from the experience of the What Works Institutes in the UK.
- Promote the phased adoption of promising and effective initiatives, avoiding rapid national scale-ups, as these often struggle to reproduce the results of pilot projects at scale.
For Ansa, this review was the start of a new chapter in the story of social experimentation in France. Over the coming years, we will seek to follow-up on the report’s recommendations. We will also continue to promote social policy experimentation and its improvement. In short, we will do our best to ensure social policy experimentation fulfils its promise of better, more evidence-based public services.