Policy labs have emerged globally with a mission to support public policy development through innovative solutions. However, it is unclear to what extent policy labs provide new ways of addressing challenges.
A paper in Policy & Politics explores the role of policy labs and assesses the extent to which their purpose, structures and processes address three challenges:
The article concludes that creating synergies between evaluation inquiry and policy labs can improve the design and implementation of public policy and programs.
The study focused on labs with a focus on:
The research covered 20 labs in 16 countries across Europe, North America, Latin America and Asia, including the NSW Behavioural Insights Unit and the Auckland Co-Design Lab.
Half of the selected labs are exclusively owned by public sector entities representing different levels of governance. The labs operate at local, regional and national levels and some are also active at the international level.
The experience of the labs varies from two to 23 years. Labs’ regular partners include local and regional authorities, public administration institutions and private stakeholders. Civil society organisations are also a key partner for labs as they represent wider groups of citizens or specific social interests.
Three main lab specialisations were identified:
The study found limited evidence on labs’ longer-term impact and the measurement of meaningful change. Instead, labs focused on the short-term and in some cases, process-related outcomes.
The processes and toolboxes of policy labs are heavily based on design practice including design thinking and industrial design. The study mapped labs’ activities and found five main areas accompanied by a variety of tools and techniques.
The five areas are:
All labs use participatory orientation, involving representatives of various stakeholders in the exploration and co-creation of solutions. Methods include ethnographies, interviews, workshops and crowdsourcing. This human-centred approach welcomes citizens’ ideas and solutions, embraces ambiguity and identifies the conditions for co-design and implementation. Some labs also highlight the need to involve participants during the prototyping and testing stage to increase the take up of solutions and empower beneficiaries.
Behavioural insights was another methodology used. This uses insights about human motivation and decision-making to redefine policy problems. Exploration involves mapping behavioural bottlenecks - the factors that obstruct desired behaviours. Creation uses behavioural change strategies in the design of potential solutions. The testing stage relies on small, low cost interventions accompanied by experiments, usually randomised control trials.
1. The challenge of establishing what solution works
Looking at current lab practices, the study found labs have little to contribute to the ongoing debate addressing the challenges of valuing interventions. They could benefit from incorporating existing knowledge from evaluation into their practices as evaluation practice offers a high level of emphasis on obtaining strong causality.
2. Explaining why a solution works or not
Both evaluation practice and labs use a theory-driven approach that defines public intervention as a ‘theory’ – a hypothesis yet to be tested and verified in a real-life situation. The value of the theory-driven approach is its explanatory dimension. It reveals how a program works, with whom and under what circumstances.
The use of behavioural insights also helps explain why a solution works or not. The basic strategy is to break down the ‘journey’ of the policy actor throughout the intervention into a sequence of interactions and decisions, and then analyse behavioural bottlenecks. This approach:
3. The challenge of transferring research findings into policy actions
Labs’ scope of operations provides an opportunity to better utilise research by addressing the issue of timing. Most labs focus on working at the early stages of the policymaking process – problem definition and policy formulation. Within the main policy cycle, labs often try to build a smaller loop of design–testing–adaptation. This allows them to deliver meaningful feedback on ‘what works and why’ exactly when decision-makers need it – early in the policy process.
Well-established evaluation practices can support labs in tackling issues that require a more sophisticated understanding of the social dynamics and environments surrounding a certain policy intervention. In particular, evaluation brings together various perspectives of causality and complexity theories that can help labs measure their impact and understand how systems change over time as well as from place to place.
Lab practices can add to evaluation practice with their strong focus on the final users of policy. Some labs seem to be ahead of evaluation practice in terms of integrating behavioural insights into public policy. Policy labs seem to be better equipped than evaluation in addressing the timing issue of delivering results to knowledge users.
Policy labs: the next frontier of policy design and evaluation? – Karol Olejniczak, Borkowska- Sylwia Waszak, Anna Domaradzka-Widła and Yaerin Park, Policy & Politics, Volume 48, Number 1, January 2020
Each fortnight The Bridge summarises a recent piece of academic research of relevance to public sector managers.
Sign up for the Bridge