Introduction
‘Evidence-based policy’ has become something of a holy grail for policymakers intent on formulating pragmatic, effective policies which work on the basis of evidence rather than ideology. In the last decade it has increasingly been seen as both an indicator of and way of attaining ‘good governance’ across the world, and in developing countries in particular. The evidence-based policy agenda has been widely purported by international donors, who view it as a way of building the capacity of governments to formulate and implement pro-poor policies which target poverty reduction and ensure the responsible allocation of resources.
Also known as ‘evidence-informed’ policy, moves towards policies based on evidence have provided researchers, including those from think tanks, universities, and research institutes, with an opportunity to ensure that policies in developing countries are at least informed, to a degree, with some form of research-based evidence. Equally, pressure has been placed upon policymakers to move beyond policy expediency, and instead engage with research findings in order to arrive at decisions. The research-to-policy literature is vast, encompassing all policy sectors and all regions of the globe, and is therefore written from a number of perspectives for a variety of audiences, including researchers; policymakers; and donors.
This guide is concerned with the relationship between research and policy, and is predicated on the belief that policy should be based – or at least informed by – policy. It therefore presents some of theoretical basis upon which the evidence-based policy agenda is formed, as well as the relevant critiques of this approach. It considers, from the point of view of the researcher, how to influence policy using research, while offering guidance on how to approach the building of capacity in this field.
The Guide
This guide was prepared by a transnational team comprising of researchers from CIPPEC and Mendizabal Limited on behalf of GDNet. It is intended to provide a non-exhaustive and evolving overview of the literature which shapes and forms the current discussions concerning the research and policy interface; and identify a number of key issues for the Global Development Network (GDNet) to pursue further as part of its evolving research agenda in this area. The guide attempts, where possible, to showcase literature from rather than on developing countries, with a further call for additional southern literature in planned the near future.
The guide is organised under five key headings (Research agenda and prodution; Research in the policy process; The communication of research; The M&E of policy research; and capacity building) and a number of pertinent sub-topics identified by the research team. Key texts and perspectives on each of the issues have been selected and presented in summary form, alongside an introductory overview of each issue discussed.
The 12 topics presented reflect an iterative discussion which took place amongst the research team in November. Starting with a tentative list of potential areas to cover, based for instance on a review of contemporary debates amongst online communities, the team were able to collectively refine the focus and scope of the initial list. The issues presented in the guide reflect further refinement during the subsequent literature identification and review process.
Guided by a tentative list of issues, the research team undertook an extensive review of the literature in each area, with the aim of including between 10-15 summarised references. Literature was found on the following bases:
Prior knowledge, with team members responsible for areas they were most knowledgeable in;
Recommendations from experts (researchers and practitioners);
Sourcing from online communities and community depositories, such as GDNet and the Evidence-Based Policy in Development Network;
Literature searches, including journals and organizational websites.
In order to guide searches and selection of literature, a number of criteria were established. These included:
Types of literature: the inclusion criteria included policy research papers, journal articles, evaluations, policy briefs, blog articles, magazine articles, workshop reports, conference proceedings, online community discussions.
Quality: is the document/article well-written, accessible, and well-referenced?
Source credibility: references in other literature; affiliation of author
Contribution and relation to the debate: does the document/article offer a new perspective or offer a useful summary of the debate?
Guidance: does the document/article provide tools and/or guidance for researchers and/or communicators of research?
Language: English or Spanish, with translations where possible
Southern origin: is the document/article written by a southern author or does it offer a southern perspective?
Some documents appear more than once due to their cross-cutting nature. It is also important to note that the guide is evolving, with relevant literature being added on a rolling basis.
Research and policy: An overview
The term ‘evidence-based policy’ gained political currency in the UK under the Blair administration in 1997, intended to herald a modernizing, forward-thinking government who invited ‘outside’ experts to inform policy, based on the premise that it is desirable for policy to be formed on the basis of a review of evidence and rational analysis. In the last decade the concept has become increasingly prominent in relation to developing countries, where policy based on evidence is thought to be even more significant due to the greater potential for change than in their developed country counterparts. The received wisdom surrounding the evidence-based policy agenda is that in order to reduce poverty and accelerate economic growth adequate policies need to be in place; and that a better utilization of evidence is required in the formulation of these policies.
Although there is considerable debate as to what counts as ‘evidence’, evidence-based policy is often characterized by a political environment in which the links between research and policy (or researchers and policymakers) are strong. In developing countries, research often plays a weak role in policymaking, and as a result policies are not well-targeted, sufficiently-resourced, appropriate or timely. Carden (2009) identifies a number of reasons for this, citing factors such as non-inclusive governments with short democratic precedent; the role of donor actors in constraining the autonomy of policymakers in the policy process; large staff turnovers in both research and government leading to weak relationships and also ‘brain drain’ to foreign countries with better prospects; the personalization of politics, in which corrupt practices rather than evidence determine policy decisions; researchers lacking legitimacy amongst policymakers; a lack of hard data for researchers to use; a lack of demand for research; institutionalized interactions between researchers and policymakers are missing; and there is little use of intermediaries to transfer research to policymakers. These problems are often conceived as relating to three over-arching factors: problems relating to the supply of research; problems relating to demand for research; and problems relating to bridging the ‘gap’ in-between.
As an introduction to the discussions it is worth presenting a number of key insights which best represent the starting point of many current research-to-policy debates, and which the guide will further extrapolate:
There is no ‘best practice’ when it comes to linking research to the policy process. Policy influence is a process rather than a product, involving multiple relationships and various activities. Influencing policy must be understood as a way of achieving an objective rather than an objective in-itself. It takes time and a long-term strategy.
Though this may not be desirable, the reality is that research is not the only one element in what is a fiercely complicated mix of factors and forces behind policy decisions. Policies are often far from a government’s original intention or design. Engaging with the policy process means also engaging with this reality (Mendizabal, 2012) (Carden, 2009).
For this reason, it is a mistake to approach policy as a rational, orderly, or unitary and linear progression from the statement of a problem to a decision and solution. The formation of policy is actually much more complex and non-linear (Sutton, 1999).
It is important to acknowledge that ‘research’ is not pure, monolithic, or singleminded. Research is infused with diverse intentions, motives, and expectations.
Analyzing the factors which influence the relationship between research and policy is important in understanding how the two relate. A much-cited framework has been developed by the Research and Policy in Development (RAPID) programme at the Overseas Development Institute, citing three overlapping areas: the political context; the evidence; and the links between policy and research communities, all within a fourth set of factors: the external context (Court et al, 2004).
Resources
Sutton, R. (1999). The Policy Process: An Overview. London: Overseas Development Institute.
This ODI paper presents an important introduction to past and current understandings of the policy process, how policy is formed, and how policy changes. Its main contention is that the traditional ‘linear’ view of policymaking does not reflect policy realities which alternative approaches are based. The ‘linear’ or rational model consists of a number of stages which include: recognizing and defining the nature of the issue to be dealt with; identifying possible courses of action to deal with the issue; weighing up the advantages and disadvantages of each of these alternatives; choosing the option which offers the best solution; implementing the policy; and evaluating the outcome. The paper then reviews five cross-cutting themes: (a) the dichotomy between policy-making and implementation; (b) the management of change; (c) the role of interest groups in the policy process; (d) ownership of the policy process; and (e) the narrowing of policy alternatives.
Solesbury, W. (2001). Evidence Based Policy: Whence it Came and Where it’s Going. ESRC UK Centre for Evidence Based Policy and Practice. London: Queen Mary University.
This concept of ‘evidence-based policy’ has been gaining currency over the last decade, particularly as the agenda has moved on to a concern with policy delivery as much as with its policy development. From the research perspective, more particularly that of social science, there has been a drive by funders of research for researchers to undertake what is considered ‘useful’ research, research that helps us not just to understand society but offers some guidance on how to make it better. This has come with a realization that to be useful research must be usable. Thus, academic researchers are increasingly engaging in ways that users of research find helpful. For instance, how to structure a report, write in plain English, and make a five minute presentation are now seen to be as important as how to design a questionnaire, conduct an interview or analyze data. To a large extent this can be attributed to the arrival of the pragmatic and non-ideological government of New Labour in the UK in 1997, when civil servants were encouraged to open up the policy process to outsiders to provide ‘evidence’ (wider than ‘research’). Evidence is no doubt an important part of the weaponry of those engaged in policy discourse, but is a weapon that must be used with care as with other sources of power.
Sophie Sutcliffe and Julius Court (2005). Evidence-Based Policymaking: What is it? How does it work? What relevance for developing countries? London: ODI.
This paper considers the origins of the evidence-based policy concept and the factors which contribute to its continued use. It also highlights three main issues surrounding the use of evidence-based policy (EBP):(i) What evidence is used in the policymaking process?; (ii) How is evidence incorporated into policymaking; and (iii) factors other than evidence that influence policymaking. It is important to acknowledge that at each stage of the policy cycle, a number of different factors will also affect policy. This occurs at an individual level – for example, a policymaker’s own experience, expertise and judgement; and at an institutional level, for example in terms of institutional capacity. There are also a number of constraints to the influence of research on policy – for example, the pressure to process information quickly. However, despite there being challenges to creating an EBP approach, it is generally seen as a desirable approach to achieve development outcomes. Policymakers need to develop their capacities to use evidence, and in the UK government a number of tools are employed to this end. These include impact assessments and appraisals; risk assessments; gender mainstreaming; policy pilot studies; and community engagement. However, the authors note that evidence-based policy is not always directly transferable to developing countries, as capacity is more limited and resources scarce. Thus, approaches need to be adapted in order to ensure relevance to political realities.
Enrique Mendizabal (2012). ‘Politics of the evidence-based policy mantra’. Onthinktanks blogpost. October 17th 2012.
In this blog article the author considers criticisms of the evidence-based policy ‘mantra’. Concerns regarding the discourse include that it is anti-democratic, narrowing participation down to ‘experts’ rather than the public; that it is difficult to ‘export and attempt to apply’ the evidence-based policy ideal to all contexts where institutional structures simply do not exist; the assumption that evidence-based policy works ‘better’ to deliver measurable development outcomes, ignoring other factors such as political repression and human rights abuses; and finally, that actually evidence-based policy approached make it harder to think about policy in a nuanced way limiting the kinds of questions being asked about the actual factors that contribute to policy decisions. The author concludes by arguing that “evidence based policy is as ideological as the very approaches to policymaking it seeks to discredit.”
Court, J., Hovland, I., and Young, J. (2004). Chapter 1: “Research and policy in international development: introduction.” In Bridging Research and Policy in Development. Rugby: ITDG Publishing.
(Book form only)
Related brief: Access online
This introduction precedes a number of research-to-policy case studies based on the Research and Policy in Development (RAPID) team’s work, which is premised upon the belief that a better use of research-based evidence in development policy and practice can help save lives, reduce poverty and improve quality of life. However, researchers often fail to see the importance of ‘influencing’ and communicating their work. For researchers to do this they must develop a detailed understanding of the policymaking process; the nature of evidence they have or hope to get; and all the other stakeholders involved in the policy area who could help communicate their work. The ‘Context, Evidence, Links’ framework developed by RAPID helps researchers do these things. In the course of their work, the RAPID team have replaced the traditional question of ‘How can research be transported from the research to the policy sphere?’ with a more complex question: ‘Why are some of the ideas that circulate in the research/policy networks picked up and acted on, while others are ignored and disappear?’.
Carden, F. (2009). Knowledge to Policy: Making the most out of development research. Ottawa: IDRC.
The findings of this study are based on 23 case studies borne out of the IDRC’s work on getting research into policy in developing countries. The case for evidence-based policy is made on the basis that development is based on good governance, which the IDRC’s work on improving the role of knowledge for sustainable and democratic development attempts to inform. Simply put, to improve lives, especially the lives of poor people, development research has to influence policy in order to influence development. The process by which they do this is less clear, however. The collection of case studies allow for a number of identifiable conclusions, the most important being that research can contribute to better governance in a number of ways: first, research encourages open inquiry and debate; second, it empowers people with the knowledge to hold governments accountable; and third, research enlarges the array of policy options and solutions available to the policy process. Thus, when research is well designed, executed, skillfully communicated, it can inform policy that is more effective, more efficient, and more equitable. Yet experience also leads to another conclusion: development research frequently fails to register any apparent influence. Secondly, the case studies show that things change, both in research and in policymaking, and research projects must adapt to their changing surroundings.