Research within the policy process
In this section:
The political economy of research uptake
As the discussion in Section 1 demonstrated, the relationship between research and policy is not free from the complexities of financial incentives and funding relationships. However the question of the ‘political economy’ of research uptake is not only an economic discussion; it also encompasses (inter-related) issues relating to culture, social norms, and beliefs. While these are arguably not always readily separated from economic concerns, such a reductionist view can lead to analysis which lacks depth (Broadbent, 2012).
This literature in this section delves further into the ‘politics’ of how research is produced, used, and understood in policy. Principally, three pertinent issues are reflected:
The ‘politics’ of the evidence-based policy agenda
Different types of ‘knowledge’/evidence
Power relations in policy decisions and their effects
The foundations upon which the promotion of ‘evidence-based policy’ (now sometimes referred to as ‘evidence-informed policy’) is based have always been subject to some debate, particularly in regard to the seeming trumping of ‘western’ rational, scientific knowledge over knowledge generated, understood, and used in developing country policy contexts. The normative assumptions which underlie the desire to ensure policy is based on ‘evidence’ are not, of course, only operational in donor approaches to recipient government policy; ‘evidence-based policy’ is a mantra which percolates policy discussion in donor countries, too.
Stone (2001) draws attention to the difficulty of isolating what ‘policy’ is, arguing that the varied interpretations of policy lead to quite different perspectives on how to support research-policy relations; while Broadbent (2012) critically assesses the evidence-based policy agenda’s assumptions, suggesting that –ironically– the idea currently lacks a rigorous evidence base itself. Further, it is argued that initiatives to enhance the role of research in policy ignore the very real incentives that exist within developing country policy environments not to use research (Igwe, 2012). The most recent and vehement critique comes from du Toit (2012) whose eloquent discussion on how the evidence-based policy agenda’s assumptions present a misleading understanding of the policy process as rational and linear. Based on the South African experience, du Toit examines how the very different meanings placed upon evidence make it very difficult to hold a unified discussion on evidence-based policy.
Multiple meanings, interpretations and perspectives arguably lie at the heart of the discussion on the politics of research in the policy process, taking the issue out of the purely economic realm. As Broadbent (2012) shows, policy debates in Africa involve a number of different (though unreflected-upon) ‘types’ of evidence or knowledge which can constrain the dynamics of research use. This is not to say that different types of evidence or knowledge are unwelcome: although the value of ‘indigenous’ knowledge has been questioned (Newman, 2011), these perspectives remain controversial.
In general the literature is agreed that the policy process benefits from knowledge acquired not only from ‘scientific’ means, and that ‘citizen’ and ‘practical’ knowledge gained from experience are vital to informing the decision-making process (Jones et al, 2009), as well as being empowering for citizens who are able to contribute to the evidence collecting process by drawing upon their own experience (Shahvali, 2011). Understanding how evidence is constructed and understood in developing country contexts is therefore critical to the process of trying to understand the role of ‘evidence’ in policy (Powell, 2006; Broadbent, 2012).
Apart from the social and cultural aspects of how understandings of evidence affect the relationship between research and policy, it is clear that political considerations weigh heavily upon policymakers in their use of research when making decisions. Policymaking is fundamentally a political endeavour which is better understood as being framed not by the question ‘what works best?’ but ‘what works best for me?’ While this is not a problem specific only to developing countries, it is argued that it is in such contexts that evidence-based policy is sorely needed.
The political dynamics underlying the research-policy nexus will inevitably vary by country, sector, and the specific conditions of the time. Broadbent (2012) and Nash et al (2006) both present approaches to assessing this political context, which will include considerations such as the origin of think tanks (Kimenyi and Datta, 2011) and the formation of universities (Zeleza, 2002), and human resources (Mansell, 2010). Further, as the RAPID Framework shows, the external influences also play a prominent role in this heady political mix, and are increasingly becoming more politically engaged in terms of their strategies (Jones, 2011).
Overall, it is possible to conclude that there is an undoubtable trend towards further work in this area, including increasing reflexivity on ‘evidence-based policy’, the power relations which sustain the assumptions underlying its promotion, and different understandings of evidence and the dynamics which determine the value places on different types of evidence. On a wider note, there is growing recognition that the ‘politics’ of research production, dissemination, and use is a cross-cutting issue that lies at the heart of all research-to-policy issues, both theoretical and practical and therefore the analysis of political context is central to research and policy endeavours.
Resources
Politics of evidence-based policy
Broadbent, E. (2012). ‘The Politics of Research-Based Evidence in African Policy Debates: Synthesis of case study findings’. London: EBPDN/Mwananchi
This article is a presentation of the findings of the research project ‘The Politics of Research Uptake’, and looks at the relationship between research and policy by focusing on the role of research – based evidence in African policy debates. Four policy cases are analyzed, in which policy debates are understood as an integral part of the policymaking process. The mainstay of the paper is that research use has a political nature and political context is key to understand the role of research – based evidence policy in Africa. It offers a three pronged framework that links the framing role of discourse and dominant cognitive understandings of what constitutes knowledge with agency – related factors to explain the role of research based knowledge in policy debates. The author also considers what is meant by ‘evidence-based policy’ and presents a critique of the un-examined nature of the concept.
Stone, D. (2002) ‘Using Knowledge: the dilemmas of ‘Bridging Research and Policy’’. Compare, Vol. 32, No. 3, pp. 285-296.
This paper focuses on the use of policy research in the social sciences, an aspect of development discourse, and the dilemmas that have been met by development agencies and researchers in communicating and using that research. It suggests that varying interpretations of policy – making provide different parameters of understanding of the research – policy nexus: different policy environments, institutional structures and political arrangements produce different sets of opportunities and constraints for dialogue, varying strategies for researchers and diverse implications from one political system to the next. Twelve perspectives are offered on improving research and policy linkages, which are categorised into three broad categories, supply – side, demand – led and policy currents.
Du Toit, A. (2012) ‘Making sense of evidence: Notes on the Discursive Politics of Research and Pro-Poor Policy Making’. PLAAS Working Paper 21. Cape Town: PLAAS.
Evidence – based policy offers poor guidance to those who want to ensure that social policy is informed by social sciences research, because it relies on a technocratic, linear understanding of the policy making process and is based on a naïve empiricist comprehension of the role of evidence. This leaves aside the role of underlying discursive frameworks and paradigms that give meaning to evidence. Politically loaded and ideologically compelling policy narratives that contest rival policy frameworks need to be taken into consideration in order to understand how evidence is used.
Types of knowledge
Shahvali, M. (2011) ‘Enriching indigenous knowledge: an alternative paradigm for empowerment’ Knowledge Management for Development Journal, Vol. 6, No. 3, pp. 194-205.
The concept of indigenous knowledge (IK) has gone through a development cycle, in which it was first considered useful and now seems to be considered less useful. How can the concept of IK be prevented from going through the same cycle of critique and rejection as concepts such as “sustainable development”, “community – based conservation” and “participatory development”? IK is essential for the eradication of poverty and the localization of development, which is why academics should enrich the quality of IK with theories and methods to empower local individuals and communities through a series of questions.
Igwe, L. (2011) ‘Critical Thinking and the African Identity’. Africa Unchained Blogpost, 25 November.
Igwe criticizes the belief he perceives to be held among his African peers that critical reasoning is part of a white, western mentality. He points out that nobody has ever offered a definition of the “African” way of thinking, and laments the fact that beliefs informed by superstition and primordial thinking is glorified in this continent. “Western” reasoning might be considered as unacceptable, but it is still considered reasonable and sometimes even superior.
Powell, M. (2006) ‘Which Knowledge? Whose Reality? An Overview of Knowledge Used in the Development Sector’. Development in Practice , Vol. 1, No.6, pp. 518–32.
This article explores the various definitions of knowledge that exist in a world of many cultures and intellectual traditions, as well as the role of language. It looks at their relationship with each other, and with the many and varied ‘informational developments’ – information – related changes in work, culture, organizations and technology around the world. Most current practice consistently acts against the type of relationship and type of communication that is essential if development policy and practice is to be something other than an imposition of external ideas. These issues pose a series of fundamental challenges to the development sector. It also considers where development organizations get their information and knowledge from and identifies problems with many of the channels used.
Jones, N., Datta, A. and Jones, H. (2009) ‘Knowledge, Policy and Power: Six Dimensions of the Knowledge–Policy Interface’. London: ODI.
This paper explores six key areas of the knowledge – policy interface in order to provide a summary of insights on the linkages between knowledge and development policy. The knowledge – policy interface is too complex to encapsulate in a single framework, which is why it’s critical that those seeking to engage in evidence – informed development policy dialogues also use additional tools and frameworks take into account different types of knowledge, such as citizen-based and practical knowledge. This synthesis has the objective of stimulating more nuanced debates and developing tailored tools for actors involved in knowledge translation processes.
Newman, K. (2011) ‘Should We Use Indigenous Knowledge […] Even If It’s Wrong?’ Research to Action blog post, 30 November 2011.
Even though recently there has been a surge of interest among international development circles in protecting and using indigenous knowledge, the author disagrees with the notion that indigenous knowledge is as valid as research – based evidence when it comes to deciding what works, and therefore what policies to make. The problem lies in the fact that there are many examples of populations which have shared knowledge about what works when it is in fact wrong. Indigenous knowledge is not a reliable source of evidence.
Politics of knowledge and decision making
World Bank/Oxford Policy Management (2008) ‘The Political Economy of Policy Reform: Issues and Implications for Policy Dialogue and Development Operations’. Washington D.C: Social Development Department, World Bank.
In the past few years there has been a move from the rigid policy conditionalities associated with the structural reforms era and towards a new development approach that emphasises the need for reform processes to be underpinned by a sufficient level of commitment within the country concerned to be sustainable over the long term. This paper presents two case studies in order to understand the significance of power relations within the economic policy sector, vested interests, and the links to national political processes, all of which can be critical to being an effective actor in policy dialogue. Its objectives are to analyze the political economy of reform by looking at stakeholder interests, incentives, institutions, risks, opportunities and processes from a social analysis perspective; and to illustrate what works, why and how for a better understanding of political economy issues in the design and implementation of reforms and development operations.
Jones, H. (2011) ‘Donor Engagement in Policy Dialogue: Navigating the Interface between Knowledge and Power’. Thinkpiece for AusAID. London: ODI.
Public policy in developing countries is a complex matter that involves a diverse collection of players, a large range of moving parts, and long – evolved spoken and unspoken cultural and political norms. This is why many aid agencies and international development organizations are increasingly engaging in policy dialogue in these countries, in order to recognize the role of power, and the multiple ways in which understanding and action are linked and interact with each other. This paper also offers tools with which to deal with these issues.
Mwangi S. Kimenyi and Ajoy Datta (2011). Think tanks in sub-Saharan Africa: How the political landscape has influenced their origins. London: ODI.
Two key factors help explain think tanks’ origin in Africa: the concentration of power across nation states and the role of external influence. Concentration of power factors include French and British research institutes during the colonial era, set up to help them govern; ‘big man’ politics after independence which reduced the political space for academics; and military governments that further reduced civic input into policy – making and brain drain. External influence factors are the heavy reliance after independence on foreign academics and intellectuals; and structural adjustment in line with Western inspired policies in the 80s and 90s, which resulted in research being transferred to non-state actors. The study also raises questions relating to how the political context has played a role in agenda – setting, and the impact of racial, ethnic or religious cleavages on research production.
Zeleza, Paul Tiyambe(2002) ‘The Politics of Historical and Social Science Research in Africa’. Journal of Southern African Studies, Vol. 28, No. 1, pp. 9- 23.
This paper argues that in most African countries, the development of universities and research has been mainly tied to the vagaries of state policies and politics, the shifting missions and mandates of international donor agencies, and the unpredictable demands and dislocations of civil societies. Also important factors are the internal challenges, the cultures of universities themselves, their goals and governance, management of resources and infrastructures, their capacities to pursue intellectual excellence and equity, political autonomy and public accountability, local relevance and international recognition. African social scientists have been caught in the difficulty of addressing African realities in other languages and paradigms, conversing with each other through publications and media controlled by foreign academic communities.
Mansell, R. (2010). ‘Power and Interests in Developing Knowledge Societies: Exogenous and Endogenous Discourses in Contention’. IKM Working Paper No. 11. Bonn: European Association of Development Research.
This paper presents two models for development, the exogenous model and the endogenous models, and argues that the former has been the predominant paradigm so far and has underpinned many of the interventions by the international development community aimed at employing ICTs to meet poverty reduction goals. Meanwhile, the endogenous model focuses more directly on human beings and their resources and aspirations. The overshadowing of the endogenous model by the exogenous one has serious social, cultural and economic consequences because the exogenous model cloaks the interests of investors in the global “North” whose principal ambition is profits from the sale of digital technologies and the content that is hosted or circulated through them.
Nash, N., Hudson, A., and Luttrell, C. (2006) ‘Mapping Political Context: A Toolkit for Civil Society Organizations’. RAPID Toolkit. London: ODI.
Policy is the result of interactions among different organizations about what course of action should be taken – the sum of these interactions constitutes the policy process, which is in turn shaped by political context. This toolkit lays out a range of tools that civil society organizations (CSOs) can use to understand and map political context in order to engage more effectively in policy processes. Some of these use the nation – state as the unit of analysis; others focus on the mapping of power, or the mapping of institutions, organizations and stakeholders, and formal and informal institutions. Each tool is first described, then a brief outline is provided of how the tool works, and finally an attempt is made to identify those elements of the tool that might be of particular interest to CSOs.
The role of narratives & ideas in policy arguments
There is a growing concern with how power is constituted through soft skills such as persuasion and rhetoric (Klein, 2001). Indeed, while ‘discursive’ approaches to understanding the relationship between knowledge and power in development processes have yet to be adequately applied (Jones, 2009), the role of research in producing indirect influences over general ideas, concepts, and discourses within the public policy realm has been emphasized by Weiss (1978) (see Section 2). Further, in considering what makes research influential, discussions on the role of research in policy have emphasized the need to make research findings fit into a dominant ‘narrative’ or story which is compelling to policymakers and the general public (see Section 2. on Policy Influence); and in addition to this, the literature has also highlighted the importance of engaging with the media to ensure research is disseminated widely and is thought to be an important part of the public agenda is also a crux issue in the area of research communications (see Section 2. on the role of the media). However, while ensuring that research findings cohere with policy narratives in order to maximize is advisable according to the literature, what does this mean for the quality of research presented and the ability of users of research to develop expertise in a particular area?
A narrative in the field of international development is understood as a mechanism to establish causal relationships between two negative aspects of a particular problem, drawing upon common sense and commonly-accepted beliefs and assumptions which reinforce widely-shared values. These narratives, argues Molle (2008), are often self‐validating because they tend to ‘produce’ evidence rather than vice-versa, manifesting themselves through “success stories”, “best practices”, “bright spots”, or “promising technologies”. Policy narratives are popular with policymakers for they eradicate complexity and, as Hass (1989) argues in his seminal work on the role of epistemic communities in policy discussions, policy narratives act as ‘uncertainty reducers’, placing a seeming order on what are complicated policy problems.
Yet there are serious concerns that the prominence of narratives to understand complex policy problems both crowds out minority voices who do not present arguments that fit neatly within existing frameworks; as well as fears that the concern for research findings being kept ‘simple’ will lead to what Enrique Mendizabal has called ‘dumbing down the audience’ (2010). Writing about their experiences in community research findings to DFID advisors, both Booth (2011) and Cleaver & Franks (2008) demonstrate that there is little demand for the ‘full picture’. Reflecting on his research program’s difficulty in finding a willing set of ears when presenting findings that were contrary to the ‘received wisdom’ regarding the efficacy of demand-led service delivery, Booth criticizes the development community’s insatiable appetite for “certain simple messages” over complex (and uncertainty-generating) research findings.
A number of further case studies support this literature, building what is a growing critique and evidence base regarding the potential downfalls of placing too much emphasis on the communication of bite-size research at the expense of building the capacity of research users to understand the complexities of policy problems (de Grassi, 2007; Stevens, 2011; Broadbent, 2012).
Resources
Knowledge and power
Klein, G. (2001) Sources of Power: How People Make Decisions. Cambridge: The MIT Press. (Book)
In this book, Klein sets forth an understanding of the policy process as an intuitive rather than linear process, with decisions being based on four ‘sources of power’: intuition; mental stimulation; metaphor; and storytelling. Instead of relying on manuals and toolkits, the MIT researchers argue, decision-making is strengthened by focussing on these four sources of power. These are based on the ‘soft’ skills it is not possible to train a computer to enact, including confidence and imagination. The role of metaphor and storytelling here is paramount, offering wisdom, drama, and empathy. The author implies that storytelling is instructive and enlightening rather than obstructive; and highlights that researchers need good creative writing skills in order to draw from their real-life experiences rather than following toolkits.
Jones, H. (2009). ‘Policy-making as discourse: a review of recent knowledge-to-policy literature’. ODI-IKM Working Paper No. 5. Bonn: European Association of Development Research.
This paper takes a wider view of research and policy linkages and considers ‘knowledge’ in the policy process. The paper covers much of the usual research-to-policy literature, but also addresses the generation and use of knowledge by political actors, looking at how knowledge is infused with power and is expressed in various ways. These include: cognitive paradigms, normative frameworks, frames, categories, policy narratives, and programmatic models and ideas. The focus on discourse provides a compelling window on understanding the interplay of knowledge and power in policy processes, in particular how they vary in different country contexts. For instance, efforts to understand the way in which policy discourse is understood and expressed in the South have highlighted the African concept of Ubuntu, which sees knowledge and communal ways of life as intimately linked.
Reducing complexity
Haas, P.M. (1992) ‘Introduction: Epistemic Communities and International Policy Coordination’. International Organization. Vol. 46, No.1, pp. 1–35.
This paper examines the role of ‘epistemic communities’ – networks of knowledge-based experts – in articulating cause-and-effect relationships to policymakers involved in complex decision-making processes. The paper assumes that policymakers are ‘uncertainty reducers’, with knowledge networks functioning to explain uncertainty through models of social and physical processes; and the influence such epistemic communities wield over policy decisions and the process of arriving at them is significant.
Molle, F. (2008) “Nirvana concepts, narratives and policy models: insights from the water sector”, Water Alternatives, Vol.1, No. 1, pp. 131–156.
This paper draws upon existing literature on policy narratives to furnish the author’s own experience in the water sector. It proposes a tentative on how the development and management of water resources is influenced by narratives and ideas that manifest themselves through “success stories”, “best practices”, “bright spots”, or “promising technologies”. Molle identifies three different types of concepts that shape policy and decision making in the contemporary water sector: nirvana concepts – these underpin overarching frameworks that promote or strengthen particular narratives or storylines (simple, causal, and explanatory beliefs) and legitimize specific blueprints or models of both policies and development interventions. Nirvana concepts, narratives and models are ideological objects which function to offer a “solution”. This paper argues that these influential concepts are not neutral or scientific; they do not emerge by chance but, rather, are the result of complex webs of interests, ideologies, and power. In return, they also shape the ways things are framed; options are favored, disregarded or ignored; and particular social groups are empowered or sidelined.
Dumbing down the audience?
Cleaver, F. and Franks, T. (2008) ‘Distilling or Diluting? Negotiating the Water Research–Policy Interface’. Water Alternatives 1(1): 157–76.
Based on their experiences of working with policymakers in the water sector, the authors critique instrumental approaches to the generation of knowledge and policy based on the amalgamation of perceived ‘success stories’ and ‘good practice’. Instead they favour approaches that attempt to understand water governance arrangements and outcomes for the poor within wider frameworks of negotiations over the allocation of societal resources. It is argued that current approaches to research and policy view the problem merely as one of ‘translation’, despite the existence of fundamentally different ideas relating to the nature and respective roles of research and policy. The paper draws particular attention to the difficulties in reconciling the role of social science researchers as uncertainty creators (unsettling established categories, questioning conventional wisdoms) and the imperative for policy makers to be uncertainty reducers. The authors reflect upon their experiences of having to ‘dilute’ research messages in order to fit narrow policy frameworks and understandings, thereby simplifying complex processes and arguments.
Mendizabal, E. (2010) ‘Dumbing down the Audience’. ODI Blogpost, July 6 2010.
In this opinion piece Mendizabal argues that researchers should not be satisfied with catering to the demands of policymakers who often prefer to read two-page research briefs rather than engaging in the complexities of issues at hand. Rather, researchers have a responsibility to raise the bar of policymaker’s evidence literacy. The article also warns of researchers ‘de-skilling’ themselves by focusing too much on shorter, ‘policy-relevant’ work at the expense of innovative, long-term research which challenges paradigms.
Booth, D. (2011) ‘Working with the Grain and Swimming Against the Tide: Barriers to Uptake of Research Findings on Governance and Public Services in Low-Income Africa’. APPP Working Paper 18. London: ODI.
In this paper Booth reflects upon his experience with the African Power and Politics Programme in trying to communicate research findings that go “against the tide” in terms of dominant policy narratives on the role of civil society in demanding better service provision in Africa. Contrary to the widely-accepted and popular narrative of ‘bottom-up’ and ‘demand-led’ service delivery being successful, the program’s research findings indicate the opposite. However, Booth argues that communicating this message has been difficult due to the over-selling of “certain simple messages” which do not reflect the complexities of the issue. Speaking more widely the paper argues that: “More fundamentally, incentives, ideologies and vested interests stand between research and the adoption of its findings into operational development thinking. This aspect of the problem of research ‘uptake’ needs to be taken more seriously by all concerned.”
Case studies
Aaron deGrassi (2007) ‘Linking Research And Policy: The Case Of Ghana’s Rice Trade Policy’. GSSP Background Paper 10. Accra/Washington D.C: IFPRI.
This paper reviews the role of knowledge in the formation of rice trade policy in Ghana, finding that there are three principal discourses shaping the way in which policy is understood: youth; populism; and modernization. The author traces the ways in which these discourses shape the relationship between research and policy, finding that they do not inherently inhibit research-policy links, and indeed might be ‘reframed’ to promote them. However, these discourses also shape and are shaped in various ways by several important political dynamics that influence connections between research and policy.
Stevens, A (2011). ‘Telling policy stories: an ethnographic study of the use of evidence in policy-making in the UK’. Journal of Social Policy. Volume 40, Issue 02, pp. 237- 255.
This article examines the use that is made of evidence in making policy amongst a group of UK civil servants. It shows that while civil servants displayed a high level of commitment to the use of evidence, their use of evidence was hampered by the availability of various kinds of evidence and by the unsuitability of much academic research in answering policy questions. In response to this, the civil servants used evidence to create persuasive policy stories that aided their various agendas, both in making acceptable policies and in advancing careers. They often involved the excision of methodological uncertainty and the use of ‘killer charts’ to boost the persuasiveness of the narrative. In telling these stories, policies which were ‘totemically’ strong were favoured over those that lacked a compelling narrative. Stevens concludes that a selective, narrative use of evidence is ideological, and in this case was used to support systematically asymmetrical relations of power.
Broadbent, E. (2012). Research-based evidence in African policy debates: Case Study 4 – Chieftaincy Reform in Sierra Leone. Evidence-based Policy in Development Network Case Study 4. London: EBPDN/Mwananchi.
This case study examines arguments made in the debate over the reform of the chieftaincy in Sierra Leone. A contentious and politicized debate involving multiple interests and dovetailing with more general debates about the country’s history and future trajectory, this paper argues that the debate is dominated by international development discourses which are not always well-understood by those propagating them. The paper draws attention to the role of the international community in diffusing policy narratives and ideas, and argues that concepts such as ‘good governance’ are not always helpful in promoting evidence-based arguments in policy discussions.
Evidence-informed/evidence-based policy and influence
While there is significant discussion on how to increase the influence of research on policy; and the research-to-policy literature clearly assumes that research should influence policy, and that decision-making processes in developing countries should be more influenced by evidence than they are presently, what is actually meant by policy influence?
Like notions of ‘policy’, ‘evidence’, or ‘knowledge’ there is no straightforward answer. Policy influence is understood in a number of ways, with there being different types of mutually-inclusive ways in which policy can be said to have been ‘influenced’ (see Section 3.1 on the Monitoring and Evaluation of Policy Influence).
The conflation of evidence-based policy and policy influence
Recent commentaries have been at pains to point out what policy influence is not. Broadbent (2012) and Newman (2012) both argue that policy influence is often conflated with evidence-based policy in the research and policy literature and practical support offered to civil society organizations in developing countries. Newman states that there is no inevitable link between the presence research-based evidence the policymakers’ awareness of it and policy influence. Further, just because research has had some influence in the policymaking process (variously conceived) this does not mean that the policy is ‘evidence-informed’ per se. In a similar vein, Broadbent warns against supporting researchers and research organizations to promote ‘my’ research at the expense of focussing on supporting the developing of a more general enabling environment for evidence-based discussion rather than, as Weiss (2001) notes, supporting the movement of policy research into the territory of advocacy. These commentaries emphasise the need to clarify objectives when it comes to supporting research-based evidence and supporting policy influence.
How policy changes
Alongside these critiques, the literature offers rich understandings of what policy influence is. The first issue to grapple with is the question of how policy changes, which is addressed in the literature by Kingdon (1995), Stachowiak (2009), Keeley (2001), and Ordóñez et al (2012). Policy change is understood to occur in a number of ways, including:
“Large Leaps” or Punctuated Equilibrium Theory, in which significant seismic changes in policy and institutions can occur when the right conditions are in place;
“Coalition” Theory or Advocacy Coalition Framework, which views policy change as occurring through coordinated activity among a range of individuals with the same core policy beliefs;
“Policy Windows” or Agenda Setting, in which policy can be changed during a window of opportunity when advocates successfully connect the way a problem is defined with the policy solution to the problem or the political climate surrounding their issue. Thus, approach also understands agenda setting as being fed by three ‘streams’ – the recognition of a problem; policy solutions to the problem; and the political environment which frames both the problem and the policy (Kingdon, 1995).
Messaging and Frameworks Theory, in which individuals’ policy preferences or willingness to accept them will depending on how options are framed or presented;
“Power Politics” or Power Elites Theory, in which policy is understood as the result of power-holders and thus change as a result of working directly with power holders, often incrementally and from ‘the inside’ with strategic allies;
“Grassroots” or Community Organizing Theory, which views policy change as the result of collective action by members of the community who work on changing problems affecting their lives.
The literature is agreed that when planning to influence policy it is essential to understand that policy is non-linear and hard to predict, therefore responsiveness is a required skill; and that is essential to match your influencing strategy with both your means and the objectives you want to achieve, meaning that different strategies based on different theories of change might be employed at different times. Further, Ordóñez et al (2012) argue that influence cannot always be read in terms of researchers marketing their own research, but a dialogue between stakeholders in which leadership and active collaboration are exercised in order to find the best solutions. Continuous learning and the role of evaluation is important here, as Weyrauch and Langou (2011) examine in their work on the role of impact evaluation in influencing policy change.
Types of policy change
Examples of types of policy changes are offered by both the IDRC and ODI. Lindquist (2001) describes ways in which research can influence policy in terms of: Expanding policy capacities, in which research improves knowledge and supports the development of innovative ideas, lines of thought and questions, as well as supporting actors to communicate ideas; ii) Broadening policy horizons, in which research provides opportunities for networking/learning, framing debates with new concepts and stimulating public debate and quiet dialogue between policymakers, and helping researchers to adopt a broader understanding of issues; and iii) Affecting policy regimes through modification of existing policies and programmes or their redesign. Jones (2011) relies on a different typology when explaining how to assess the impact of research on policy, referring to five different types of changes: i) Attitudinal change; ii) Discursive change; iii) Procedural change; iv) Policy content change; and v) Behavioural change.
Resources
Problems with conflating evidence-based policy with policy influence
Kirsty Newman (2012) ‘Policy influence versus evidence-informed policy’. Blogpost August 8th 2012.
In this blogpost Newman argues that evidence-based policy and policy influence are two different things that should not be confused. Finding out that a given piece of research has had impact does not necessarily mean that policy is evidence-informed – it could potentially just mean that those communicating the research have lobbied more effectively than others. If the research was good and the resulting policy change is good for poor people, this can seem like a positive outcome; however the danger is that the policy makers will be equally swayed by the next lobby group who comes along and argues their point effectively.
Broadbent, E. (2012). The Politics of Research Uptake – Synthesis of Case Study Findings. London: . Evidence-Based Policy in Development Network/Mwananchi
As part of this study’s conclusions the author argues that the promotion of evidence-based policy and supporting civil society organizations to influence policy constitute different sets of activities and should not be conflated, though they can often be integrated. At present, researchers are supported to present ‘my’ research, which does not necessarily improve how ‘evidence-based’ the wider policy environment is. A review of focus from individual and organization to systemic and discursive levels would be fruitful.
Weiss, C. (2001) ‘Policy research as advocacy: Pro and con.’ Knowledge & Policy, Vol 4, No.1, 2001, p.37-56.
This article considers how policy research influences policy, arguing that it is more influential and persuasive when it resembles advocacy. Thus, policy research moves away from ‘traditional’ research in which objectivity is of vital operational importance. The author questions what stance researchers should adopt in order to influence policy, asking whether they can influence policy as advocates yet still do justice to the complexities of their knowledge and concluding that there is something ‘uncomfortable’ about the close relationship between policy research and advocacy that we should be wary of.
How policy changes
Kingdon, J. (1995) Agendas, Alternative and Public Policies, 2nd ed. Harper Collins, New York.
(Book form only)
In this seminal book on policy analysis in the areas of health and transportation, Kingdon uses the notion of evolutionary ideas to explain how policy is formulated. At the crux of the policy process is the ‘agenda setting’ stage, where a number of ideas or solutions float around in ‘the policy primeval soup’ until those which survive are coupled with an emerging problem, and at the same time a ‘policy window’ is opened up by a crisis, political events, or by a policy entrepreneur. Thus, Kingdon uses the metaphor of three ‘streams’ to demonstrate the processes which contribute to agenda-setting: 1) the problem stream, in which issues are recognized as problems when there is capacity and/or interest in addressing the issue, or interest is stimulated by, for instance, lobbying groups; 2) the policy stream, where interest stimulated by lobbying groups (and similar) is targeted. The policy stream consists of ideas or solutions which have been pushed by governmental actors, experts, and others, and can disappear depending on whether the idea is considered ‘good’ according to prevailing advice or preference at the time; 3) the politics stream, which consists of the wider political context and which determines the policy and problem stream. The three streams come together in the concept of a ‘policy window’, which refers to a ‘focusing event’ (for example an external crisis or due to the skills of a policy entrepreneur) which provides an opportunity for all three streams to coalesce, as long as the proposals which emerge from the stream meet a number of criteria including coherence with societal values, national mood, financial and technical feasibility, and level of political support.
Stachowiak, S. (2009) ‘Pathways for Change: 6 Theories about how Policy Change Happens’. Seattle: Organizational Research Service.
This discussion brief present six understandings of how policy changes (otherwise known as Theories of Change or Logic Models). Different approaches will be suitable at different times due to internal or external ‘positioning’, capacity, and media interest. The approaches identified are: i)“Large Leaps” or Punctuated Equilibrium Theory, in which significant seismic changes in policy and institutions can occur when the right conditions are in place, often involving large-scale policy change and major advocacy and media campaigns; ii)“Coalition” Theory or Advocacy Coalition Framework, which views policy change as occurring through coordinated activity among a range of individuals with the same core policy beliefs, particularly useful when there is a strong group of allies and a sympathetic administration in place; iii) “Policy Windows” or Agenda Setting, in which policy can be changed during a window of opportunity when advocates successfully connect the way a problem is defined with the policy solution to the problem or the political climate surrounding their issue, particularly useful when readily-mobilized internal capacity exists; iv) Messaging and Frameworks Theory, in which individuals’ policy preferences or willingness to accept them will depending on how options are framed or presented, thus drawing attention to how problems are defined and communicated; v)“Power Politics” or Power Elites Theory, in which policy is understood as the result of power-holders and thus change as a result of working directly with power holders, often incrementally and from ‘the inside’ with strategic allies; v) “Grassroots” or Community Organizing Theory, which views policy change as the result of collective action by members of the community who work on changing problems affecting their lives, in which the advocacy organization can and is willing to play a “convener” or “capacity-builder” role rather than the “driver” role.
Keeley, J. E. (2001) ‘Influencing Policy Processes for Sustainable Livelihoods: strategies for change’. Lessons for Change in Policy & Organizations, No. 2. Brighton: Institute of Development Studies
Based on research in SSA, this booklet considers change within the policy process, suggesting a simple, but multi-faceted, framework for understanding it, as well as tools and ideas for engaging with and influencing policy to promote sustainable livelihoods. At base, the author states that how one engages with policy processes depends on what one wants to achieve, and where an individual or organization is coming from. Given that: i) policy is non-linear and complex; and ii) those wishing to influence policy possess different agendas, this brief offers advice. Key steps to be taken when influencing policy include the need to analyze context, to recognize and use policy narratives, to map actors, and identify policy spaces (i.e. moments for change or ‘windows of opportunity’).
Ordóñez, A., Bellettini, O., Mendizabal, E., Broadbent, E., and Muller, J. (2012) ‘Influence as a Learning Process: Think Tanks And The Challenge Of Improving Policies And Promoting Social Change.’ Buenos Aires: GRUPO FARO.
This working paper – prepared for the Conference “Think Tank Exchange” organized by the Think Tank Initiative in South Africa – is a synthesis of reflections on twelve stories of policy influence from organizations participating in the Think Tank Initiative (TTI) from Africa, Latin America, and South Asia. The paper makes a number of important observations on the role of think tanks (and their influencing strategies) on policy formation. With a number of varying roles, including facilitating dialogue, advising policymakers, and advocating for a policy, a review of the different cases shows that organizations play more than one role in the policy influencing process. ‘Influence’ therefore refers to different aspects of the policy process. The authors conclude that the stories have moved the understanding of policy influence as a tactical and narrow one to a more strategic and holistic one. Rather than viewing influence as an attempt to market ‘my’ research, this paper concludes that influence is more accurately understood as an exercise of leadership and collaboration with other stakeholders to find solutions and carrying out tasks beyond the production of relevant research.
Types of policy change
Evert A. Lindquist (2001) ‘Discerning Policy Influence: Framework for a Strategic Evaluation of IDRC-Supported Research’. Ottawa: IDRC.
This document was produced to help IDRC assess how the research it supports has influenced policy processes. In terms of how policy influence is understood this paper provides three key approaches: i) Expanding Policy Capacities: Research can have influence by improving knowledge and supporting the development of innovative ideas, lines of thought and questions, as well as supporting actors to communicate ideas; ii) Broadening Policy Horizons: Research can provide opportunities for networking/learning , frame debates with new concepts and stimulating public debate and quiet dialogue between policymakers, and helping researchers to adopt a broader understanding of issues; and iii) Affecting Policy Regimes through modification of existing policies and programs or their redesign.
Jones, H. (2011) ‘A guide to monitoring and evaluating policy influence’. ODI Background Note. London: ODI.
In discussing how to approach monitoring and evaluating policy influencing activities this background note considers both approaches to influencing policy and different types of policy influence. Five types of policy change are identified: i) Attitudinal change – framing debates, changing perceptions, affecting awareness, and getting issues on to the political agenda; ii) Discursive change – discursive commitments from states and other policy actors by affecting language and rhetoric; iii) Procedural change – changes in the process whereby policy decisions are made, such as opening new spaces for policy dialogue; iv) Policy content change; v) Behavioral change. Approaches to influence policy will reflect the types of changes aimed at. One way to categories these approaches is to distinguish between those that take the ‘inside track’, working closely with decision-makers, versus ‘outside track’ approaches that seek to influence change through pressure and confrontation. It is also important to distinguish between approaches that are led by evidence and research; and those that involve, primarily, values and interests.
Weyrauch, V., and Díaz Langou, G. (2011) ‘Sound expectations: from impact evaluations to policy change’. 3ie, Working Paper 12. New Delhi: Global Development Network.
This paper outlines a comprehensive and flexible analytical conceptual framework to be used in the production of a case study series. The cases are expected to identify factors that help or hinder rigorous impact evaluations (IEs) from influencing policy and improving policy effectiveness. This framework has been developed to be adaptable to the reality of developing countries. It is aimed as an analytical-methodological tool which should enable researchers in producing case studies which identify factors that affect and explain impact evaluations’ policy influence potential. The approach should also enable comparison between cases and regions to draw lessons that are relevant beyond the cases themselves.
For more information on how to monitor and evaluation policy influence, please go to Section 4.
Research functions/roles in different policy contexts
What is the function of research in different policy contexts? As the preceding sections have indicated and the literature on this subject argues, the role research can and does play depends on the following key variables:
The political context
The research (or communicator of research’s) objectives
The origin of research (including funding and definition of problem statement)
Financial and human resourcing
How research is communicated
While the intellectual origins of policy research rests in a quest to arrive at an empirical ‘science’ of society, politics, and the economy it is fair to say that in practice the functions of research can differ wildly from intent, meaning that the functions or uses of research need to be assessed separately from its aims. This discussion attempts to characterize some of the functions identified in the literature with reference to the above variables.
How is research used?
The literature is unanimous in arguing that the ways in which research functions within policy contexts differs, leading to the development of conceptual frameworks to best understand these functions (Tseng, 2012) (Weiss, 1977). A key insight from which these discussions proceed is that research does not ‘speak for itself’ and is therefore dependent upon how it is interpreted by users of research. This places a huge responsibility on ‘translators’ of research (see Section 2.5 on Knowledge Systems). Weiss’ typology of research utilization has arguably been one of the most influential contributions to the research-to-policy discussion, and remains a starting point for any consideration of research functions. The five models she identifies are still in currency:
Knowledge-Driven model – where research feeds directly into the policy process in a linear fashion;
Problem-solving model – where research fills a gap in knowledge relating to a particular problem and thus leads to a decision being arrived at
Interactive model – where social scientists are one actor amongst many in a large dialogue, and together with the inputs of others help policymakers arrive at a policy decision;
Political model – where policymakers appeal to research to justify a pre-determined position (Hope & Walters, 2008);
Tactical model – where the sponsoring of research is used tactically to justify decisions and delay decisions;
Enlightenment/conceptual model – the most innovative model proposed, involving the indirect influence of research findings upon general ideas, assumptions, and beliefs. The percolation of research-based ideas is problematic, primarily because it is not directly traced to a particular finding but a general ‘sense’ of what research on a subject indicates.
Other forms of utilization are described in NCDDR (1996) and Davies & Nutley (2008). In recent years this typology has been added to, notably Tseng (2012) offering an ‘imposed’ model, where the use of research (or paying lipservice to having used research) is compulsory; and Monaghan’s (2011) ‘processual’ model in which a number of competing groups try to shape the policy agenda, but that this is an open, unequal, and unpredictable competition, though, no inevitability that research will play any role in decision making and instead is subject to the dynamics of political context.
Importantly, the function research plays is highly dependent on the kind of relationship the producer or communicator of research has with policymakers (Weaver, 1989), the overall role a research organization or think tank plays (Chudnovsky et al, 2007), and a research organization or think tank’s origins, including its funding arrangements (Braun et al, 2007).
Resources
How is research used?
Tseng, V. (2012) ‘The Uses of Research in Policy and Practice’. Social Policy Report, Vol. 26, No.2. Society for Research in Child Development. New York: William T. Grant Foundation.
This paper presents a conceptual framework for understanding the uses of research in policy and practice, findings from recent empirical work, and early lessons from the field. The framework describes the ways policymakers and practitioners define, acquire, interpret, and ultimately use research. The author identifies a number of ways that policymakers and practitioners use research, including instrumental, conceptual, political, imposed, and process uses. Importantly, the paper argues that because research does not speak for itself, policymakers and practitioners must always interpret its meaning and implications for their particular problems and circumstances, and this in turn influences how research is used. ‘Translating’ research into policy discussion in an accurate way is therefore critical.
Weiss, C. (1977) ‘The Many Meanings of Research Utilization’. Public Administration Review, Vol. 39, No. 5, pp. 426-431.
This short article presents Weiss’ seminal framework with which to understand how research is used by policymakers. A number of models are offered: Knowledge-Driven model; Problem-solving model; Interactive model; Political model; Tactical model; and Enlightenment model. For Weiss, it is the Enlightenment model which offers the most fruitful perspective on the way in which policymakers are rarely “able to cite findings of a specific study that influenced their decisions, but they have a sense that social science research has given them a backdrop of ideas and orientations that has had important consequences” (p. 429). This insight has paved the way for a great deal of analysis on the way in which research can have an indirect influence on policy through the shaping of ideas, beliefs and values which are not directly traced to a particular piece of research.
Davies, H.T.O., and Nutley, S.M. (2008). ‘Learning More About How Research-Based Knowledge Gets Used: Guidance in the development of new empirical evidence’. New York: William T. Grant Foundation.
This primer offers an introduction to the discussion concerning how research is used in policymaking. As part of the William T. Grant Foundation’s ongoing work on how to understand the uses of research in order to assess impact, the authors argue for a better understanding of what Carol Weiss termed the ‘conceptual’ use or ‘enlightenment function’ of research. In this model, research broadens or challenges people’s understanding of issues and potential remedies. Davies and Nutley also call for increased study of the social processes and social contexts involved in research use. They also draw attention to the ‘political’ act of labelling knowledge ‘research’ and ‘evidence’, arguing that there is no easy way of separating research from the context of its use. Uses identified in the framework include: Research under-use, overuse and misuse; Strategic, political, and tactical uses of research, where research-based knowledge may be used to support pre-existing positions, undermine the positions of others, or legitimate already defined courses of action; Instrumental and conceptual uses of research, where decision-making is diffuse and research-based knowledge provides a background of theory and data that slowly engages local discourse; Process benefits from engagement with research, where the process of arriving at findings by engaging policymakers can have an influence; Individual research use and system-embedded uses, involving how knowledge is systematized and embedded.
National Center for the Dissemination of Disability Research (NCDDR) (1996). ‘A Review of the Literature on Dissemination and Knowledge Utilization’. Austin: NCDDR.
Research use and think tank context
Belletini, O. (2007). ‘El papel de los centros de investigacion de politica publica en las reformas publicas implementadas en America Latina’. In Garce, A. and Gerardo, U. Think Tanks y políticas publics en Latinoamerica: Dinamicas globales y realidades regionales. Buenos Aires: IDRC/Konrad Adenauer Stiftung/Prometeo libros.
In this book chapter the authors argues that the function of research cannot be divorced from the context in which it is produced. Based upon ongoing work on Latin American think tanks undertaken by CIPPEC, this refers specifically to exogenous influences upon think tanks, which play a number of roles. In turn these roles reflect upon the kinds of research they do, and why. These roles include articulating (problematizando) public problems and making them the objective of public policies; designing alternative policy prescriptions based on research; helping to decide the ‘right’ or preferred course of action implementing reforms and public policies; and monitoring politicians and public policies, for instance though budget monitoring, evaluation of policies, and transparency initiatives.
Weaver, R. K. (1989). ‘The Changing World of Think Tanks’. Political Science and Politics, Vol. 22, No. 3, 563-578.
Based on analysis of think tanks in the United States, Weaver attempts to account for their emergence and functions. In this article the author argues that different research functions reflect different types of organizations, or think tanks. Key types include ‘government’ think tanks (or contract research organizations) which possess a close relationship with policy makers and thus produce on a demand-led basis; academic universities and ‘universities without students’ that offer a greater degree of autonomy and long-term influencing objectives which are oriented towards changing the climate of opinion rather than directly influencing a policy in a particular way; and ‘advocacy’ think tanks which have a strong political or ideological bent. The functions and objectives of research cannot, it is argued, be divorced from organizational objectives and orientations.
Braun, M., Chudnovsky, M., Ducote, N., & Weyrauch, V. (2007). ‘Lejos de “Thinktanklandia”: los institutos de investigacion de politicas en los países en desarrollo’. Buenos Aires: CIPPEC.
(In Spanish)
This paper presents 18 case studies considering what affects policy influence in Latin American, Asian, African, and Central European contexts. It adopts a framework which identifies exogenous and endogenous factors to explain how think tanks have influenced policy with their research. Institutional and organizational factors relating to how a think tank was formed, why, and when are particularly important, alongside considerations of who a think tanks communicates research to (i.e. who they are trying to influence) and how they are funded. This paper therefore suggests that the function of research depends upon both exogenous and endogenous factors, and in many ways reflect them.
Examples of how research functions
Court, J. and Young, J. (2003). ‘Bridging Research and Policy: Insights from 50 Case Studies’. ODI Working Paper 213. London: ODI
As part of the first phase of the three-year Global Development Network (GDN) Bridging Research and Policy project, the Overseas Development Institute (ODI) collected and analyzed 50 summary case studies on research-policy links. This paper reports on the process, findings and implications of the case study work. The cases include examples of a wide range of types of research undertaken by a variety of organizations. A few cases describe situations where research had an immediate and direct impact on policy, although in most cases the impact was less direct and took some time, requiring strenuous advocacy efforts. The cases have been examined from the perspective of research uptake pathways, addressing the question: why are some ideas that circulate in the research-policy arenas picked up and acted on, while others are ignored and disappear? In terms of research functions, the main point to emerge from this rich discussion is that although the case studies focus on specific pieces of research around specific policy events, much research influences policy indirectly by informing the general discourse within which specific policies are made, for example the cases of biodiversity in Saudi Arabia, measuring inflation in Peru, fiscal policy in Chile, and the Water and Power Development Authority (WAPDA) in Pakistan, all illustrate how research can provide stores of knowledge that filter into policy as the need arises. Further, projects and programs need to be clear about distinguishing research objectives from capacity-building and policy influence.
For further discussion on differentiating objectives see Section 2.3.
Tim Hope, Reece Walters (2008). Critical thinking about the uses of research. Evidence-based policy series. London: Centre for Crime and Justice Studies King’s College London
This monograph is concerned with the policy process in the UK and the former Labour government’s seemingly politicised use of evidence in justifying the ‘war on terror’, drugs policy, and the demise of civil liberties amongst others. The authors accuse the former government of a ‘whitewash’, and argue that they did not encourage enlightened public debate. In particular, the paper examines how the former government engaged in ‘half-truths’ in order to present criminological data ‘for political gain’.
Monaghan, M. (2010) ‘Adversarial Policies and Evidence Utilization: Modeling the Changing Evidence and Policy Connection’. German Policy Studies, Vol. 6, No. 2, pp. 17–52.
This journal article considers the role of evidence in ‘adversarial’ policy formulation where there is a contested evidence base. By reviewing the recent literature on UK drug policy – an adversarial policy domain – and the literature on research utilization, the paper argues that in most cases policy is usually a ‘blend’ of evidence-based policy and policy-based evidence. While existing models of research utilization have been employed to explain how research is used politically, the author argues that these tend to offer only limited descriptions of the evidence and policy connection. This therefore paper puts forward a newer ‘processual’ model which it claims can account for the many subtleties involved in explaining the evidence and policy connection in adversarial domains. This model assumes that a number of competing groups try to shape the policy agenda, but that this is an open, unequal, and unpredictable competition. Unlike the ‘evolutionary’ model of research uptake (Stevens, 2007) there is, however, no inevitability that research will play any role in decision making. The processual model therefore accepts that evidence is embedded in the policy process and could percolate into decision-making, but that this is always contingent on numerous factors including the impact of the wider political process.