By Anne Davis 773 under CC at flickr.com
Editor’s note: This post was written by Ulviyya Mikayilova, Policy Unit manager at the Center for Innovations in Education (CIE), from Azerbaijan. In 2013, a small Policy unit on Social Policy was set up at CIE with the support of the OSF Think Tank Fund. This post is part of a series of reflections from participants and facilitators of the Online training to strengthen M&E&L on policy influence in Central and Eastern Europe, supported by the Think Tank Fund
I have been invited to provide my reflections on the M&E&L course delivered by the Politics and Ideas. I hope these reflections will be rather provocative.
Let me start with several thesis: 1) no doubts that M&E is important and beneficial to organizations and their development; 2) no doubt that any organization and its staff should pass through the M&E training; the more trainings would be the better; 3) only those theories influence practices which are applicable enough or could be adapted without damaging its core concepts; 4) any theory to be applicable for practice should be and could be adapted to the context of the particular organization and to the context of the particular environment the organizations is settled in.
I’ll try to support my statements with some samples from our own practice.
M&E&L system
The CIE has had many M&E studies conducted both internally and externally. External M&E&L has only been conducted if funding was available for contracting people to do that job. Internally M&E&L was used to meet monitoring indicators provided by donors or to provide data for development of the CIE annual reports. In spite that experience unfortunately M&E&L has never become a formal part of organizational on-going operations. Why? Because it was considered as time consuming, needing a lot of staff/ time commitment and in general often does not seem to be affordable for small organizations.
Until the right formula for M&E is found, organizations like ours will be balancing between donor demands and daily routines/realities of the organization where everyone is performing several roles and everyone is under high workload. The course offered a practical and feasible way of addressing M&E&L challenge for small organizations.
Can we afford having a formal well-structured and build based on the best theory M&E system? I think, no. We simply do not have sufficient resources to afford that. Can we try to build that system into organizational practice and budget? Yes, but will be that sustainable? I am not sure. This will bring about more fundraising, more resource allocation, will make our services more expensive; will increase a pressure of responsibility to maintain it and so on.
Several of the M&E&L course “lessons” like setting up our own “notions of success and failure” and our own standards to measure ourselves were important to us. We have come to understand that through being a part of internal organizational conversation, discussion and reflection, M&E&L becomes implicitly and informally embedded into the organizational life. Probably this seems to be a right solution for organizations like ours. The course through assessment of whether the organization is heading in the right direction has made us confident that this is our way!
Policy Unit’s Influence: The Story on Success and Failure
For PU, assessing what success is or not in terms of policy influence is more important today than establishing an objective measurement of what we have or nor have achieved. Let me share this reflection based on a recent experience.
The PU policy paper on private-public partnership in Early Childhood Education has justified a need for promoting community-based alternative preschool service provision and shared results of successful piloting three community-based models. One of alternative models was representing provision of paid services by local municipalities. Low cost of service was making them affordable for the majority of the community members. This PU policy paper was publicized using various channels including round table presentations and discussions with participation of policy makers; mass media; and a presentation at the annual Education Research Forum. It also was used as a case study for learning activities during training on education policy planning conducted by the PU staff.
Young researchers being part of the PU were upset with the lack of feedback provided by policy makers on the PU policy analysis and recommendations. Another concern was related to the overall policy influence of the CIE/PU research and policy recommendations. PU research studies imply significant funding, hard team work and advocacy efforts; a high investment is made in policy papers. However we first perceived that their influence at the policy level was either not significant or even opposite to what was expected.
For example, after the PPP in the ECE policy paper was released the Ministry of Education has started piloting paid ECE services provided by public general schools. But the high cost of these services has made them affordable for only a small minority. Is that influence? Yes. Is that kind of influence researchers were looking for? Definitely, no. Is that success or failure?
Before, we were thinking that it was a failure, but the course has changed our perception. It has helped us revisit our notions of success and failure and conduct a critical analysis of what “good” policy influence is for us. We exercised an internal discussion on possible reasons of the “specific” influence at policy level (instead of “low” or “false” as were thinking initially) in the above mentioned case and reviewed some assumptions. One finding was that CIE/PU did not deliver the right message to the right audience. So we deduced that we probably need to better understand our stakeholders.
Another finding was that due to complexity of the context we operate in it would be a purely theoretical assumption that if we deliver right message to right audience, then the expected policy influence will take place. The problem is that we hardly can deliver our message to the right audience. In fact, those who attend the PU events (and many similar events) and those who participate in our discussions very often are not those who make decisions we want to see. This means that we intend to influence policy but we are aware now that it is actually is beyond our control or maybe beyond our current reach?! What to do? Should we give up? Why not to initiate another research on the long terms social and political consequences of access to Early Childhood Education limited to only better off minority population? Why not to continue educating policy makers? Why not to revisit our advocacy and awareness campaign?
Even if the ideas produced as a result of this internal discussion cannot be implemented immediately (taking into account funding available and already approved research agenda plans) the exercise provided an opportunity for transformation of our initial “failure” perception of the “policy influence” towards accepting it as an opportunity to learn and act further. This is where we found a lot of value from the course.
Advantage of the M&E&L
Based on our organizational experience dealing with the M&E&L and the particular experience we have had with implementation of the M&E&L tools offered by the course, I see M&E not as a system to measure our success or failure in terms of influence policy. I also see M&E not as a system to meet donor demands (because, honestly, even without sophisticated M&E we can do that using project generated data). The highest advantage and benefit of the M&E&L as an exercise and even as a system is that it proposes a way to engage an organization and people into the joint reflection, thinking and rethinking processes. It is a learning, professional development and as a team building exercise. M&E becomes a part of the team conversation: that is a great success!