[Editor’s note: This post is the first of a series of posts focused on the nitty-gritty of designing and implementing M&E systems by sharing on the ground learning and experience. This series is produced by Dena Lomofsky of Southern Hemisphere]
Implementing M&E systems is an organisation wide effort that requires rethinking organisational structure and culture. At Southern Hemisphere we have developed a participatory process to design the M&E system. A participatory process is important because it builds buy-in, institutional knowledge and enhances the relevance of the plans and tools. It is important to start where the organisation is at, building on their strengths and addressing their challenges. It is a process of co-design.
In our process, we begin with a situation analysis of the M&E system and stakeholders and produce a diagnostic report. We then develop a results based M&E plan (usually based on logical framework approach) which leaves the client with a very good understanding of what they want to measure, why they want to measure it, how it will be measured, reported and analysed and how the information will be used for decision making purposes to improve performance. We prepare a detailed guideline document explaining the M&E plan (with a focus on indicator definitions and analysis) and tools and reporting formats.
Ideally this is done together with those that will be using the tools so they understand them. If we have the luxury of time and budget, we provide training in core M&E concepts and approaches. This is where our engagement usually ends, and it is up to the organisation to operationalise the plan and use the tools and guidelines. We may check in now and again, but the organisation has to take ownership. Our engagement usually ends here for two main reasons 1) funds – organisations do not have the funding to engage us in a hands-on process of implementation 2) it shouldn’t be a consultant driven process and organisations are best placed to build the organisational systems.
However, this is where they usually get stuck: in operationalising the system.
One of the main reasons for this is because they have not thought through the institutional arrangements or the systemic elements that are needed to implement the plan.
Let me explain in more detail.
In our understanding at Southern Hemisphere, an M&E system is made up of three main components as depicted in the diagram below; 1) the M&E plan (inner circle); 2) the guideline document for the plan and the tools and reporting formats and to collect and report on data (2nd circle), and 3) the institutional arrangements to support the operationalization of the plan (outer circle). Let us delve deeper into each, what it means and its relevance for the implementation of M&E systems.
Core M&E system elements
The monitoring, evaluation and reporting plan is the heart of the system.
This is a plan (in table format) to manage the collection, analysis and reporting of data and to help organisations think about how they will use the information from each indicator, and to allocate budget to the M&E effort.
The M&E plan builds on objectives and indicators that are derived from a logical framework or other design process. The template that we use looks like this:ObjectivesPolicy makers attend workshops on G20IndicatorNo. of policy makers attendingData sourceAttendance register;Event reportDisaggregationType of policy makerWho will collect the dataProject officerFrequency of data collectionEach workshopWho will analyseProject officerFrequency of data analysis2 weeks after each workshopWho will reportProject officer, Programme HeadReporting format and frequencyEvent report; quarterly reports, annual reportHow will the information be usedTo manage stakeholders engagementBudgetUSD 500 per workshop for M&E
Central documents guiding the implementation of the M&E plan.
This includes
a guideline document for the M&E plan (that at least includes a theory of change, description of the programme, definitions of each indicator and how they should be measured, a list of data sources and reporting formats, and how each indicator is mapped to these).
Forms for tools for data collection
Reporting formats
Institutional arrangements
This refers to the institutional or systemic arrangements that need to be in place to support the implementation of the M&E plan. This includes
Clearly defined roles and responsibilities for monitoring, evaluation and reporting. Some of these are identified in the M&E plan in terms of data gathering, analysis and reporting, but who manages the organisation wide M&E effort and makes sure that all the institutional arrangements are in place? Who provides leadership on the M&E effort
Human and physical resources – are the people in place able to implement and lead the M&E system? Is there sufficient budget for what the M&E plan requires?
Electronic data base – are there information systems for data storage and analysis?
Continuous improvement – are there processes for using the evidence arising from the M&E system to improve projects and processes – e.g. decision making mechanisms based on evidence, reflection sessions to encourage a culture of learning?
Capacity building – Everyone in the organisation should understand the scope and elements of the M&E system. The people tasked with M&E responsibilities must have the necessary coordination and research skills to be able to effectively interact with others.
This final component is usually underestimated and we find there is large room to better incorporate these dimensions in order to help organisations implement the M&E plan that has been designed with help from us as experts.
In the following blog posts, we can look in more detail about why a systemic approach is important and unpack some of the common barriers encountered by organisations and share some possible solutions.