Monitoring and evaluation and the policy process

Presented at the Public Policy Executive Program conducted by the Strathmore

  1. pasicUganda
    Presented at the Public Policy Executive Program conducted by the Strathmore
    Transcript Header:
    Monitoring and evaluation and the policy process
    Transcript Body:
    • 1. Topic: Monitoring and Evaluation the Policy Process Stephen Wainaina, MBS email: wainainacs@gmail.com 1
    • 2. Overview 2 Part 1 1. Policy cycle 2. Monitoring 3. Evaluation Part 2 4. M&E in Kenya 5. Conclusions
    • 3. Policy cycle - Recap 3 Public policy is an attempt by the government to address a public issue.It is generally defined as a system of laws, regulatory measures, courses of action, and funding priorities concerning a given topic promulgated by a governmental entity or its representatives(G. Kilpatrick).  When term Public policy is used: 1. Official Government policy (legislation or administrative guidelines that govern how laws should be put into operation); 2. Broad ideas and goals in political manifestos and pamphlets; 3. A company or organization's policy on a particular topic. For example, the equal opportunity policy of a company shows that the company aims to treat all its staff equally; and, 4. In Kenya, major policies -which are nation wide affects the whole population or are sectoral in nature - are initiated by use of specific policy documents, legal notices, Sessional Papers, official pronouncements and Acts of Parliament. Policies address public issues that require attention and are also meant to achieve sustainable development
    • 4. Policy cycle…cont’n The rational model for the public policymaking process can be divided into four parts: 1) Agenda-setting: the agencies and government officials meet to discuss the problem at hand(Problem identification); 2) Option-formulation: alternative solutions are considered and final decisions are made regarding the best policy(Policy Formulation); 3) Implementation: the selected/best policy option is implemented in the final stage(Policy adoption – formal acceptance as law, regulation, administrative directive, or other decision made according to rules of relevant arena), and, 4) Monitoring and Evaluation(M&E). M&E are necessary for the achievement of evidence-based policy making, budget decisions, management, and accountability (Makay) 4
    • 5. Monitoring Monitoring: A system used to continuously measure the quantity, quality, and targeting of the goods and services—the outputs—that the state provides and to measure the outcomes resulting from these outputs. Monitoring responds to four questions as follows: • Who - Internal; • What – Regular assessment of progress or deliverables(outputs) against those in the plan in terms of performance; and, • How – record of and reporting on outputs; • When – Annually, semi-annually or quarterly/monthly. 5
    • 6. Monitoring…cont’n  Practice is part of management and implementation;  Progressive tracking of the policy, blueprint, project/program is carried as data is gathered during the implementation process; and  Monitoring data is gathered by all stakeholders – and can include project implementers (management), private sector, CSOs, donors and can then be reported at the project/program level, County or National ;  Reports are produced as predetermined, and used by management and or policy implementers to influence implementation in subsequent period; Good sources of information for policy includes government statistics, key stakeholders, interviews with beneficiaries, researchers, etc;  Monitoring is short term and data sources and procedures for monitoring need to be established at the beginning;  Data gathered should reflect a clear understanding of the issue but should not necessarily be exhaustive. Indicators, can be selected to facilitate policy monitoring; 6
    • 7. Monitoring…cont’n  As work on policy is going on, the policy problem can be changed or refined;  In many policy cases, a balance that protects the interests of citizens, even where there are many beneficiaries, is key; Monitoring review can be done to establish:  Sufficiency of data, usefulness of reports or whether deliverables as reported are useful to decision makers;  Whether there is feedback to learn from the implementation process; and,  Data gathered becomes the source of information for Evaluations. 7
    • 8. Evaluation  Evaluation: A system used periodically to measure the quantity, quality, and the amount of the goods and services provided and to measure the outcomes and impacts resulting from these outputs.  Evaluation responds to four questions as follows: • Who – External – usually by independent consultant or body. Can be Internal, but should involve stakeholders; • What – Objective assessment of all stages of policy work – assessing content against objective, relevance and performance( effectiveness, efficiency, approach) and output/impact against objective; and, • How – Data collected through a composition of methods, which allows for validation and verification; and, • When – typically biannually, mid-term and end-term. 8
    • 9. Evaluation…cont’n  Evaluation criteria clarifies policy goals and objectives and helps in the judgement of the solutions to the problem addressed;  Data gathered during monitoring is the main source of information for evaluations;  Criteria, both explicit and implicit, is critical for the policy makers;  Range of criteria may be long. Includes efficiency, cost, net benefits, equity, political concerns, administrative ease, legal matters, etc; and Regular evaluations and midterm evaluations informs on required changes/options to achieve policy goals and/or required refinements. Some key aspects of Evaluation also include: • Unit of analysis for the evaluation – program level, full or partial; • Key questions to be responded to; Principle audiences to benefit from the evaluation – policy makers, government level, etc; Type of evaluation – Implementation, impact, etc; Cost; and, Timing and Duration. 9
    • 10. PART 2 • M& E in Kenya • Conclusion 10
    • 11. M&E in Kenya The Government introduced countrywide Monitoring and Evaluation in 2003 after the launch of the ERSWEC following the formation of NARC Government; Prior to this effort, no comprehensive M&E was undertaken. The practice was only carried out for projects and programs, especially those funded by development partners; A fully fledged department was created to steer this process in the Ministry of planning; The Government, which was committed to promote good governance and accountability, identified M & E as an appropriate tool to facilitate this process; This policy direction led to establishment of the National Integrated Monitoring and Evaluation System (NIMES) by the Government in 2004; 11
    • 12. M&E in Kenya …cont’n NIMES was meant to address the following: Track implementation of policies, programmes and projects at all levels; Create a culture and practice for monitoring and evaluation and hence promoting accountability; Enhance public service delivery  The specific objectives of the NIMES are to: • Build an M&E system for reporting at both central government and the lower devolved levels (the county level). • Promote a culture and practice of M&E at all levels of government and civil society, i.e. institutionalizing its application. • Increase the use of M&E throughout the government for planning and implementation of development programmes. • Provide timely and reliable feedback to the budgetary preparation process through the preparation of reports (MTEF, MPER/PER and APR). 12
    • 13. M&E in Kenya…Cont’n • Provide regular, timely and reliable reporting on the effectiveness of government programmes and projects – to government itself, wider stakeholder group and to development partners. • Ensure active participation of the civil society - (CSOs, NGOs, the academic community, private sector etc.) in the National M&E System;  NIMES framework is aligned to five broad development areas: - 1. Enhanced policy coordination and Capacity Development within the NIMES framework to manage, communicate, evaluate and make revisions to the implementation of the framework. 2. Development of an integrated data acquisition and data management framework to support research and evaluation of results as well as other government policy actions. 3. Research and Results analysis component – interrogates key report findings as a basis of informing policy and budgetary issues. 4. Enhanced Project Monitoring and Evaluation Systems at central and devolved levels to support Public value and results based Service delivery. 13
    • 14. M&E in Kenya…cont’n 5. Integrated dissemination and communication of M&E results and finding and providing feedback mechanisms to support streamlined reporting on the government’s national and global commitments to a wide cross-section of stakeholders. It also strives to strengthen partnerships and capacity (within government and non-government sectors) to be more effective in using M&E information.  The Ministry of Devolution & Planning is functionally responsible for coordination, monitoring and evaluation of all of Government policies, programmes and projects.  Within the Ministry, responsibility for the implementation and oversight of the NIMES Framework has been conferred to the M&E Directorate (MED).  Within the MED, there are 5 technical advisory Groups (TAGs) that essentially act as the NIMES Secretariat to ensure proper representation and redress of all M&E issues of national interest; 14
    • 15. M&E in Kenya Cont’n 5 TAGs: 1. Research and Results analysis. 2. Quantitative and qualitative indicator development, data collection and storage. 3. Project Monitoring and Evaluation. 4. Dissemination for Sensitisation and Advocacy. 5. Capacity development and Policy coordination  The Technical Oversight committee (TOC) is the primary body responsible for setting the conceptual direction for the development and implementation of the NIMES. TOC is chaired by the Economic Planning Secretary (EPS) The National Steering Committee (NSC) is the primary body responsible for providing general/overall policy direction of NIMES. NSC is chaired by the Principal Secretary (PS) Responsible for Planning.  County Level M&E - Government intention is to have all the above 5 strategic areas on which NIMES is focussed are being implemented at both the National and sub-national/county levels. 15
    • 16. M&E in Kenya – deliverables/Results • In 2003 – ERSWEC launched and 33 indicators developed; • APRs produced annually to Monitor implementation of ERSWEC; • 2008 – End Term Review of ERS; • In 2008 – Kenya Vision 2030 and MTP I launched; • First Handbook of National Reporting (Indicators) Launched; • APRs produced to Monitor implementation of MTP; • 2013 – End Term Review of MTP I; • 2013 – MTP II launched; • 2014 – Second Hand Book of National Reporting; • Has 5 parts: o1- 56 indicators for monitoring 56 national outcomes(MTP II) oII – Framework for monitoring Flagship projects; oIII – Indicators for monitoring Counties and Constituencies; oIV- Gender monitoring framework; 16
    • 17. M&E in Kenya Challenges: Inadequacy of M & E skills in public sector; Culture for M & E is still low; Cost of operationalizing NIMES is very high; Absence of an M & E policy (Draft available); In Counties – challenges of structures, capacity, equipment and resources, as well as Communication and linkages with National Government; Weak link to the budgetary preparation process; and, Budgetary constraints. 17
    • 18. Conclusions 18 1. M&E is part of the policy cycle - both are necessary for the achievement of evidence-based policy making, budget decisions, management, and accountability. 2. Monitoring Progressively tracks policy, blueprint, project/program as it is carried – out and data is gathered during the implementation process. 3. Monitoring review can be done to establish sufficiency of data, usefulness of reports or deliverables as reported as reported to decision makers; to get a feedback to learn from the implementation process. 4. Evaluation, which is periodic, helps clarify the policy goals and objectives and helps in the judgement of the solutions to the problem being addressed. 5. Range of criteria used for evaluation process includes efficiency, cost, net benefits, equity, political concerns, administrative ease, legal matters, etc. 6. Evaluation informs on required changes/options to achieve policy goals and/or required refinements and on successes and failures. 7. Kenya started comprehensive national M&E in 2003. NIMES framework and system is still evolving. 8. Culture of M&E is still low and we do not as yet have an M&E policy 9. NIMES has achieved some successes but has challenges, especially with devolved structures. 10. The yardstick of success of M&E is the extent to which information is being used to improve government performance.
    View More