9/4/2018

Module Objectives
Uganda Martyrs University
MSc in M&E  On completion of this module, you should:
 Have acquired knowledge in applications of Monitoring
and Evaluation
 Be able to explain the processes of developing the
Module: M&E Principles and Perspectives monitoring applications and frameworks
Topic: Introduction to M&E  Be able to carry out evaluation guided by research
Facilitator: Mugerwa George methods approaches
 Use the different frameworks and tools for their planning,
monitoring, performance determination and analysis

September 4, 2018 mugerwageorge@[Link] 2

Reading List Reading List

 Patton, M.Q. (1997) “Toward Distinguishing Empowerment
 Louisa Goslin and Mike Edwards (2007); A practical Guide to Evaluation and Placing It in a Larger Context” Evaluation Practice
Planning, Monitoring, Evaluation and Impact Assessment, 1997 18(2)
Save the Children  Pretty, J. (1996): Participatory Learning and Action: A Trainer’s
Guide. London: International Institute for Environment and
 Harold Kerzner,(2009) Project Management; A Systems
Development.
Approach to Planning, Scheduling and Controlling
 Jules Pretty, Irene Gujit, Ian Scoones and John Thompson (2002).
 Civicus; Monitoring and Evaluation Toolkit; [Link] Participatory Learning and Action A Trainer’s Guide; International
 UNDP; 1997; Results Oriented Monitoring and Evaluation; A Institute for Environment and Development (IIED), London, 1995.
Handbook for Program Managers; In Mind Design, L.L.C.  Rugh, J. (2007).Self-Evaluation: Ideas for Participatory Evaluation of
 Bhavesh M. Patel (2000). Project Management; Strategic Community Development Projects World Neighbours
Financial Planning, Evaluation and Control; Vikas Publishing  Narayan, D. (2008).Participatory Evaluation: Tools for Managing
House PVT Ltd Change in Water and Sanitation; Technical Paper Number 207.
Washington, DC: The World Bank.
 PACT, (1998): Participatory Monitoring, Evaluation and
Reporting.
September 4, 2018 New York, PACT
mugerwageorge@[Link] 3 September 4, 2018 mugerwageorge@[Link] 4

Presentation Outline Session Objectives

 By the end of the session, you should be
 Session Objectives
able to:
 Overview of M&E  Explain the evolution of M&E
 Definition of M&E  Explain the concept M&E
 Rationale for M&E  Explain the rationale for carrying out M&E
 Performance indicators activities
 Demonstrate ability to design project/
 M&E Plan
organizational performance indicators

September 4, 2018 mugerwageorge@[Link] 5 September 4, 2018 mugerwageorge@[Link] 6

1
 9/4/2018

Overview of M&E
 As soon as your organization decides to
 The evolution of evaluation in Africa began
engage to have something done it has
three problems: early in the 1950s.
 Get people to do it!  The rapid growth in international
 Monitor whether they do it! development assistance in the 1970s saw the
 Find out whether it was done well!
shoring up of M&E systems, as the need for
the control and accountability increased.
MONITORING  The conventional M&E effort focused on
physical inputs and outputs and financial
EVALUATION expenditure against capital items.

September 4, 2018 mugerwageorge@[Link] 7 September 4, 2018 mugerwageorge@[Link] 8

Overview of M&E Overview of M&E

 It relied on objective methods, facilitated by – Justifying activities to external funding bodies and
outside experts and often initiated by donor donors;
agencies. – Ensuring that the group or organization is
 It was by and large concerned with: developing in a sustainable manner;
 Establishing the financial and technical feasibility – Sharing successes and failures with others to build
of the project solidarity; and
 Producing a comprehensive and detailed project – Building community.
design or blueprint

September 4, 2018 mugerwageorge@[Link] 9 September 4, 2018 mugerwageorge@[Link] 10

Evolution of M&E Evaluation in OECD Developed Countries

 However in China evaluation can be traced as back as 2200  OECD countries have mature M&E systems
BC- personnel selection
 In a recent survey (2013): Australia, Canada, the
 In the 19th century western countries adopted performance Netherlands, Sweden, and the USA had highest
measurement in education as a mechanism for evaluation.
“evaluation culture ranking”
 In the 1960s USA - Social programs Great society and War on  These countries have developed an evaluation
Poverty. Optimism of 1960s: evaluation would change the
world culture with:
 Professional evaluators in different fields
 In the 1970s there was a growing concern for Aid evaluation  Institutional government framework for collecting and
(Netherlands, UK, Australia) disseminating information. It is a requirement for
 In 1990s we see a new wave for Development-oriented institutions to show their worth.
evaluation.  Evaluation has moved beyond audits to policy and
September 4, 2018 mugerwageorge@[Link] 11
outcomes – e.g. the
September 4, 2018
regulatory impact assessments 12
mugerwageorge@[Link]

2
 9/4/2018

OECD Approaches Introduction to M&E

 Monitoring and evaluation (M&E) of development
 OECD encourages three pronged approach to activities provides government and development
evaluation. practitioners with information for :
 Whole-of-Government: Broad-based,  Better means for learning from past experience,
comprehensive M&E at all levels of government.  Improved service delivery,
E.g. the National Integrated Monitoring and  Planning and allocating resources, and
Evaluation System.  Demonstrating results as part of accountability to key
stakeholders.
 Enclave: Focused on one part or sector of the
 Within the development community there is a strong
government (one ministry or cabinet). E.g. Local
focus on results—this helps to explain the growing
government monitoring systems
interest in M&E.
 Mixed Approach: Blended whole of government  However there is also some confusion about what M&E
and enclave. Some areas have comprehensive entails e.g. going to field, fault finding, meeting donor requirements etc.
approaches; others
September 4, 2018
more sporadic attention.
mugerwageorge@[Link] 13 September 4, 2018 mugerwageorge@[Link] 14

The Power of Measuring Results But what is M&E?

 But why measure results?
 If you do not measure results, you cannot tell success
from failure
 If you can not see success, you can not reward it
 If you can not reward success, you are probably
rewarding failure
 If you can not see success, you can not learn from it
 If you can not recognize failure, you can not correct it
 If you can demonstrate results, you can win public
support

September 4, 2018 mugerwageorge@[Link] 15 September 4, 2018 mugerwageorge@[Link] 16

M&E M&E
 Monitoring and evaluation are essentially the core  M&E is the bringing together of information and
processes for “learning from experience.” learning.
 Learning implies a process of analysis which, in turn,  Data are the raw material used to fuel the
requires the existence of relevant information or monitoring and evaluation process.
evidence on which to base the analysis.  Access to appropriate data and data sets that can be
 The primary function of M&E is to provide pointers processed into usable, timely, and relevant statistical
on how to do things better through a better information is essential for effective monitoring and
understanding of what works and what does not. evaluation that in turn can lead to a learning
experience.

September 4, 2018 mugerwageorge@[Link] 17 September 4, 2018 mugerwageorge@[Link] 18

3
 9/4/2018

Monitoring versus Evaluation Monitoring Evaluation

Monitoring Evaluation  The continuous collection  A systematic process to

and review of information on determine the extent to which
programme implementation, service needs and results have
A planned, systematic process A process that assesses an
coverage and use for been or are being achieved
of observation that closely achievement against preset comparison with
criteria. and analyse the reasons for
follows a course of implementation plans. any discrepancy.
activities, and compares Has a variety of purposes, and  Open to modifying original  Attempts to measure service’s
follow distinct plans during implementation relevance, efficiency and
what is happening with methodologies (process,  Identifies shortcomings effectiveness. It measures
what is expected to happen outcome, performance, before it is too late. whether and to what extent
(planned vs actual) etc).Looks at relevance,  Provides elements of analysis the programme’s inputs and
Looks at inputs, processes, effectiveness, cost as to why progress fell short services are improving the
activities and outputs effectiveness and impact of expectations quality of people’s lives.

September 4, 2018 mugerwageorge@[Link] 19 September 4, 2018 mugerwageorge@[Link] 20

Why M&E?
Comparison between M&E There are a lot of good reasons for conducting M&E.
M&E is important for:
• Determining whether resources are being expended in
the manner planned and according to the program
budget.
• Getting frequent feedback about how your project is
progressing and making sure that things are on track.
• Improving the effectiveness of projects by allowing for
mid-course corrections if there are aspects that are
not having the desired impact.

September 4, 2018 mugerwageorge@[Link] 21 September 4, 2018 mugerwageorge@[Link] 22

Why cont’s Why cont’s?

•Serving as a basis for sound decisions about According to the World Bank, effective M&E provides
answers to the following questions:
whether and how to revise your project.
• Are we doing the right things?
•Promoting accountability among project staff. –Are our interventions contributing to the project objectives?
•Demonstrating the impact and success of your • Are we doing the right things right?
project. –How effective have we been in achieving expected
outcomes? How efficient have we been in optimizing
•Identifying lessons learned, enabling resources? Are these results sustainable?
institutional learning and informing decisions • Are there better ways of doing the right things?
about future programs. –What are the best practices we’ve identified?

September 4, 2018 mugerwageorge@[Link] 23 September 4, 2018 mugerwageorge@[Link] 24

4
 9/4/2018

Emerging Trends & Implications Top of International Development Agenda

 Changes in M&E as a result of emerging Issues  Sustainable Development Goals (MDG)
 Globalization  Debt Initiative for Heavily-Indebted Poor Countries
 Growing incidence of conflict (HIPC)
 Terrorism and money laundering  The Emergence of New Actors in International
 Widening gap between rich and poor Development Assistance (China, Korea, India, Turkey,
 More development players EU, South Africa, Singapore, Libya, Iran)
 Drive toward debt reduction
 Conflict Prevention and Post-conflict Reconstruction
 Focus on improved governance and accountability
 Governance and accountability
 Drive toward comprehensive, coordinated, participatory
development with results

September 4, 2018 mugerwageorge@[Link] 25 September 4, 2018 mugerwageorge@[Link] 26

Top of International Development Agenda New Actors in International Agenda

 Anti-Money Laundering and Terrorist Financing
 Workers’ Remittances  Large foundations:
 Gender: From Women in Development (WID) to  The Bill and Melinda Gates Foundation
Gender and Development (GAD)  The Ford Foundation
 Clinton Foundation
 Mainstreaming Private Sector Development
(PSD) and Investment Climate  China –Africanisation and the AID industry
 Environmental and Social Sustainability  All these actors must become part of the
 Global Public Goods process to design and develop evaluations

September 4, 2018 mugerwageorge@[Link] 27 September 4, 2018 mugerwageorge@[Link] 28

Trends particularly in the south…. Trends particularly in the south…

 New aid paradigms  New management in states and NGOs: results /
 PRSP December 1999 performance oriented
 MDGs September 2000  New indicators in international monitoring protocols
 Paris Declaration March 2005 e.g.
 Choice of indicators
 New conditionalities
 e.g. enrolment vs. completion rate
 Old-style conditionalities are replaced with conditionalities
 Level of aggregation
that focus on in-country processes - i.e. opening up
 e.g. world vs. regional scale
discussion among stakeholders, ‘ownership’,
 Narrow definition of indicators
 Enhancement of M&E in developing countries, new
 e.g. ownership measured by number of countries with national
instruments, measurability of MDGs “makes developing development strategies (see Paris Declaration)
countries nervous”

September 4, 2018 mugerwageorge@[Link] 29 September 4, 2018 mugerwageorge@[Link] 30

5
 9/4/2018

Trends particularly in the south… What do these trend mean to M&E?

 New actors:  Driving developing countries to build
 M&E systems established at central and decentralized evaluation capacity and systems
levels of government- e.g. local government  Donors called upon to provide technical
performance assessments
 Civil society organizations as watchdogs of government
assistance
performance- e.g. parliamentary performance index  Involvement in non traditional areas of
 Civil society organizations to respond to RBM trend development
themselves
 Countries must commit to accountability and
transparency -openness
 Subscribe to aid effectiveness
September 4, 2018 mugerwageorge@[Link] 31 September 4, 2018 mugerwageorge@[Link] 32

Where are We Today?

What do these trend mean to M&E?
 The new development agenda calls for broader
understandings of:
 Gender budgeting mandatory
 sectors
 Objective measures of business regulations  countries
and their reinforcement (env’tl degradation)  development strategies
 Corporate Social Responsibility (CSR): actively  policies
taking into account the economic,  Emphasizes learning and continuous feedback at all
environmental and social impacts, and phases of the development cycle
consequences of business activities  Evaluations have moved from traditional
 Evaluation of global public goods in regard to implementation focused evaluation models to results
environment and sustainable development based evaluation models.
September 4, 2018 mugerwageorge@[Link] 33 September 4, 2018 mugerwageorge@[Link] 34

Characteristics of M&E practice and

Where Are We Today? systems in South (LDCs)
 Emphasis is now being placed on working (Khan, 1998:313)
 Evaluation practice is of recent date
together and growing a number of partnerships
 Direct relationship between quality of governance and
 Performance of individual partners now needs the success of M&E
to be evaluated according to their respective  Weak institutional and methodological capacity
contributions and obligations  Most evaluations are donor-driven
 Many are top-down and lack wider participation
 Evaluation has become more difficult to design  Limited inter-ministerial cooperation and coordination
with a more demanding, fragmented, and  Monitoring is first- tracking physical progress
participatory approach to development  Feedback system is absent or weak
 Limited budgets for evaluations
September 4, 2018 mugerwageorge@[Link] 35 September 4, 2018 mugerwageorge@[Link] 36

6
 9/4/2018

Characteristics cont’ How do we execute M&E?

 In executing the M&E functions, M&E
 There are highly placed champions willing to
assume political risks of evaluations e.g evaluation practitioners focus on the performance
networks indicators.
 These indicators can be at the different
 Efforts to professionalize evaluations are taking
root
levels of objectivity hierarchy i.e.
 Impact
 Some credible institutions
 Outcome
 Availability of basic statistical systems and data
 Output
 Sometimes appropriate quality and quantity of  Activity
empirical evidence
 Input
 Increasing demand from the citizens
September 4, 2018 mugerwageorge@[Link] 37 September 4, 2018 mugerwageorge@[Link] 38

Hierarchy of Performance Concepts

What are performance indicators?
 Performance indicators are measures of inputs,
processes, outputs, outcomes, and impacts for
development projects, programs, or strategies.
 When supported with sound data collection and
management system, they enable managers to track
progress, demonstrate results, and take corrective
action to improve service delivery.
 It is important to involve the key stakeholders when
defining performance indicators.
 This enables them to have a thorough understanding of
these indicator and the rationale for capturing them.
 PMT can base on them to make management decisions.

September 4, 2018 mugerwageorge@[Link] 39 September 4, 2018 mugerwageorge@[Link] 40

Where is the relevance of M&E? How does M&E inform Mgt functions?
 M&E is a key function for service delivery  M&E helps to make informed decisions
improvement. regarding the on-going programs.
 M&E shows whether a service/program is  It facilitates effective and efficient use of resources
accomplishing its goals.  It helps to determine whether the program is right on track
 It identifies program weaknesses and and where changes need to be considered
strengths, areas that need revision, and  M&E help stakeholders conclude whether the
areas that meet or exceed expectations. program is a success or not.
 To do this, you need to analyse the
program’s domain.  M&E preserve institutional memory.

September 4, 2018 mugerwageorge@[Link] 41 September 4, 2018 mugerwageorge@[Link] 42

7
 9/4/2018

Three Primary Uses of M&E Findings When should we do M&E?

 M&E is a continuous process that occurs throughout the life
 Rendering judgments of a project/ program.
 Summative evaluations of program’s overall effectiveness  But to be more effective, it should be planned at the design
 e.g. audit, renewal, quality control, accreditation stage of a program, with the time, money and personnel that
will be required calculated and allocated in advance.
 Facilitating improvements  Monitoring should be conducted at every stage of the
 Formative evaluation to improve program program, with data collected, analyzed and used on a
 e.g. program’s strengths/weaknesses, progress continuous basis.
 Evaluations are usually conducted at the beginning, in the
 Generating knowledge middle and at the end of programs.
 Conceptual use of findings  Evaluations should be planned for at the start because they
 e.g. generalization, theory building rely on data collected throughout the program, with baseline
data being very important.
September 4, 2018 mugerwageorge@[Link] 43 September 4, 2018 mugerwageorge@[Link] 44

END

September 4, 2018 mugerwageorge@[Link] 45