Lectures Series by Assoc.Prof Dr.
Amer
Aziz for Post Graduate Course ‘Poverty
Alleviation’
Contents
Lectures Series on Poverty Alleviation Understanding Impact?
by
Dr. Amer Aziz
[email protected]
Lecture:6 Poverty Impact
Evaluation Methods-2
Reading Source: Emmanuel Skoufias The World Bank PREM KL
Lectures Series by Assoc.Prof Dr. Amer Lectures Series by Assoc.Prof Dr. Amer
Regression Discontinuity
Aziz for Post Graduate Course ‘Poverty Aziz for Post Graduate Course ‘Poverty
Alleviation’ Alleviation’
Impact Evaluation Methods
Design (RDD)
A regression discontinuity design is a way of
• Experimental methods/Randomization
undertaking causal inference, usually of some
• Quasi-experimental methods
• Propensity score matching (PSM) policy intervention. It provides robust,
• Regression discontinuity design (RDD) convincing estimates of causal impacts under
fairly weak conditions or minimal assumptions.
• Other Econometric methods
• Before and After (Reflexive comparisons)
A regression discontinuity is design is
• Difference in Difference (Dif in Dif) appropriate where a treatment/intervention
• Instrumental variables /policy is given to individuals for whom
• Encouragement design
measured characteristics lies on one side of a
‘cut off’ and characteristics cannot be perfectly
manipulated by individuals.
Lectures Series by Assoc.Prof Dr. Amer Lectures Series by Assoc.Prof Dr. Amer
Aziz for Post Graduate Course ‘Poverty Aziz for Post Graduate Course ‘Poverty
Regression Discontinuity Alleviation’
Regression Discontinuity Alleviation’
Design (RDD) Design (RDD)
1
Lectures Series by Assoc.Prof Dr. Amer
Regression Discontinuity
Aziz for Post Graduate Course ‘Poverty
Alleviation’
Design (RDD) Regression discontinuity
When to use this method?
o The beneficiaries/non-beneficiaries can be ordered along
a quantifiable dimension.
o This dimension can be used to compute a well-defined
index or parameter.
o The index/parameter has a cut-off point for eligibility.
o The index value is what drives the assignment of a
potential beneficiary to the treatment (or to non-
treatment.)
Intuitive explanation of the method:
o The potential beneficiaries (units) just above the cut-off
point are very similar to the potential beneficiaries just
below the cut-off.
o We compare outcomes for units just above and
below the cutoff.
Example: Effect of cash transfer on
Regression Discontinuity
Consumption
Design-Baseline
Goal Target transfer to poorest households
Method
o Construct poverty index from 1 to 100 with pre-
intervention characteristics
o Households with a score ≤ 50 are poor
o Households with a score >50 are non-poor
Implementation Cash transfer to poor households
Evaluation Not Poor
Measure outcomes (i.e. consumption, school attendance Poor
rates) before and after transfer, comparing households
just above and below the cut-off point.
Regression Discontinuity
Design-Post Intervention
Sharp and Fuzzy Discontinuity
Sharp discontinuity
o The discontinuity precisely determines treatment
o Equivalent to random assignment in a neighborhood
o E.g. Social security payment depend directly and
immediately on a person’s age
Fuzzy discontinuity
Treatment o Discontinuity is highly correlated with treatment .
Effect o E.g. Rules determine eligibility but there is a margin of
administrative error.
2
Lectures Series by Assoc.Prof Dr. Amer
Aziz for Post Graduate Course ‘Poverty
Alleviation’
Advantages of RD for Evaluation
RD yields an unbiased estimate of
treatment effect at the discontinuity
Can take advantage of a known rule for
assigning the benefit
o This is common in the design of social interventions
o No need to “exclude” a group of eligible households/
individuals from treatment
Training Course on Analytical
Diff in Diff
Methods (Quantitative) for
Planners and Social Scientists by
Potential Disadvantages of RD Poverty evaluation
Dr. Amer Aziz
Local average treatment effects Difference in differences (sometimes
o We estimate the effect of the program around the
cut-off point abbreviated DID or DD is a statistical technique
o This is not always generalizable used in quantitative research and in the social
Power: sciences that attempts to mimic an
The effect is estimated at the discontinuity, so we generally experimental research design using
have fewer observations than in a randomized experiment
with the same sample size. observational study data, by studying the
differential effect of a treatment on a 'treatment
Specification can be sensitive to functional group' versus a 'control group' in a natural
form: experiment.
Make sure the relationship between the assignment
variable and the outcome variable is correctly modeled,
including: (1) Nonlinear Relationships and (2) Interactions. 16
Lectures Series by Assoc.Prof Dr. Amer Lectures Series by Assoc.Prof Dr. Amer
Diff in Diff Diff in Diff
Aziz for Post Graduate Course ‘Poverty Aziz for Post Graduate Course ‘Poverty
Alleviation’ Alleviation’
Poverty evaluation Poverty Evaluation
Difference in differences requires data Not all of the difference between the treatment and
measured from a treatment group and a control groups at time period 2 (that is, the difference
control group from two or more different
time periods, from before and after a
between P2 and S2) can be explained as being an effect
'treatment'. In the example pictured, the of the treatment, because the treatment group and
outcome in treatment group is represented control group did not start out at the same point at time
by the line P and the outcome in the control period 1. DID therefore calculates the "normal"
group is represented by the line S. The difference in the outcome variable between the two
outcome (dependent) variable in both
groups are measured at time period 1,
groups (the difference that would still exist if neither
before either group has received the group experienced the treatment), represented by the
treatment (i.e., the independent or dotted line Q. (Notice that the slope from P1 to Q is the
explanatory variable), represented by the same as the slope from S1 to S2.) The treatment effect
points P1 and S1. The treatment group then is the difference between the observed outcome and
receives or experiences the treatment and
both groups are again measured at time
the "normal" outcome (the difference between P2 and
period 2. Q).
3
Lectures Series by Assoc.Prof Dr. Amer
Diff in Diff
Aziz for Post Graduate Course ‘Poverty
Alleviation’
Example
Poverty Evaluation
Water supplied to households by competing
private companies
Sometimes different companies supplied
households in same street
In south London two main companies:
Lambeth Company (water supply from
Thames Ditton, 22 miles upstream)
Southwark and Vauxhall Company (water
supply from Thames)
Example: In 1853/54 cholera Diff in Diff :The effect of clean water
outbreak on cholera death rates
Counterfactual 2:
Death Rates per 10000 people by water company 1849 1853/ Difference ‘Control’ group
Lambeth 10 54 time difference.
Southwark and Vauxhall 150 Assume this
Lambeth 150 10 -140 would have been
true for
Might be water but perhaps other factors ‘treatment’
Vauxhall 125 150 25 group
and
Snow compared death rates in 1849 epidemic
Southwark
Lambeth 150
Southwark and Vauxhall 125 Difference -25 140 -165
In 1852 Lambeth Company had changed supply from
Hungerford Bridge Counterfactual 1: Pre-Experiment difference between
treatment and control—assume this difference is fixed
over time
Lectures Series by Assoc.Prof Dr. Amer
Diff in Diff: A Graphical Diff in Diff
Aziz for Post Graduate Course ‘Poverty
Alleviation’
Representation Poverty Evaluation
A Treatment Suppose a company wants to measure impact of
y its poverty Program in the following two cities.
Program
C counterfactual Lahore D1=Dprog+ Dtrend
2008 D1 2010
B Control
DD= D1-D2
D3 D4= Dprog+Darea D4 DD= D3-D4
Faisalabad D2
Pre- Post-
Time
A – B = Standard differences estimator
C – B = Counterfactual ‘normal’ difference
A – C = Difference-in-Difference Estimate
4
Lectures Series by Assoc.Prof Dr. Amer
Diff in Diff
Aziz for Post Graduate Course ‘Poverty
Alleviation’
Poverty Evaluation
Y Pre Post
Lahore 30 70
Faisalabad 20 90
D1D1= 70-30=40
D2D2= 90-20=70
DD=70-40=30
