Cash or Condition? Evidence from a Randomized Cash Transfer Program

Transcription

1 Cash or Condition? Evidence from a Randomized Cash Transfer Program Sarah Baird (George Washington University) Craig McIntosh (UC San Diego) Berk Özler (World Bank)

2 Outline Summary of findings Background & motivation The Zomba Cash Transfer Program (ZCTP) Sampling and survey design Research design and implementation Program impacts on: Education (enrollment, attendance, attainment, and achievement) Pregnancy and marriage Mental health, sexual behavior, STI infection Conclusions Implications for LAC? Not for citation without explicit permission from the authors. 2

3 Summary 1. While the program is effective in reducing dropout from school, we are unable to detect any gains in enrollment or attendance from conditioning the payments on regular school attendance. However, girls in the conditional treatment arm had higher achievement in English reading comprehension tests. Not for citation without explicit permission from the authors. 3

4 Summary 2. On the other hand, CCT had no effect on being ever married or pregnant at follow-up while UCT had a large and significant impact. Similarly, while mental health improved in both treatment arms, the improvement was significantly higher in the UCT arm. 3. Finally, for a variety of other outcomes, such as sexual behavior and STI infection, there were no differences in impact between the CCT and UCT treatment arms. Not for citation without explicit permission from the authors. 4

5 Summary 4. These findings do not make a solid case for conditioning transfer payments to households on regular school attendance. 5. They also suggest that we should take into account the overall effect of the conditionality rather than just its effect on school enrollment/attendance as it affects multiple outcomes that are pertinent. 6. The decision to condition or not will depend on the relative weights the policy-makes attaches to these outcomes (and the relative costs ). Not for citation without explicit permission from the authors. 5

6 Background and Motivation As of 2007, 24 countries around the world had some type of a Conditional Cash Transfer program (CCT) in place, with many others planning or piloting one (WB PRR, 2009) Whether to implement UCTs or CCTs is a current debate in at least some countries in Sub-Saharan Africa. Despite the above, we still don t have enough evidence on crucial elements of new program design. Not for citation without explicit permission from the authors. 6

7 Evidence on Conditionality Evidence points us in favor of the conditions. de Brauw and Hoddinot (2008); Schady and Araujo (2008) using implementation glitches Bourguignon, Ferreira, Leite (2003); Todd and Wolpin (2006) using structural models. Two common themes to these papers: All from Latin America (Brazil, Ecuador, and Mexico) All with very little income effect. These findings may not apply to Sub-Saharan Africa Not for citation without explicit permission from the authors. 7

8 Zomba Cash Transfer Program Zomba Cash Transfer Program (ZCTP) is a two-year randomized intervention that provides cash transfers (and school fees) to young women to stay in or return to school. Program has multifaceted research design with contract variation in various dimensions. Schoolgirls in randomly selected villages receive unconditional transfers. Transfers split between parents and girls: Parents transfer randomized at village level between $4-10. Girls transfer randomized at individual level between $1-5. Not for citation without explicit permission from the authors. 8

9 Sampling and Survey Design 3,798 young women were sampled from 176 enumeration areas (EAs) in Zomba, a district in Southern Malawi. EAs randomly drawn from three strata: urban, near rural, and far rural. All households in each sampled EA were listed using two forms, then the sample selected from the pool of eligible young women. Please do not cite without explicit permission from the authors.

10 Sampling and Survey Design Eligibility into the program was defined as follows: Eligible dropouts: unmarried girls and young women, aged 13-22, already out of school at baseline (<15% of the target population), AND Eligible schoolgirls: unmarried girls and young women, aged 13-22, who can return to Standard 7-Form 4, enrolled in school at the time of their first interview. Otherwise, there was no targeting of any kind. The surveys employed at baseline and at follow-up are comprised of two parts: Please do not cite without explicit permission from the authors.

11 Sampling and Survey Design Part I is administered to the HH head, and collects information on the following: household roster, dwelling characteristics, household assets and durables, consumption (food and non-food), household access to safety nets & credit, and shocks (economic, health, and otherwise) experienced by the household mortality Not for citation without explicit permission from the authors. 11

12 Sampling and Survey Design Part II is administered to the core respondent, who provides further information about her: family background, education, labor market participation, time allocation, health and fertility, dating patterns, detailed sexual behavior at the partnership level, knowledge of HIV/AIDS, social networks, own consumption of girl-specific goods (soaps, mobile phone airtime, clothing, braids, handbags, etc.). Not for citation without explicit permission from the authors. 12

13 Additional data collection instruments School Census (2009), School Survey (2010); Biomarker data on HIV, HSV-2, and syphilis (2009); Learning assessment in mathematics, English reading comprehension, and cognitive skills (2010). Not for citation without explicit permission from the authors. 13

14 Timeline Baseline data collection: September 2007 January Cash Transfers begin: February 2008 Round 2 data collection: October February Biomarker data collection: June - September Cash Transfer Program ends: December Round 3 data collection: February - June Includes educational testing. Please do not cite without explicit permission from the authors.

15 Zomba Cash Transfer Research Design Malawi Research Design: Treatment EAs (N=88) Control EAs (N=88) Baseline Dropouts (N=805) Conditional (N=46) T1 CCT T2 Unconditional (N=27) T1 CCT T2a S2 T2b S2 Baseline Schoolgirls Within- Uncon- Within- (N=2,741) CCT village ditional village control CT control T1 Only (N=15) T1 CCT S2 Withinvillage control C1 Pure control C2 Pure control Individual transfer randomized at individual level EA treatment saturation randomized Household transfer randomized at EA level Not for citation without explicit permission from the authors. 15

16 Zomba Cash Transfer Research Design Conditionality Analysis: Treatment EAs (N=88) Control EAs (N=88) Conditional T2 Unconditional T1 Only (N=46) (N=27) (N=15) T1 T1 T1 C1 Baseline Dropouts (N=804) CCT CCT CCT Pure control T2a S2 T2b S2 S2 C2 Baseline Schoolgirls Within- Uncon- Within- Within- Pure (N=2,742) CCT village ditional village village control control CT control control Impact of CCTs Impact of UCTs Differential Impact of Conditionality Not for citation without explicit permission from the authors. 16

17 Zomba Cash Transfer Program Implementation For CCT recipients, attendance is checked monthly at each program school using a combination of physical checks and phone calls (with random spot checks in Year 1, i.e. 2008). For CCT recipients, the payment for the next month is withheld if attendance is below the required threshold. However, the girl remains in the program. UCT recipients receive their transfers by only showing up. Not for citation without explicit permission from the authors. 17

18 Two questions on implementation and measurement. 1. Were the programs rules understood properly in each treatment arm? As we will see the differential impacts on various outcomes shortly, the two groups clearly did not perceive the program to be the same. But, so what did they perceive exactly? The program administration was very diligent to distinguish the two types of offers. In Year 2, treatment status was reinforced every month during the cash transfers. Qualitative data confirm the participants understanding of the program rules. Not for citation without explicit permission from the authors. 18

19 Two questions on implementation and measurement. 2. Q: Why did we not conduct spot checks to measure attendance? A: As can be seen from the qualitative data, UCT beneficiaries were aware of the attendance monitoring for CCT beneficiaries and knew that this was linked to their monthly payments. The study PIs were worried that any monitoring of attendance in the UCT group could give the impression that their payments were also conditional on school attendance. Critics correctly claim that spot checks for attendance in CCT programs do not have any discernible impact on attendance. But, none of these programs had an unconditional treatment arm! Not for citation without explicit permission from the authors. 19

20 Randomization and Attrition Characteristics of control and various treatment groups are balanced at baseline. Attrition from the sample is small after one year (5-6%) and two years (<10%) and again balanced across control and various treatment groups. Treatment group was more likely to take the educational tests, but: No differential attrition was there between UCT and CCT, and No sign of a higher propensity to perform better among those in the control group who did not take the tests. Not for citation without explicit permission from the authors. 20

21 Program impacts on schooling: Enrollment Program Impacts on School Enrollment by Year and Term (1) (2) (3) (4) (5) (6) (7) Year1 Year2 Term1 Term2 Term3 Term1 Term2 Term3 Postprogram Conditional treatment ** 0.046** 0.065*** 0.075*** 0.084*** (0.010) (0.012) (0.018) (0.024) (0.025) (0.029) (0.037) Unconditional treatment 0.030*** 0.048*** 0.054*** 0.072*** 0.098*** 0.109*** 0.053* (0.009) (0.010) (0.019) (0.023) (0.025) (0.029) (0.028) Time trend *** *** *** *** *** *** *** (0.007) (0.007) (0.013) (0.013) (0.014) (0.017) (0.016) Number of observations 4,178 4,178 4,177 4,176 4,176 4,177 4,177 Prob > F(Conditional=Unconditional) The dependent variable is whether the student reported being enrolled in school for the relevant year/term. Regressions are individual fixed-effects models with robust standard errors clustered at the EA level. All regressions are weighted to make the results representative of the target population in study EAs. Parameter estimates statistically different than zero at 99% (***), 95% (**), and 90% (*). Not for citation without explicit permission from the authors. 21

22 Program impacts on schooling: Enrollment Enrolment Rate in Percentage Points 60% 70% 80% 90% 100% Baseline Year 1 Year 2 Post-Program Schooling Trimester Conditional Control Unconditional Not for citation without explicit permission from the authors. 22

23 Summary of program impacts on enrollment and attendance We detect no difference between the CCT and the UCT treatment arms in school enrollment or attendance. Using school surveys from Round 2 and Round 3, there is no significant difference in enrollment or regular attendance (as reported by the teachers of the beneficiaries) between the CCT and UCT arms. Not for citation without explicit permission from the authors. 23

24 Summary of program impacts on enrollment and attendance Time use data (from Round 3) do not suggest any differences in time spent in school either. However, it is possible that the program impact is at a margin above 80%: Miguel and Kremer (2004). This is where spot checks would have been useful, but would have come at possibly too large a price for this experiment. Not for citation without explicit permission from the authors. 24

25 Program impacts on schooling: Attainment Educational Attainment (1) (2) (3) (4) (5) (6) (7) highest grade PSLC JCE MSCE completed Take Pass Take Pass Take Pass Conditional treatment 0.125* ** 0.044* (0.066) (0.022) (0.024) (0.027) (0.026) (0.015) (0.011) Unconditional treatment (0.108) (0.026) (0.026) (0.036) (0.032) (0.020) (0.018) Mean in Control Group Number of observations Prob > F(Conditional=Unconditional) PSLC is the primary school leaving certificate; JCE is the Junior Certificate Examination; and MSCE is the Malawi Schools Certificate of Education. Regressions are OLS models using Round 3 data with robust standard errors clustered at the EA level. All regressions are weighted to make the results representative of the target population in study EAs. Baseline values of the following variables are included as controls in the regression analysis: age dummies, strata dummies, household assets, highest grade completed, and ever had sex. Parameter estimates statistically different than zero at 99% (***), 95% (**), and 90% (*). Not for citation without explicit permission from the authors. 25

26 Program impacts on schooling: Attainment The impact on highest grade completed is roughly the same: Approximately one additional year for one student out of 10. CCT beneficiaries seem more likely to take the JCE exam and pass it, although the results are only suggestive (p=0.24 and 0.19, respectively). Not for citation without explicit permission from the authors. 26

27 Program impacts on schooling: Achievement Educational Attainment (1) (2) (3) (4) Cognitive Score Math Score TIMMS English Score Conditional treatment 0.180*** 0.094* 0.111* 0.131** (0.048) (0.055) (0.066) (0.054) Unconditional treatment (0.122) (0.095) (0.101) (0.091) Observations Prob > F(Conditional=Unconditional) Cognitive score is based on a test of Raven's Progressive Matrices. Math and English reading comprehension tests were developed based on the Malawian school curricula. Five questions from TIMMS mathematics tests (four from the Fourth Grade test and one from the Eight Grade test) were added to the Math test. All test scores have been standardized to have mean and standard deviation equal to zero and one, respectively. Regressions are OLS models using Round 3 data with robust standard errors clustered at the EA level. All regressions are weighted to make the results representative of the target population in study EAs. Baseline values of the following variables are included as controls in the regression analysis: age dummies, strata dummies, household assets, highest grade completed, ever had sex, and whether the respondent participated in the pilot phase of the development of testing instruments. Parameter estimates statistically different than zero at 99% (***), 95% (**), and 90% (*). Not for citation without explicit permission from the authors. 27

28 Program impacts on schooling: Achievement CCT beneficiaries performed significantly better in mathematics, English reading comprehension, and cognitive skills tests: The improvements varied between 0.1 and 0.2 standard deviations. There is no sign that UCT beneficiaries are performing better than the control group: Cognitive skills may have improved slightly. But, these improved also for untreated girls in treatment communities (spillover effects). Not for citation without explicit permission from the authors. 28

29 Summary of findings on Schooling: We detect on difference between CCT and UCT in school enrollment, attendance (self-reported or as reported by the teachers), or grade attainment. Evidence is only mildly suggestive that CCT beneficiaries were more likely to sit for and pass junior secondary school certification exams. Learning outcomes improved in the CCT arm compared to the UCT arm, but the difference is only significant for English reading comprehension. Not for citation without explicit permission from the authors. 29

30 Possible explanations of schooling impacts How can the CCT arm perform better more than the UCT arm without any improvements in attendance? 1. CCT beneficiaries are not spending more time studying and doing homework than UCT beneficiaries (time use data). 2. Perhaps they attend more regularly (above the 80% margin) and pay more attention at school. But, we cannot measure this Not for citation without explicit permission from the authors. 30

31 Possible explanations of schooling impacts How can the CCT arm perform better more than the UCT arm without any improvements in attendance? 3. There is strong evidence that all program beneficiaries eschew low quality government schools for the private sector at the secondary level, but that CCT beneficiaries go to better schools than UCT beneficiaries. These schools: i. have teachers who are more qualified; ii. have a higher average passing rate for JCE; iii. are more likely to be approved (regulated) by the government; iv. But, they are not significantly more expensive than the UCT schools. 4. Reasons for this? Other ideas? Not for citation without explicit permission from the authors. 31

32 Program impacts on marriage and pregnancy (1) (2) Ever Married Ever Pregnant Year 1 Time trend 0.043*** 0.066*** (0.007) (0.008) Conditional treatment (0.013) (0.015) Unconditional treatment ** (0.012) (0.014) Prob > F(Conditional=Unconditional) Year 2 Time trend 0.181*** 0.224*** (0.016) (0.016) Conditional treatment (0.031) (0.035) Unconditional treatment *** *** (0.027) (0.029) Prob > F(Conditional=Unconditional) Number of observations 6,255 6,265 Not for citation without explicit permission from the authors. 32

33 Possible explanations of marriage impacts Unconditional transfers protected girls who were at high risk of dropping out of school at baseline from getting married. Conditional transfers had no such impact. Disutility from schooling? Furthermore, as transfers offered to the parents were increased (randomly), the probability of marriage at R3 declined significantly in the UCT group. Again, no such effect was present in the CCT group. Not for citation without explicit permission from the authors. 33

34 Program impacts on mental health Psychological Distress GHQ 0/12 Baseline Schoolgirls Baseline Schoolgirls (1) (2) Unconditional treatment *** *** (0.034) (0.229) Conditional treatment * * (0.030) (0.175) Constant 0.339*** 2.597*** (0.070) (0.441) Observations Mean in control P-Value: Cond treatment = Unc treatment Not for citation without explicit permission from the authors. 34

35 Possible explanations of mental health impacts While mental health improved significantly in both treatment arms, the improvement was twice as large in the UCT group. The difference was reflected mainly in mild psychological disorders, such as anxiety and depression (as opposed to, say, loss of confidence) Mental health steadily deteriorates in the CCT group as the amount given to the parents increases. Burden of responsibility for cash income when the stakes are sufficiently large for the rest of the HH? Beneficial spillovers for siblings Not for citation without explicit permission from the authors. 35

36 Program impacts on sexual behavior and STIs. Almost one in 10 females between ages 15 and 24 are infected with HIV in Malawi. The program had significant impacts on the sexual behavior of school-age girls, in terms of delaying sexual debut, reducing the intensity of sexual activity, and, perhaps most importantly, partner selection. Prevalence of HIV and HSV-2 at 18-month follow-up was lower by 63% and 76%, respectively in the treatment group. CCT and UCT treatments were equally effective. Not for citation without explicit permission from the authors. 36

37 Summary: pros and cons of conditionality We detect no marginal gains in school enrollment, attendance, or attainment from imposing a schooling condition to the transfers in this experiment. However, there is some evidence that girls in the CCT arm have higher achievement. On the other hand, UCT outperformed CCT in reducing marriage & pregnancy, as well as improving mental health. Finally, for a host of other outcomes, including risk of HIV infection, the two treatments performed equally well. Not for citation without explicit permission from the authors. 37

38 Summary: pros and cons of conditionality The choice of CCT vs. UCT then depends on: the weights attached to various outcomes by the social planner, and the relative costs (including political economy) of each program. However, these findings also make it clear that a schooling conditionality does not only alter school enrollment. Rather, it is costly (especially for the person bearing the burden) and can negatively affect related outcomes. Brings up intra-household issues, such as target individual. Not for citation without explicit permission from the authors. 38

39 Implications for LAC: differences and similarities Many countries in LAC are richer than Malawi by an order of magnitude. This means that small amounts of money go a long way in improving schooling outcomes in Malawi. In another paper, we conclude that the improvements in HIV are due to increased cash income rather than schooling. Counterfactual for schooling is child labor in LAC while it s mainly marriage in Malawi. Is income effect different for marriage vs. child labor? Edmonds and Schady (2009) Not for citation without explicit permission from the authors. 39

40 Implications for LAC: differences and similarities On the other hand, while fertility rates among females are still much higher in Malawi (135 per 1,000 women) than Ecuador, Brazil, or Mexico (83, 76, and 65 per 1,000 women, respectively), teen pregnancy is still considered a big problem in LAC. It is also possible that parent-child conflict plays a crucial role in schooling (as well as marriage and child labor) decisions in both settings. Bursztyn (2009) program design issues re: target individual within the HH. Not for citation without explicit permission from the authors. 40

41 THE END Not for citation without explicit permission from the authors. 41

42 Offer Letters Conditional Transfers The Zomba Cash Transfer Program (ZCTP) with funding from the World Bank would like to offer you, _[NAME], a cash transfer to help you and your family with the burdens of school attendance for the 2009 school year. By accepting this offer, in return for going to school you will be given: You are receiving this money in order to help you return to school or stay in school. In order to receive this money you MUST attend school at least 80% of the days for which your school is in session. Unconditional transfers The Zomba Cash Transfer Program (ZCTP), with funding from the World Bank, would like to offer you, [NAME], a cash transfer to help you and your family. By accepting this offer you will be given: These monthly transfer amounts specified above are given to you as a result of a lottery. You are not required to do anything more to receive this money. You will receive this money for 10 months between February and November, Not for citation without explicit permission from the authors. 42

43 1. The rules of the program were well understood by the girls in the UCT arm: Not for citation without explicit permission from the authors. 43

44 2. Girls in the UCT arm knew about the CCT arm: Int: Earlier you talked of conditional and unconditional. What did you say about the rules for conditional girls? Res: They had to attend class all the time not missing more than 3 days of classes in a week like I already explained. Int: How did you say the program managers knew about the missed school days? Res: They would go to the schools For example, I have a friend, Jane [not real name], who was learning at Samama. They would go each month to the school to monitor her attendance, and if she was absent for more than three days she would not get her monthly money. Not for citation without explicit permission from the authors. 44

45 Summary of the conditions under which the UCT experiment took place: 1. However, UCT beneficiaries fully understood their treatment status and were never worried about not receiving their payments due to school attendance. 2. However, the UCT experiment did not happen in a vacuum. It took place in a district where a CCT program was simultaneously running in neighboring communities. Hence, the UCT experiment took place under a rubric of education. Not for citation without explicit permission from the authors. 45

46 Table: Marriage and Pregnancy (1) (2) Ever Married Ever Pregnant Transfers to Girls Conditional treatment (0.009) (0.016) Unconditional Treatment (0.011) (0.014) Prob > F(Conditional=Unconditional) Transfers to Parents/Guardians Conditional treatment (0.010) (0.014) Unconditional Treatment ** (0.007) (0.008) Prob > F(Conditional=Unconditional) Conditional Treatment ($5/month total) (0.053) (0.072) Unconditional Treatment ($5/month total) * (0.032) (0.037) Number of observations 4,166 4,177 Not for citation without explicit permission from the authors. 46

47 Are we missing something? 1. If we had conducted the UCT/CCT experiment among the baseline dropouts, would we have found a significant price effect? The answer, from Round 2 (Round 3 data are under analysis), is no. 2. Is it possible that there is contamination? Analysis of impact in the UCT group by the share of CCT group at the monthly cash transfer point suggests no significant enrollment spillovers from CCT to UCT. Not for citation without explicit permission from the authors. 47