Design of an Impact Study to Evaluate the Scaling up of the WFP Voucher Scheme Dr. Helen Guyatt, Head of Research helen.guyatt@kimetrica.com www.kimetrica.com
Develop a set of analytical tools: To inform on the scale up of vouchers, and help WFP determine the most effective mix between GFD and vouchers, given available resourcing.
Research objective To investigate how the scaling up of the voucher scheme will affect the net distribution of costs and benefits for: refugees traders host community With particular emphasis on: vulnerable groups effects on markets (esp. prices) influence of exogenous factors (e.g. food cuts, security restrictions, changes in camp population)
Roadmap Stakeholder interviews Secondary data analysis Theory of Change (ToC) definition Model development Definition of key research questions Key indicator and variable identification Evaluation of WFP s proposed M&E plan Design of data collection instruments Data analysis plan
Research Design Process
Primary data analysis: Stakeholder interviews
Stakeholder interviews In Nairobi, Kakuma and Dadaab. Donors (Dfid, ECHO, USAID, Germany) FAO WFP officials Implementing Partners (World Vision, CARE, NRC) UNHCR DRA County Government Traders in the camps
Key findings Benefits and risks may be unevenly distributed; voucher scheme may marginalize vulnerable groups (e.g. HH size 1, new arrivals, HHs with no phones, traders not in the camps or not in the scheme). Cereal component of the ration will affect resale behavior, demand and prices for different cereal types in the market. Exogenous factors may influence costs, benefits and risks and should be monitored (eg. ration cuts, security restrictions). Conflicts, particularly related to intra-hh decision-making, may increase. Commodity prices can change with season. Need to monitor Kakuma and Dadaab are very different and should be considered separately
Main differences between Kakuma and Dadaab Kakuma Dadaab Culturally Multi-cultural Mostly Somali Nutritional status Low Moderate Markets Rely heavily on Kitale More developed Transport and access Good road from Kitale Poor road from Garissa Host community tensions High (resources) Low (business)
Secondary data analysis
Secondary data analysis Refugee demographic data Nutrition and health surveys Market monitoring Voucher documentation Cost data Other cash/voucher programmes Other activities in the camps Estimated elasticities of food items
UNHCR demographic data Kakuma Dadaab Total population 184,183 351,538 Number of HHs 54,836 83,277 HH size 1 31% na Originating from Somalia 31% 95% Arrivals (2014) 45,627 na Departures (2014) 3,048 na By sub-camp: new arrivals, Country of Origin, HH size 1, female headed HH?
Annual nutrition surveys, FSOM and HIS Food insecurity varies by camp and subcamp e.g. Kakuma >Dadaab, Ifo2 and Kambioos. Food ration lasts < 11 days Phone access < in HHs with lower FCS (38% phones in Kambioos) Price of minimum healthy food basket (2013) higher in Kakuma
Market monitoring : retail prices Vary by camp, sub-camp and season for meat and milk May 2014 Sep 2014 Dec 2014 Rice 93 (78-120) [7] 95 (79-199) [5] 109 (100-119) [3] Dry beans 81 (70-95) [7] 76 (38-98) [5] 78 (70-84) [3] Goat meat 361 (220-400) [6] 398 (390-400) [4] 484 (480-488) [2] Goat milk 67 (59-80) [4] 65 (60-70) [2] 75 (-) [1] Tomato 110 (96-123) [6] 104 (48-138) [4] 99 (78,120) [2] Bananas 102 (60-170) [5] 67 (20-120) [4] 88 (46,130) [2] Missing: Fish, potatoes, pasta, sukuma, eggs? Wholesale prices? Suppliers?
Voucher programme documentation Market assessments, FFV evaluation and FGDs 30% phone ownership in Kakuma 4 Higher prices charged to FFV customers Host community shop in the camps Prices and availability seasonal Can traders meet increased cereal demand?
Cost data WFP costs are aggregated into composite unit prices which include purchase, transport and overheads Prices for each commodity in each camp is assumed to be the same Unit prices are all expressed per metric tonne distributed: this is a problem for some resources (e.g. staff at FDP)
Other cash/voucher experiences in refugee camps Somalia, Sudan, Burundi and Rwanda Traders need to be regularly monitored Markets suffer if a break in the voucher pipeline occurs Explore reductions in HH debt Reduce mobile phone loss by insisting on police reports Relative impacts on nutrition depend on baseline values
Other activities/surveys in the camps Activity Details Will affect WFP supplementary feeding MCHN, food4assets HH food security UNHCR NFI targeting Female headed HH HH SE HSNP2 cash targeting 40,000 HHs Turkana HH SE host community FAO-Mastercard charcoal 8,000 refugees Kakuma HH income NDMA Livelihood surveys Supplementary data World Bank SE survey Supplementary data
Estimating elasticities for food supply and demand To calibrate the model : farm gate, retail and wholesale levels For retail demand elasticities, budget share can be specified as: S i = a i + b i ln M n æ ö ç + åg ij ln P j +e i è P * ø FSOM HH expenditure not disaggregated into quantities and unit price j=1 Where: S i is the i th budget share estimated as S i = P j X j /M P j are nominal retail prices X j are quantities γ ij are price coefficients M is the total expenditure on all goods P* is an aggregate price index
Implications for the study design HH survey : Expenditure on all relevant commodities, disaggregated into quantity and unit cost alongside HH demographic data Market/trader survey : Prices on all relevant commodities from all markets and sub-camps Producer/farmer survey or secondary data : farm prices, acreage, output and input prices (e.g. labour)
Theory of change Achieving impacts through outcomes and outputs
Assumptions Refugees benefit if: Refugees can access and use vouchers Traders supply food demanded at acceptable quality and price Refugees purchase some non-gfd commodities with voucher Traders benefit if: Refugees use markets as above (a risk that some traders may lose business) Traders can access MPESA agents and are not targeted by thieves
Major risk: Prices may increase As a result of limited supply or exogenous factors Resulting in negative effects on HHs and markets Potentially causing increased tension and conflicts Minor risk: Diminished employment opp. For porters and temporary staff at the FDPs Adversely affecting HH incomes for these specific groups
Exogenous factors Food rations: Cuts and composition may affect HH spending behavior Insecurity: Curfews affect trading hours and transport costs, the closing of the Somali border has increased prices, and the discontinuation of money transferring modalities has affected remittances Other programmes: Receipt of other food, NFIs or cash aid should be recorded Other: Weather conditions, wider market conditions
Model development
Cost and benefit trade-offs Refugees Host consumers Reduced GFD Increased Vouchers Net Benefit GFD volume Unit price GFD commodity prices Income Prices for all items Non-GFD commodity prices Traders GFD resale volume Non-GFD commodity sales Local producers Improved market for GFD commodities Improved market for non-gfd commodities Probable net gain Net loss Probable net gain Net gain
Anticipated changes for GFD commodities Increase in prices (P2) Net shift in supply curve (S2)
Anticipated changes for non - GFD commodities Increased income shifts demand (D2) ST: supply fixed, and prices increase (P2) LT: quantity increases (Q2), and prices stabilize (P3)
General equilibrium model Static model : calibrated to monthly or seasonal data One model for each camp: calibrated with campspecific data Markets (c): all sub-camps, host community rep, outside rep. Commodities (i): set of agricultural commodities (e.g. maize, meat, fish etc.), transport service and other good Household types (h): in sub-camps differentiated by eg. income, labor type, ethnic group Production factors (f): e.g. labor, capital, land
Model objective Compute price responses to voucher injection and compare with in-kind GFD
Consumption and prices Households: maximize their welfare subject to budget constraints Vouchers: modeled as imperfect substitutes to food commodities In-kind: modeled as additional local production, monetized using market prices Production and transport Goods: assumed to be purchased from cheapest source Agricultural commodities: produced by combining labour with capital and/or land Transport services: provided by traders using labour and capital
Key research question How will the voucher scheme affect the net distribution of cost and benefits as it scales up?
Components of net distribution Distribution of costs and benefits: across all groups (refugees, traders, host community) and across income and voucher and food market access strata Impact on market growth: including up the value chain to imported goods Influence of exogenous factors Risk of price increases: negatively affecting refugees and host population
Gap Analysis What are the key indicators?
Research Design Process
Key indicator identification Indicator HH food security HH consumption HH income Trader income Tensions & conflicts Access to voucher Access to food Access to income Measurement Standard metrics (e.g. FCS, DD, daily food energy avail.) Patterns in markets and HHs Direct and indirect from HH expenditure Direct and indirect from increased employment Police reports and intra-hh Phone ownership, pipeline etc. Prices & quality, resale of ration No thefts, adequate supply chain
Key variable identification Variable/outcome Measurement Exogenous factors Ration cuts, income losses, supply constraints Costs Cash and opportunity costs, provider costs Prices Commodities, price indexes and inflation Supply Into camps and outlying host communities Real income HH income deflated by price index Welfare % change or monetized equivalent
Can proposed WFP M&E provide this information? Tools Limitations Beneficiary contact monitoring 20 per FDP FSOM HH survey 139 HHs (Kakuma 1-3), HH size 1 = 2% Voice monitoring telephone 150/month, but need a phone FSOM market monitoring Missing items, wholesale prices and sub-camps MVAM-SMS 500/week, but need a phone Mystery shopping, trader feedback, FGDs?
Data collection and analysis 1. HH survey 2. Trader survey 3. MM survey
HH Survey Section Interview and HH details Wealth indicators HH roster Food consumption Expenditure GFD collection Voucher Coping strategies/income Details Sub-camp, HHH, CoO, arrival, relatives Housing, cooking, electricity, assets HH profile, age, sex, schooling, other nutrition prog. Quantities and changes in demand with price Food, NFIs, capital Resale, duration, opp. costs Purchasing behavior, opp. costs CSI, income (work, gifts), debts
Proposed HH survey sampling 400 refugee HHs to be randomly sampled in each sub-camp 1. UNHCR list of all HHs with location 2. Sampling grid over each sub-camp and the start at center and move outwards Shortened HH survey: wealth and consumption in a random sample of host community in the main town (100) and in the next nearest town down the supply chain (100).
Trader Survey Section Trader details Voucher experience Main food traded Employees Assets Details CoO, location, type of shop Use and opportunity costs Demand and prices, suppliers Number, wages Transport and stores
Trader survey 400 traders randomly selected across the camps from list? 50 traders from the main town outside the camps. Concurrent with HH survey Builds on FFV
Market monitoring Every market in every sub-camp, main town, and next market town. Conducted monthly. Retail and wholesale prices Comprehensive list of commodities incl. eggs, fish, pasta and potatoes. Demand, availability and main suppliers
Cost data Provider costs (WFP) : voucher versus in-kind Unit costs and quantities explicitly Itemized frameworks Set-up versus recurrent costs Fixed versus variable costs By activity: procurement, sensitization, training, monitoring, distribution By resources: staff, capital, rental, consumables etc.
Supporting data Triangulation with WFP M&E framework Triangulation with other secondary data sources e.g. KHIBS, the World Bank SE survey in Kakuma and the NDMA Systematic monitoring of conflicts using police reports KI with local producers: prices, inputs and outputs Qualitative data Regular FGDs with host community, refugees and traders every 6 months Security restrictions, adverse conditions affecting supply of commodities, changes in the camp population and GFD composition and ration cuts
Data analysis plan Food security: FCS, DD, daily food energy availability per capita Welfare estimates : based on commodities consumed (quantities and values) and HH income. Model benchmark values: consumption, production and transport Model response parameters eg. Elasticiity of substitution between goods Combine programme cost data with model outcomes on percentage change in welfare (or its monetized equivalent) for different HH types.
Develop a set of analytical tools: To inform on the scale up of vouchers, and help WFP determine the most effective mix between GFD and vouchers, given available resourcing. Under what conditions (e.g. critical price level) should vouchers be scaled back?
Thank you helen.guyatt@kimetrica.com www.kimetrica.com