8 WHEN NATIONAL PRIDE IS BEYOND FACTS Navigating Conflicting Stakeholder Requirements Felix Muramutsa Felix Muramutsa, MA Psychology, is a freelance consultant with a background of more than 20 years in evaluation, monitoring, and social research in various domains. 51
52 Part II Engage Stakeholders The accusation was blunt: How can you act like a foreigner? It started out as a standard evaluation, but somewhere along the way I began to be treated by some officials as the accomplice of foreigners that wanted to destroy my country s economy! Let me tell you a story that involves navigating the tricky relationships between international donors and government in collecting baseline findings. ABOUT ME My name is Felix Muramutsa. I m a freelance consultant holding 20 years of experience with more than 30 assignments in research, monitoring and evaluation. I have worked with UN agencies, U.S.-funded projects, international and local nongovernmental organizations (NGOs), civil society organizations, government institutions, foreign and national universities, and the private sector. I have a master s degree in psychology and a bachelor s degree in education. DESCRIPTION OF THE EVALUATION The evaluation I m about to relate was a baseline prevalence survey of a child labour project run by an international NGO (INGO) for which I was working as senior monitoring and evaluation (M&E) advisor and deputy director. The purpose of the evaluation was to assess the prevalence of child labour in the project s zone of intervention, in order to set up benchmarks and other key indicators of project implementation. The scope of the evaluation was to assess the prevalence of child labour in the country s tea sector prior to project intervention. The major stakeholders were a foreign government donor agency that funded the project, the government (represented by the Ministry of Labour), tea companies, tea cooperatives, district and local authorities, the implementing international organization, local NGOs, and beneficiary parents and children. My role as the senior M&E advisor was to coordinate the overall baseline survey and ensure the consultancy company hired to conduct it delivered accordingly, respecting the expectations of the donor, INGO, and government. The role of government staff was to ensure the baseline data reflected the local reality. The role of local authorities was to ensure the participation of the sampled population in the survey. The tea companies and cooperatives were involved to ensure the baseline data reflected their attempts at combating child labour in the tea sector. Although there were laws, regulations, and policies in place to prevent child labour, enforcing it was very challenging on the ground. And the role of the implementing partners, including local NGOs and a federation of cooperatives, was to support the evaluation team in their respective zones of intervention. As the overall baseline coordinator, I had to ensure that all stakeholders roles and expectations were aligned. I participated in defining the Terms of Reference, selected the consultancy company, and trained and supervised the research team.
Chapter 8 When National Pride Is Beyond Facts 53 I also had the difficult task of liaising between the INGO s headquarters back in the United States, the foreign donor agency, and the government. THE MISTAKE OR CHALLENGE The complication I faced was that the requirements of the donor and the INGO sometimes conflicted with those of the government, and my job as the evaluation facilitator was to respect them all. For example, as the funder of the survey, the donor considered that the baseline results were first and foremost their business only. However, the government also required me to inform them of all the steps occurring and expected me to validate the findings with them prior to publication. In fact, all major household studies such as this one were required to go through an approval process with the National Institute of Statistics and the National Ethics Committee, particularly if children are involved. But I was instructed by my employer (the INGO) to not share the preliminary findings with the government. Instead, they were discussed and validated by the consultancy company, the INGO, and the foreign donor agency. Later on, the INGO shared a final version with the government, and I was tasked to follow up with dissemination. The alert came when the local NGOs and tea cooperatives started challenging baseline results during meetings with government officials. Tea is a very important product of the country, and the government was worried that a high prevalence of child labour would hamper tea exports and hence the national economy. To my dismay, the government questioned our methodology, sampling, and findings and ultimately rejected the report. I was personally caught in the middle, with each side requiring me to convince the other of their position. I believe things went wrong when I failed to question and, instead, accepted instructions from the INGO and donor not to involve the government in all steps of the baseline survey. I should have known there would be complications at the validation stage. What contributed to this mistake was my blind respect of the donor and INGO s request to prioritize their interests before the interests of the government, all the while knowing it conflicted with the government s requirements of being involved at all stages of the project, including the baseline. There were, however, some factors not under my control. The decision to not share the preliminary findings with the government prior to sharing them with Jana Curll
54 Part II Engage Stakeholders the donor was made by my employer, the INGO. If I had chosen to ignore their instructions, I would have been in breach of my contract. But informing my employer of the government s expectations was under my control. I did it, but only informally and probably not sufficiently. When things began to go wrong, I became a scapegoat. I was very frustrated to be treated by certain government officials as an accomplice of foreigners that want to destroy the country s economy by exaggerating a fake child labour prevalence. I was shocked, as I knew that was not at all my intention! To address the situation, I held a number of personal conversations in and out of the office with government officials and other key players to better understand the problem. I learned that it was not a methodological issue, but rather a fear of how the report would be perceived by higher officials. We discussed what improvements needed to be made to the draft report and what could be reviewed and/or presented differently. Nevertheless, it was a dilemma, and I continued to struggle with selecting which information to disclose to which side and the best way to reach a compromise. Although the baseline survey was conducted and completed in 2014, it was not validated until the end of the project in April 2017. The situation worsened to become more than a technical issue between the INGO, donor, and the government. In fact, in the end the donor had to engage their own embassy to resolve the issue with the national government as a bilateral political issue. LESSONS LEARNED Thinking back on this situation, there are several things I would have done differently to avoid these mistakes. Involve all stakeholders. As an evaluator, this awful situation has influenced my work significantly as it has sharpened me to better understand that the early involvement and active participation of all stakeholders is key for a successful evaluation. More involvement from stakeholders would have helped me to assess everyone s expectations, plus anticipate and resolve any conflicting agendas. Set up a steering committee. In retrospect, I should have pushed to set up a joint baseline steering committee. The role of the steering committee would be to anticipate and discuss any issues that might surface at all levels: technical, administrative, or political, including contractual issues that might limit the evaluation. Advocate when necessary. Ensure that the donors and local government have the same understanding of the evaluation s terms of reference, objectives, methodology, and preliminary findings. Be active in advocating and pushing donors to understand local government and
Chapter 8 When National Pride Is Beyond Facts 55 authorities requirements and expectations and, conversely, help local government and authorities to accommodate donor and INGO deadlines and deliverables. It s a continuous and balancing role for any evaluator. REFLECTIVE QUESTIONS 1. Why do you think the international donor and INGO insisted on viewing the results prior to the government? How do you think the evaluator could have addressed this situation more proactively? 2. How can evaluators balance conflicting expectations and requirements in general between donors, employers and/or contractors, and the recipient government? 3. What protective measures might support an evaluator in the middle of conflicting stakeholder agendas?