M&E for Better Global Health: Learning What Works, For Whom, and in What Circumstances: A Conversation with Juan-Carlos Alegre

Juan-Carlos Alegre

Monitoring and evaluation (M&E) systems have played a critical role in advancing the field of global health, from applying quantitative and qualitative methods in collecting and using health data, to informing decision making, applying rigorous evaluations in assessing program effectiveness, and designing and conducting operational research that address implementation challenges.

MSH spoke with Juan-Carlos Alegre, MS, MSH’s director of results management, impact and learning, on MSH’s ongoing contributions and efforts to improve M&E in global health.

How is M&E helping to advance the field of global health?

M&E systems have helped the field of global health improve significantly in various health disease areas that affect low and middle-income countries (LMICs). By establishing strong monitoring systems to assess performance, strengthening health information systems at facility- and community-level to use data, and by tracking progress towards achievement of health targets, the field of M&E has contributed in the achievement of health outcomes and saving lives. Two tangible examples include improvements in the management of childhood diseases and HIV and tuberculosis (TB) co-infection. The number of deaths of young children globally has declined from 12 million in 1990 to 7.6 million in 2010. And the Joint United Nations Programme on HIV/AIDS (UNAIDS) reported a 13 percent reduction in TB-associated HIV deaths in the last two years, due in part to 45 percent increase in the number of people co-infected with HIV and TB accessing antiretroviral therapy (ART) between 2009 and 2011. M&E has also helped the global health community better understand complex health systems and the need to adopt a more holistic approach from vertical programs to diagonal approaches to improve the performance of health systems and the attainment of health outcomes.

Major donor agencies, including the U.S. Agency for International Development (USAID) and the U.K. Department for International Development (DFID), are updating their evaluation policies, focusing on performance, evidence of effectiveness, and impact. MSH and other international non-governmental organizations are increasing efforts and resources for monitoring, evaluation, and implementation research to bring about sustainable solutions to address global health problems in LMICs.

Despite these major advances, there are still major challenges to address for which M&E can significantly help. As 2015 is fast approaching — and with a significant number of countries expected to not achieve health-related Millennium Development Goals (MDGs) — M&E is poised to continue playing a prominent role for monitoring performance, accountability, and most importantly for learning what works, for whom and under what conditions for various health disease areas.

What is MSH’s approach to monitoring & evaluation? How is MSH contributing to the global health improvements through M&E?

We realize that monitoring process and performance is as important as evaluating high-level project effectiveness and long-term impact. The M&E function of our projects systematically informs management decision-making toward prioritizing and using cost-effective, high-impact interventions for sustainable solutions in the complex settings where we work. We are strengthening our M&E systems to ensure our projects continue to be responsive to the contextual needs and priorities of local health systems by systematically, rigorously, and ethically collecting and using high-quality data.

In addition, the role of health system research is critical to generate plausible evidence of success of innovations we use — whether for improving pharmaceutical management, bringing effective service delivery models, or improving leadership and management practices for healthcare providers and managers of public sector institutions and civil society organizations that have significant roles in the delivery of healthcare services in the countries where we work.

Efforts to improve using real-time data will help us further learn and respond to complex and adaptive health systems and appropriately deliver evidence-based, life-saving interventions in LMICs. Unpacking the “black box” – through monitoring process and performance – is critical to better understand how health systems work and to have more holistic and sustainable approaches to address global health problems that are still affecting LMICs.

MSH will be actively participating in the new Health Systems Global society recently established in Beijing to contribute to advancing the state-of-the-art evidence and knowledge translation of health systems research for universal health coverage, particularly on the critical role of access to medicines and health systems strengthening in fragile states.

We are also improving our evaluation practices for performance accountability and learning at different levels – projects, countries, operational units within MSH and at the organizational level. As we are beginning our new strategic roadmap, we recognize and value the importance of creating an “evaluative thinking” culture: being results-oriented, reflective, questioning our approaches and models and collecting and using evidence systematically to test our assumptions when delivering projects in the various health areas and strengthening health systems for achieving greater health impact.

Can you provide some examples of successes or challenges of M&E from MSH programs?

With support from USAID, the Healthy Communities & Municipalities project, led by MSH in Peru, has developed a system to empower communities to monitor and track their own health needs. The system, “SISMUNI,” is also allowing communities to link with local governments and the health system at the regional level to prioritize resources to meet the health needs of the communities.

In West Africa, MSH and partners of the USAID-funded AWARE II project devised a regional online database to track and share results from implementing high-impact practices in family planning and reproductive health, HIV & AIDS, and maternal, neonatal and child health in a number of countries in the region. The database helped to establish a transparent way for the project partners and USAID to monitor and track progress towards anticipated results. This is an example of how we can improve our monitoring systems to provide as close as real-time data to make informed management decisions as the project is implemented.

What is the future of M&E for global health?

There is a growing acknowledgement that global health programs require strong and robust M&E systems and the in-country capacity to develop them in a number of priority health areas and technical strategies for which more evidence of what works, how, for whom and under what conditions is needed.

One of these areas is leadership and governance, a critical building block for strengthening health systems, yet probably the most challenging one to improve. We need to develop robust measures of success of this health system building block to improve health outcomes.

Two health areas that need significantly more attention are neonatal health and maternal health. More implementation research is needed to demonstrate how evidence-based interventions for these priority health areas can be successfully delivered in low-resource settings. In addition, as we anticipate universal health coverage (UHC) playing a prominent role in the post-MDG era for global health, it is a priority to develop methods for measuring UHC in LMICs.

The field of M&E for global health should also produce better ways to assess how these interventions and technical strategies are delivered in complex adaptive ecologies and systems. The new and emerging evaluation methods (i.e., developmental evaluation, realist evaluation, contribution analysis to name a few) and systems approaches should help us to better evaluate performance and effectiveness of global health programs in our search for the real gold standard: the methodological appropriateness needed for the kind of evaluation and research questions donors, researchers, implementing partners, global practitioners, and policy makers seek to answer to strengthen health systems and achieve greater health impact.