advanced search

<< Back To Home

On Bathwater, Babies, and Designing Programs for Impact: Evaluations of the Integrated Community Case Management Strategy in Burkina Faso, Ethiopia, and Malawi

Wednesday, 2nd of March 2016

 

On Bathwater, Babies, and Designing Programs for Impact: Evaluations of the Integrated Community Case Management Strategy in Burkina Faso, Ethiopia, and Malawi

 

1. 1.       Elizabeth Hazel*, 
2. 2.       Jennifer Bryce, 
3. 3.       the IIP-JHU iCCM Evaluation Working Group

+Author Affiliations

1. Institute for International Programs, Johns Hopkins University, Baltimore, Maryland

+Author Notes

• Authors´ addresses: Elizabeth Hazel and Jennifer Bryce, Institute for International Programs, Johns Hopkins University (IIP-JHU), Baltimore, MD, E-mails: ehazel1@jhu.edu and jbryce1@jhu.edu.

1. ↵* Address correspondence to Elizabeth Hazel, Institute for International Programs, Johns Hopkins University (IIP-JHU), 615 North Wolfe Street, Baltimore, MD 21205. E-mail: ehazel1@jhu.edu

 

Best viewed at http://www.ajtmh.org/content/94/3/568.full.pdf+html

 

 

The adage “don´t throw the baby out with the bathwater” advises us not to discard the essential along with the unessential, whether due to impatience or frustration.1 This issue of the journal includes reports on prospective evaluations of the integrated community case management (iCCM) of childhood illness strategy in Burkina Faso,2 Ethiopia,3 and Malawi.4 iCCM seeks to reduce child mortality by making effective treatment of pneumonia, diarrhea, and malaria available from trained health workers at community level.5 In this commentary, we show results from the three countries, highlighting hard-won lessons about iCCM´s potential to prevent unnecessary deaths among children.

The independent iCCM evaluations were designed in 2009–2010, as a part of the Catalytic Initiative to Save a Million Lives6 spearheaded by Canada and involving the Bill & Melinda Gates Foundation, the U.K. Department for International Development, the United Nations Children´s Fund, the World Health Organization, the Doris Duke Charitable Foundation, and others. The goal of the Catalytic Initiative was to support and measure the impact of strong, coordinated efforts to deliver high-impact interventions to reduce under-five mortality in low-income, high-burden countries, with a particular focus on providing treatment for childhood pneumonia, diarrhea, and malaria at community level using iCCM.

The evaluation in each setting was conducted in collaboration with local research institutions, based on a locally adapted version of the common evaluation framework for maternal, newborn, and child health (MNCH).7 The evaluation designs were built on impact models reflecting the pathway to child survival. Beginning with policy and program inputs, the pathway moves to project implementation and intermediate outputs such as improved provision of quality services, strengthened health systems, and increased utilization. Continuing to coverage and behavioral outcomes, the pathway culminates in impact of under-five and postneonatal mortality. We focus on mortality among children aged 2–59 months here, because the iCCM strategy did not include guidelines for children under 2 months of age. Detailed methods are described in the individual papers; Table 1 provides a summary. All sites documented program inputs and contextual factors that may have affected program implementation or effectiveness. The evaluation teams shared intermediate and final results with governments and partners as a basis for strengthening iCCM implementation.

View this table:

• In this window

 

• In a new window

Table 1

Summary of evaluation designs

Selected results across the three countries are shown in Table 2. We reanalyzed some variables to maximize comparability across countries; in some instances, results presented here are not included in the series papers.

View this table:

• In this window

 

• In a new window

Table 2

Cross-country comparison of evaluation findings

iCCM was not associated with accelerated mortality declines in children aged 2–59 months during the 2- to 4-year evaluation periods in these three settings. The reasons for this lack of association likely varied across settings, as does the likelihood that these programs may achieve measureable impact after additional years of sustained implementation.

In Burkina Faso, iCCM implementation did not reflect current best practice for designing and implementing effective MNCH programs. The program relied on community health worker (CHW) lay volunteers. The initial training was organized as a “cascade” in which representatives from districts were trained and then requested to organize subdistrict-level training sessions to train over 3,000 CHWs in the space of a few months, with minimal support or supervision. The training included written materials even though many of the CHWs were illiterate. Systems for ensuring the availability of drugs and reinforcing health worker performance were not consistently in place at the time of training, or indeed throughout the evaluated implementation period. There were no systematic efforts to generate community demand for and utilization of iCCM services. Not surprisingly, then, the quality of iCCM service provision was poor, the services were grossly underutilized, and there was no improvement in intervention areas in the proportion of sick children who received correct treatment of their disease (Table 2). The Burkina Faso iCCM program will not have an impact unless it is redesigned to fully capacitate health workers and educate the community to increase demand. Greater attention to accountability and best practices in women´s and children´s health should help prevent similar examples of incomplete and therefore ineffective programs in the future.12

However, in both Malawi and Ethiopia, iCCM was implemented more strongly and achieved important intermediate outputs. In both settings, government policies were in place to support full implementation of the strategy, and high numbers of paid community-level workers were well trained and initially well equipped. The percent of children receiving correct treatment from CHW for iCCM illnesses was moderately high in Ethiopia (78%) and lower in Malawi (63%) (Table 2) but was at least as good as the care received in first-level health facilities.11,13 In both settings, unfortunately, children with more severe illnesses were less likely than those without life-threatening conditions to be managed correctly, pointing to continuing needs for refinement of the iCCM approach to improve and sustain health worker performance.11,13 Health systems supports for iCCM were quite strong in Ethiopia, particularly high levels of clinical supervision, which requires further investigation to generate best practice models. Careful monitoring was used to generate remedial actions when problems were identified. In Malawi, only 58% of iCCM workers received clinical supervision and only 38% reported no stock-outs of essential iCCM drugs in the previous 3 months.4 Although far from perfect, both Ethiopia and Malawi implemented iCCM strongly and at scale, providing an important foundation for continued improvement.

However, in both Ethiopia and Malawi, there was no change in overall care seeking for iCCM illnesses and no impact on mortality (Table 2). The evaluation results also make clear that the potential effectiveness of iCCM was capped by low utilization. At baseline in Ethiopia, only 23% of mothers took their sick child to any formal provider for care, and care seeking rates did not increase over time. In contrast, care seeking in Malawi was quite high at baseline and remained the same at endline. Care seeking from iCCM providers increased slightly in both settings, but at such low absolute proportions that no population impact could be expected. No matter how strong iCCM service provision may be, the strategy cannot save lives unless mothers and other caregivers take their children for care. Future iCCM programs must be designed with this in mind, and include strong, locally defined components designed to increase care seeking and utilization.

In Malawi, particularly, the results suggest that geographic targeting of iCCM based on physical access to fixed health facilities may no longer be effective, or at least not sufficient. iCCM in Malawi was implemented in “hard-to-reach” areas of each district, as defined by district management teams, based on distance to a health post. Before 2008, this appeared to make good public health sense, when overall coverage for treatment interventions was low. By 2010, however, there were no differentials in children´s receipt of appropriate treatment based on mothers´ reports of whether distance to a health facility was a barrier to family health care.4 The assumption that reaching the unreached is limited by geography is no longer true, at least in Malawi.

Returning to the bathwater and baby adage, the iCCM in 2015 can be considered as the baby and the challenges of fully implementing iCCM as the bathwater. Like a human infant, iCCM programs will require steadfast attention, investment, and developmental guidance to mature and achieve their mortality-reduction potential. iCCM is not the answer in all settings—where first-level facilities are accessible and are or can be improved to provide quality services, such as in urban settings, there may be no need for iCCM, unless it can be demonstrated that the strategy overcomes barriers to care seeking that go beyond physical access. In settings where iCCM is needed, further research, development, and investment will be required to ensure that implementation plans reflect an accurate understanding of “which” women and children are not using services, “how best” to reach and motivate them to seek care, and “what role” iCCM can play in this pathway to survival. Ongoing programs of evaluation and implementation research can continue to generate information about the bathwater and the parts of iCCM and its context that must be discarded or refreshed. Local capacity to collect and analyze relevant data is a prerequisite for generating essential knowledge and putting this knowledge to use in the service of women and children.

 

Next Section

• Members of the IIP-JHU iCCM Evaluation Working Group (alphabetical order): Agbessi Amouzou, Robert Black, Jennifer Bryce, Elizabeth Hazel, Rebecca Heidkamp, Kenneth Hill, Olga Joos, Lois Park, Jamie Perin, Andrew Marsh, Nathan Miller, Larry Moulton, Melinda Munos, Yvonne Tam, Timothy Roberton, Bryan Shaw, Cesar Victora, and Emily Wilson.
• © The American Society of Tropical Medicine and Hygiene

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Previous Section

 

1. 1.↵

 

1. Ammer C

, 1997. The American Heritage Dictionary of Idioms. Boston, MA:Houghton Mifflin Harcourt.

 

Google Scholar

1. 2.↵

 

1. Munos M, 
2. Guiella G, 
3. Roberton T, 
4. Maïga A, 
5. Tiendrebeogo A, 
6. Tam Y, 
7. Bryce J,
8. Baya B

, 2016. Independent evaluation of the Rapid Scale-Up program to reduce under-five mortality in Burkina Faso. Am J Trop Med Hyg 94: 584–595.

Abstract/FREE Full Text

1. 3.↵

 

1. Amouzou A, 
2. Shaw B, 
3. Miller NP, 
4. Tafesse M, 
5. Mekonnen Y, 
6. Moulton LH,
7. Bryce J, 
8. Black RE

, 2016. Effects of the integrated Community Case Management of Childhood Illness strategy on child mortality in Ethiopia: a cluster randomized trial. Am J Trop Med Hyg 94: 596–604.

Abstract/FREE Full Text

1. 4.↵

 

1. Amouzou A, 
2. Hazel E, 
3. Heidkamp R, 
4. Marsh A, 
5. Mleme T, 
6. Munthali S, 
7. Park L,
8. Banda B, 
9. Moulton LH, 
10. Black RE, 
11. Hill K, 
12. Perin J, 
13. Victora CG, 
14. Bryce J

, 2016.Independent evaluation of the integrated Community Case Management of Childhood Illness strategy in Malawi using a national evaluation platform design. Am J Trop Med Hyg 94: 574–583.

 

Abstract/FREE Full Text

1. 5.↵

 

1. Marsh DR, 
2. Pagnoni F, 
3. Peterson S

, 2012. Introduction to a special supplement: evidence for the implementation, effects, and impact of the integrated Community Case Management strategy to treat childhood infection.Am J Trop Med Hyg 87 (Suppl 5): 3.

 

FREE Full Text

1. 6.↵

 

1. Foreign Affairs, Trade and Development Canada

, 2015. The Catalytic Initiative to Save a Million Lives. Available at:http://www.international.gc.ca/development-developpement/priorities-priorites/health-sante/catalytic_initiative-initiative_catalytique.aspx?lang=eng. Accessed August 15, 2015.

1. 7.↵

 

1. Bryce J, 
2. Victora CG, 
3. Boerma T, 
4. Peters DH, 
5. Black RE

, 2011. Evaluating the scale-up for maternal and child survival: a common framework. Int Health 3:139–146.

 

Abstract/FREE Full Text

1. 8.

 

1. Institut Supérieur des Sciences de la Population and Institute for International Programs, Johns Hopkins Bloomberg School of Public Health

,2014. Enquête sur la mise en œuvre de la prise en charge communautaire et la qualité des soins offerts aux enfants malades par les agents de santé communautaire: rapport final. Ouagadougou, Burkina Faso: ISSP and JHSPH.

1. 9.

 

1. Shaw B, 
2. Miller NP, 
3. Tsui AO, 
4. Bryce J, 
5. Tafesse M, 
6. Surkan PJ

, 2015.Determinants of utilization of health extension workers in the context of scale-up of integrated community case management of childhood illnesses in Ethiopia. Am J Trop Med Hyg 93: 636–647.

 

Abstract/FREE Full Text

1. 10.

 

1. Callaghan-Koru JA, 
2. Hyder AA, 
3. George A, 
4. Nsona H, 
5. Mtimuni A, 
6. Zakeyo B,
7. Mayani J, 
8. Cardemil C, 
9. Bryce J

, 2013. Health systems supports for community case management of childhood illness: lessons from an assessment of early implementation in Malawi. BMC Health Serv Res 13: 55.

 

CrossRefMedlineGoogle Scholar

1. 11.↵

 

1. Gilroy KE, 
2. Callaghan-Koru JA, 
3. Cardemil CV, 
4. Nsona H, 
5. Amouzou A,
6. Mtimuni A, 
7. Daelmans B, 
8. Mgalula L, 
9. Bryce J

, CCM-Malawi Quality of Care Working Group, 2013. Quality of sick child care delivered by Health Surveillance Assistants in Malawi. Health Policy Plan 28: 573–585.

Abstract/FREE Full Text

1. 12.↵

 

1. United Nations Foundation

, 2015. UN Commission on Information and Accountability for Women´s and Children´s Health. Available at:http://www.everywomaneverychild.org/accountability/coia. Accessed August 15, 2015.

1. 13.↵

 

1. Miller NP, 
2. Tafesse M, 
3. Hazel E, 
4. Legesse H, 
5. Degefie T, 
6. Victora CG, 
7. Black RE,
8. Bryce J

, 2014. Integrated community case management of childhood illness in Ethiopia: implementation strength and quality of care. Am J Trop Med Hyg 91:10.

 

Google Scholar

• CiteULike

• Delicious

• Digg

• Facebook

• Google+

• LinkedIn
• Mendeley

• Reddit

• StumbleUpon

• Twitter

What´s this?

Articles:

• Agbessi Amouzou, 
• Mercy Kanyuka, 
• Elizabeth Hazel, 
• Rebecca Heidkamp,
• Andrew Marsh, 
• Tiope Mleme, 
• Spy Munthali, 
• Lois Park, 
• Benjamin Banda,
• Lawrence H. Moulton, 
• Robert E. Black, 
• Kenneth Hill, 
• Jamie Perin,
• Cesar G. Victora, 
• and Jennifer Bryce

Independent Evaluation of the integrated Community Case Management of Childhood Illness Strategy in Malawi Using a National Evaluation Platform DesignAm J Trop Med Hyg 2016 94:574-583; Published online January 19, 2016, doi:10.4269/ajtmh.15-0584

• Abstract

 

• Full Text

 

• Full Text (PDF)

 

• Supplementary File

OPEN ACCESS ARTICLE

Articles:

• Melinda Munos, 
• Georges Guiella, 
• Timothy Roberton, 
• Abdoulaye Maïga,
• Adama Tiendrebeogo, 
• Yvonne Tam, 
• Jennifer Bryce, 
• and Banza Baya

Independent Evaluation of the Rapid Scale-Up Program to Reduce Under-Five Mortality in Burkina FasoAm J Trop Med Hyg 2016 94:584-595; Published online January 19, 2016, doi:10.4269/ajtmh.15-0585

• Abstract

 

• Full Text

 

• Full Text (PDF)

 

• Supplementary File

OPEN ACCESS ARTICLE

Articles:

• Agbessi Amouzou, 
• Elizabeth Hazel, 
• Bryan Shaw, 
• Nathan P. Miller,
• Mengistu Tafesse, 
• Yared Mekonnen, 
• Lawrence H. Moulton, 
• Jennifer Bryce,
• and Robert E. Black

Effects of the integrated Community Case Management of Childhood Illness Strategy on Child Mortality in Ethiopia: A Cluster Randomized TrialAm J Trop Med Hyg 2016 94:596-604; Published online January 19, 2016, doi:10.4269/ajtmh.15-0586

• Abstract

 

• Full Text

 

• Full Text (PDF)

 

• Supplementary File

OPEN ACCESS ARTICLE

Articles:

• Bernadette Daelmans, 
• Awa Seck, 
• Humphreys Nsona, 
• Shelby Wilson,
• and Mark Young

Perspective: Integrated Community Case Management of Childhood Illness: What Have We Learned?Am J Trop Med Hyg 2016 94:571-573; doi:10.4269/ajtmh.94-3intro2

• Full Text

 

• Full Text (PDF)

OPEN ACCESS ARTICLE