Bulletin of the World Health Organization

Economic evaluation of neonatal care packages in a cluster-randomized controlled trial in Sylhet, Bangladesh

Amnesty E LeFevre a, Samuel D Shillcutt a, Hugh R Waters a, Sabbir Haider b, Shams El Arifeen b, Ishtiaq Mannan b, Habibur R Seraji b, Rasheduzzaman Shah a, Gary L Darmstadt c, Steve N Wall d, Emma K Williams a, Robert E Black a, Mathuram Santosham a, Abdullah H Baqui a & for the Projahnmo Study Group

a. Department of International Health, Johns Hopkins Bloomberg School of Public Health, Suite E-8139, 615 North Wolfe Street, Baltimore MD 21205, United States of America (USA).
b. Child Health Unit, International Centre for Diarrhoeal Disease Research, Dhaka, Bangladesh.
c. Family Health Division, Bill & Melinda Gates Foundation, Seattle, USA.
d. Save the Children’s Saving Newborn Lives Initiative, Washington, USA.

Correspondence to Amnesty LeFevre (e-mail: alefevre@jhsph.edu).

(Submitted: 29 December 2012 – Revised version received: 03 May 2013 – Accepted: 30 May 2013 – Published online: 12 July 2013.)

Bulletin of the World Health Organization 2013;91:736-745. doi: http://dx.doi.org/10.2471/BLT.12.117127

Introduction

Globally, an estimated 7.6 million children aged younger than 5 years – including 3.1 million infants less than 28 days old – died in 2010.1 Of the deaths that occur each year among children less than 5 years old, 99% occur in low- and middle-income countries, 28% occur in south-east Asia and 4% occur in Bangladesh alone.2 In Bangladesh, neonatal deaths comprise 74% of infant deaths and 60% of the deaths that occur in children less than 5 years of age.3 Although neonatal mortality has declined in Bangladesh over the last decade, an estimated 83 070 neonatal deaths still occur in the country each year.1 Cost-effective strategies to improve access to neonatal health services will have to be developed and implemented in Bangladesh if Millennium Development Goal 4 is to be attained.4

The Project for Advancing the Health of Newborns and Mothers was established in Bangladesh in 2002 by researchers, programme managers and policy-makers who wanted to reduce neonatal mortality in rural areas of the country.5 The Project tested the effectiveness of two strategies for the delivery of maternal and neonatal health services – known as the “home-care” and “community-care” models – in a cluster-randomized controlled trial in Sylhet division.5 The results of the trial indicated that implementation of the home-care strategy led to a 28% reduction in neonatal mortality during the final year of the intervention.5 In contrast, no significant reduction in neonatal mortality was observed in the community-care arm.

Although data on the cost-effectiveness of some community-based strategies for reducing neonatal mortality in low- and middle-income countries are emerging,6 the cost-effectiveness of either of the strategies explored in Bangladesh’s Sylhet division5 has not been investigated. The aim of the present study was to fill this gap.

Methods

Study site and population

The activities of the Project for Advancing the Health of Newborns and Mothers were implemented, among a population of about 500 000, in the Beanibazar, Zakiganj and Kanaighat subdistricts of Sylhet division.5 Sylhet was selected as a study area because it had been found to have a higher level of neonatal mortality – 63 neonatal deaths per 1000 live births – and a higher fertility rate – 4.2 births per woman – than any of the other five of Bangladesh’s divisions.4 When the Project was launched, those living in Sylhet had relatively poor access to health services and showed low levels of utilization of skilled birth attendants.7 At the same time, however, none of the nongovernmental organizations (NGOs) present in the division were considered capable of ensuring the Project’s sustainability and promoting the large-scale delivery of any test intervention.5 The study was registered as International Standard Randomised Controlled Trial number NCT00198705.

Interventions

The Project had three study arms: home care, community care and a control arm in which the pre-existing level of care was left largely unsupplemented (Table 1).5 All three arms included attempts to strengthen the health systems in subdistrict health complexes and health and family welfare centres. The three subdistrict health complexes in the Project area provided a basic package of inpatient, outpatient and emergency services for mothers with neonates but no emergency obstetric care. Women who required emergency obstetric care were usually referred to Sylhet Medical College, which lies about 8.5 km from the study area.8 The 24 health and family welfare centres in the Project area focused on the provision of outpatient services, including outpatient maternal, reproductive and child health services. During the study, the supply of antibiotics to all levels of the local health system was strengthened but remained sporadic. There was considerable between-facility variation in the working hours of medical personnel. The activities associated with the Project sought to enhance the general provision of maternal and neonatal health services – through the training of staff involved in service delivery, the provision of drugs needed for neonatal care, the strengthening of information systems for tracking service utilization, and the promotion of maternal and neonatal referral.

In the home-care study arm, women aged 20 to 35 years who had a secondary-school certificate – typically acquired after 10 years of schooling – were recruited and trained as community health workers (CHWs). Once trained, these women conducted pregnancy surveillance and, for each woman found pregnant, made two antenatal home visits to promote birth and neonatal care preparedness and three postnatal home visits to reinforce neonatal care and assess the health of each infant when the infant was aged 1, 3 and 7 days. Neonatal health was assessed using a clinical algorithm.5 The CHWs were trained and authorized to provide an initial dose of antibiotic treatment to any infant who was suspected of having a severe neonatal illness and to promote the referral of such an infant to hospital. If a sick infant was not taken to a hospital, the CHWs were authorized to continue antibiotic treatment of the infant for 10 days – under the supervision of a Project medical officer – if the infant’s parents consented to such home treatment.8

Community inputs for the home-care and community-care study arms included the orientation of traditional birth attendants and the creation of community mobilizers. In the orientation, which took 2 to 3 days, the birth attendants were taught about the proposed interventions, safe motherhood and essential neonatal care. Community mobilizers were adult volunteers who lived in the study communities and were trained to promote birth and neonatal care preparedness among community members. The mobilizers also supported the work of CHWs. Although there was no provision of home-based health services in the community-care arm, women in this arm were encouraged to seek antenatal, intrapartum and postnatal care at their local health facilities.

Costing

Economic costs were collected from a societal perspective and included programme, provider and household costs.9 Programme costs included those incurred by the implementing agencies. The provider costs were incurred at the 27 government health facilities in Sylhet. The household or “user” costs that we estimated were those incurred in seeking services for maternal and/or neonatal health care. All costs were inflated to 2010 values – using the relevant consumer price indices from the International Monetary Fund10 – and then converted from Bangladeshi taka to United States dollars (US$) using the relevant conversion factor from OANDA.11

Programme costs

The Project-related costs incurred between 2003 and 2005 by the two agencies that implemented the Project for Advancing the Health of Newborns and Mothers — the International Centre for Diarrhoeal Disease Research, Bangladesh, and Shimantik – a local NGO – were evaluated. The research costs associated with household surveys and research-specific technical assistance provided by Project investigators from the United States of America were disaggregated from all capital and recurrent programme costs. Costs were categorized as research or programme costs and by study arm and subdistrict. In 2005 and early 2006, a team of medical anthropologists conducted time allocation interviews with Project staff using an in-depth qualitative format. In these interviews, the emphasis was on ascertaining the percentages of time allocated across activities and study arms over the course of the Project. Focus group discussions were subsequently conducted to validate the time estimates made by interviewees. The results were then cross-checked with senior Project investigators before being used to determine the programme costs in each study arm. The replacement costs used for all capital items were based on local market values. The costs of capital items were annualized according to international12 or local estimates of each item’s “life expectancy”, with discounting at an annual rate of 3%.

Provider costs

Primary data on provider costs were collected from all three subdistrict health complexes and from a random selection of 12 of the 24 health and family welfare centres in the Project area. All of the health and family welfare centres in the Project area were stratified according to study arm, subdistrict and whether Project medical officers considered the centre to have optimal, average or suboptimal functionality. The level of functionality of a centre was assessed in terms of the officers’ perception of provider capacity and engagement at the centre, the centre’s operating hours and the level of the centre’s maintenance. Within each study arm, the provider costs were assumed to be the same in health and family welfare centres with the same level of functionality. This assumption allowed the total cost of outpatient neonatal care at all of the health and family welfare centres within each study arm to be estimated.

Data on the utilization of neonatal care services, the associated consumption of drug and medical supplies and the time allocated to neonatal care by the staff in the local health facilities were collected from individual patient records and pharmacy logs and in time allocation interviews with the health-care providers. Limitations in the maintenance of health facility logs and patient records, coupled with the high frequency of the procurement of drugs and medical supplies from the private sector by patients, led to the supplementation of the facility-based collection of data with household survey data. All of the data on care-seeking for routine and emergency neonatal care in 2005 were ultimately obtained via a household survey.5 Some data on the consumption of drugs and medical supplies were obtained through a users’ cost survey or extracted from the Project’s financial records. In an attempt to avoid “double counting”, provider costs included those of paying for staff while they were providing inpatient, outpatient or emergency care for neonates but excluded those of paying for staff while they were providing maternal care. Whenever possible, data were cross-checked by reviewing patient records, staff attendance sheets for 2005 and information on the utilization of services for neonatal care collected in household surveys.

User costs

The estimation of user costs began in mid-2004. During the annual household “adequacy” surveys that were conducted as part of the study of intervention effectiveness,5 households in the Project area were asked to estimate their out-of-pocket expenditures for maternal and neonatal care. The sample size required for these surveys – which was based on the number of live births occurring in Sylhet – was estimated to be 250 live births per study arm. After allowing for loss-to-follow-up and problems in the collection of data – we planned to conduct 350 interviews per arm. Data on user costs were ultimately collected from 316 to 327 households in each study arm. Data from the household surveys on the utilization of neonatal care services were used to scale up the mean out-of-pocket payment for neonatal care in each study arm, to give an estimate of the annual payment for the year 2005. The estimated annual costs were finally converted into United States dollar equivalents for the year 2010.

All of the data on effectiveness and user costs were analysed in Stata version 10.0 (StataCorp. LP, College Station, USA) or Excel (Microsoft, Redmond, USA).

Effects

Primary outcome measures included the neonatal deaths and disability-adjusted life years (DALYs) that were averted. The numbers of neonatal deaths averted were estimated using a “difference in differences” approach in which the results of a baseline survey were compared with those of the final survey conducted in the study period. Although data on neonatal disabilities were not collected as part of the impact evaluation,5 subsequent modelling was used to yield estimates of the years of life lost due to disability.13 DALYs were generally calculated using a 3% annual discount rate and assuming a mean life expectancy at birth of 62.6 years.9 Life tables based on data from the World Health Organization’s South-East Asia Region14 or the West level-26 model15 were also used in a sensitivity analysis. In this analysis – as recommended elsewhere16 – no age-weighting was used in the reference case.

Analyses

Cost-effectiveness estimates are presented from a programme perspective as well as from a societal perspective inclusive of programme, provider and user costs. The programme perspective was chosen as the reference case to reflect the need of stakeholders to know the incremental costs that would probably result from the addition of a new programme. The societal perspective was chosen to facilitate discourse on the full opportunity costs. Estimates for each year of programme implementation were calculated deterministically. To test the levels of uncertainty in our estimates of patient-level costs, a Monte-Carlo simulation was used in a probabilistic sensitivity analysis.9 This approach predicts the results that might arise from our trial if it were performed a large number of times. The mean of each cost component – programme, provider and user – was summed in calculating each iterated incremental cost-effectiveness ratio. In total, 10 000 iterations were generated by using a Visual Basic macro in Excel. Following a recommendation by the Commission for Macroeconomics and Health,17 the incremental cost-effectiveness ratios were then compared with the per-capita value for the gross national income of Bangladesh in 2010. Cost-effectiveness acceptability curves were generated in further sensitivity analyses to test the robustness of the results. Age-weighting, the assumptions that we made about annual discount rates when estimating the DALYs averted by each intervention, and the effects of a hypothetical 25% increase or decrease in the costs of each intervention were tested. Scenario analyses – in which the Project’s field staff were assumed to work full time on the Project’s activities – and a “difference in differences” analysis of the potential effects of the implementation of each package of interventions were also performed.

Results

In both the home-care and community-care study arms, estimates of the annualized programme costs were higher in 2005 than in any previous year and recurrent costs accounted for more than 90% of the total estimated costs for 2005 (Table 2). Personnel costs accounted for the largest proportion of the recurrent costs, followed by transportation. In the home-care arm, an estimated 60% of the time costs of CHWs were attributed to programme costs; the other time costs of the CHWs in this study arm were attributed to research activities that were not necessary for implementation of the home-care package. All of the time costs of the community mobilizers in this study arm were associated with programmatic activities, including community meetings and women’s groups. The health staff in the government-run health facilities included in this study arm spent less than 2% of their working time on the provision of neonatal care. The mean out-of-pocket payments per neonatal patient were higher in this study arm (U$ 3.82) than in the community-care arm (US$ 2.35) or the control arm (US$ 3.42). Although the cost of medicines represented the largest proportion of such payments in all three arms, medicines cost about US$ 1.00 more per patient in the home-care arm than in the other two arms.

The annualized total costs for 2005 were two-thirds higher in the home-care arm than in the community-care arm (Table 2). Programme costs represented 89% and 78% of the total costs in the home-care and community-care arms, respectively. The total costs in the control arm, which only comprised government costs (16%) and user costs (84%), were only a tenth of the total costs in the home-care arm.

Summary findings of the present study are presented in Table 3. Full details of the effectiveness of each package of interventions for neonatal care have already been published.5 Although the neonatal mortality observed in the control arm was significantly higher than that seen in the home-care arm, it was similar to that seen in the community-care arm.

Cost-effectiveness

Incremental cost-effectiveness ratios are shown in Table 4. Relative to the control arm, the programme costs of the home-care package were US$ 2939 (95% confidence interval, CI: 1833–7616) per death averted or US$ 103.49 (95% CI: 64.72–265.93) per DALY averted. These costs appear reasonable when compared with the gross national income of Bangladesh in 2010, which was US$ 780 per capita.18

Sensitivity analysis

Relative to the control arm, for every 1000 neonates receiving the home-care package, 11.9 deaths were averted – at an additional cost of US$ 39 418 (Table 3). The cost-effectiveness acceptability curves that we plotted indicated that, if a DALY is valued at more than US$ 214, there is a 95% probability that the home-care package will be cost-effective. There is also a high probability that such a care package would be cost-effective – in any setting with similar costs – if it leads to a reduction in neonatal mortality of at least 9%.

The working schedule of the Project’s field staff during the effectiveness trial was intense: many of the staff worked 9 h per day for 28 days per month. Such a heavy workload may be difficult to maintain in the long term, particularly if the government’s health staff take over the Project’s activities. Accordingly, a one-way sensitivity analysis was conducted to explore the effect of raising the time staff spend on the Project, from 60% of their work time to 100%. This change had little effect on the incremental cost-effectiveness ratio for home care, which became US$ 2994 per neonatal death averted – a finding previously reported.5

Discussion

Our results indicate that – when compared with the community-care package or pre-existing levels of maternal and neonatal care – implementation of the home-care strategy was highly cost-effective from both a programme and societal perspective. The programme cost of the home-care package per DALY averted – US$ 103.49 – fell well below Bangladesh’s per-capita gross national income and compared favourably with the corresponding costs of interventions evaluated elsewhere in South Asia, as part of the Disease Control Priorities Project.19 The cost-effectiveness of the home-care package is largely driven by the high number of neonatal deaths averted. Much of this reduction in neonatal mortality presumably arose from the prompt and appropriate identification and management of neonatal infection in the home.20 Such home-based identification and management of neonatal illness did not form part of the community-care package of neonatal care that we investigated, which was not associated with a significant reduction in neonatal mortality.5 Recruitment and training costs in the home-care arm were higher than expected because of the unforeseen loss of CHWs to emerging opportunities for employment by the Bangladeshi government. While it is probable that annual personnel costs may decline after the home-care strategy has been implemented for a year, continued adequate and regular supervision and encouragement will be needed to reduce attrition in the workforce. Finally, under non-trial conditions, the work of CHWs may change in content and intensity. Such changes may have unknown implications for the cost, coverage and effectiveness of the home-care package.

Limitations

Our quantification of provider costs relied on reported estimates of the time that community health workers and other essential personnel spent on implementing programme activities. To minimize recall and reporting biases and control for seasonal variations, these estimates were verified, when possible, by the examination of utilization logs in the relevant health facilities. Programme costs per patient may decrease – because of economies of scale – when the activities that we investigated are implemented on a larger scale. Out-of-pocket expenditures were estimated by asking key household decision-makers – often men or the mothers-in-law of the women of childbearing age – to estimate the household costs of neonatal care. Patient-level uncertainty was assessed for users’ costs and estimates of effectiveness. The effects of errors in the estimation of total costs were tested in models in which total costs were assumed to differ from the estimated values by an arbitrary amount: 25%. Disability estimates had to be modelled, as collection of primary data on disability was not planned.

Our results may underestimate the true value of cost-effectiveness because maternal health benefits were excluded from our analyses. However, the programme-related costs of providing maternal health services at the community level were included. Data collected in other investigations indicate that 70% of all of the costs of an intervention package similar to the home-care package that we evaluated would result in benefits to maternal health.2,2123 Additional benefits not usually included in economic evaluations – such as information sharing, gains in employment and economic productivity, increased community autonomy and empowerment, and the provision of a culturally acceptable process of care – may also have improved community health and development24 and further increased the cost-effectiveness of one or both of the packages that we investigated.

Comparison with other cost-effectiveness studies

In low-resource settings, only three neonatal care packages implemented at the community level have previously reported findings on cost-effectiveness: one in India,25 one in Nepal6 and one in Zambia.26

In the Indian study, a programme for village health workers in rural Gadchiroli was found to cost US$ 7 for each of the DALYs averted as a result of a 62% reduction in the neonatal mortality rate.25 Although the interventions implemented in the Indian trial were similar to those implemented in the home-care arm in Bangladesh, the Indian trial involved a fourfold higher density of CHWs, more extensive training of health workers, 8–12 postnatal visits per neonate, and neonatal resuscitation.25,27 The Indian study was preceded by a clinical trial – addressing pneumonia – in the same study area and was relatively well established when its cost-effectiveness was evaluated.25,27 In the estimation of the cost-effectiveness of the Indian care package, administration and training costs were ignored. As the data inputs and methodology used for the economic evaluation in the Indian study have not been reported in detail, it is difficult to make a valid comparison between the results of the Indian study and those of our trial in Bangladesh.

In Nepal, the estimated cost of an intervention aimed at improving birth outcomes was US$ 5801 – or US$ 6912 with health-systems strengthening – per neonatal death averted.6 The evaluation framework used to generate these cost estimates was similar to the one that we used in Bangladesh. However, the trials in Bangladesh and Nepal differed in terms of delivery strategy, package content, population density and local geography. In Nepal, the care package that was investigated did not include a cadre of CHWs, and no attempt was made to estimate societal costs such as household expenditures on care-seeking. The Nepali study area was more mountainous – and, in consequence, less densely populated – than the Bangladeshi study area. The relatively low population density in Nepal meant that certain fixed costs were distributed over fewer beneficiaries in Nepal than in Bangladesh.

In Zambia, the Lufwanyama Neonatal Survival Study has demonstrated that, by training and equipping traditional birth attendants to perform interventions targeting birth asphyxia, hypothermia and neonatal sepsis, neonatal mortality can be reduced by 45% at an economic cost of U$ 176 per DALY averted.26 Although there is some overlap in the interventions investigated in Zambia and the home-care arm in Bangladesh – for example, both trials promoted the home-based management of neonatal infection – only the package investigated in Zambia sought to reduce deaths due to birth asphyxia and hypothermia, through the implementation of a neonatal-resuscitation protocol.26 The home-care arm in Bangladesh included wider community engagement and mobilization than the Zambian trial. If offered as part of a package of home-based neonatal care in Bangladesh, neonatal resuscitation may increase the overall cost-effectiveness of the package, particularly if home visits are made close to the time of delivery.

Generalizability

The home-care package of interventions investigated in Sylhet should be considered for replication and scale-up throughout Bangladesh and in similar settings where neonatal mortality is high and the utilization of facility-based delivery and postnatal-care services is low. The effectiveness of the home-care package might be improved through the addition of community-based strategies to reduce neonatal mortality from birth asphyxia, and its apparent cost-effectiveness might be improved by the quantification of maternal benefits. Similarly, incentives to improve the retention of CHWs may yield additional cost savings and improve the overall estimates of cost-effectiveness. In other programmes, the attrition of CHWs has been reduced – but not eliminated – by the use of incentives.28 Where other economic opportunities exist and where CHWs face such intense poverty that they are precluded from serving effectively, competitive rates of remuneration – in cash or kind – may well be necessary.2931 The mobilization of CHWs as volunteers has rarely been effective in other settings.32 In Sylhet, it is unlikely to be effective in the long-term implementation of the home-care package, given the duration and intensity of the work set for the CHWs and the presence of employment alternatives. The effectiveness of the home-care package for neonatal care will depend, in part, on the local health infrastructure and the habits of the target population – such as how the women seek health care during pregnancy and delivery and postpartum. Affordability is another concern. In Sylhet, implementation of the home-care package of interventions cost U$ 7.92 per person–year at a time when Bangladesh only spent US$ 26.50 per person–year on health care. These figures emphasize the importance of external funding.18

In the years since the encouraging results of the effectiveness trials by the Project for Advancing the Health of Newborns and Mothers were first published,5 various packages of community-based interventions for improving maternal and neonatal health have been investigated throughout Bangladesh, as well as elsewhere in South Asia and in Africa. In Sylhet, the usefulness of supplementing the basic home-care package with other interventions – such as the cleansing of the umbilical stump with chlorhexidine33 and the provision of postpartum family planning services – has been explored. The data collected in Sylhet are helping to guide the development of new health programmes in Bangladesh, such as the MaMoni Project, which aims to integrate safe motherhood, neonatal care and family planning services in parts of the Sylhet and Habiganj divisons.34 Elsewhere in Bangladesh, alternative models for neonatal care have been tested in Mirzapur. However, while improvements in care-seeking and caregiver knowledge were observed, these models had no significant effect on neonatal mortality.35

Outside Bangladesh, efforts continue to test community-based programmes for the improvement of maternal and neonatal health – through varying delivery strategies and in a wide variety of contexts.36 In Sindh, Pakistan, a community-based intervention package – which was principally delivered by female health workers, traditional birth attendants and community health committees – reduced neonatal mortality by 12%.37 In Haryana, India, the Integrated Management of Neonatal and Child Illness programme significantly reduced mortality among neonates born at home.38 In Ghana, recent findings from the Newhints home-visits intervention also indicate significant reductions in neonatal mortality.39 In a meta-analysis of home-visit strategies in sub-Saharan Africa and South Asia – which included data from Sylhet and two other South Asian trials – a 12% (95% CI: 5–18) reduction in neonatal mortality was reported as the summary estimate of the impact of the implementation of such strategies.39

Conclusion

The home-care package of interventions trialled in Sylhet is highly cost-effective. This result is robust to modelling assumptions and sensitivity analyses. Policy-makers, donors and stakeholders should consider the feasibility of implementing similar strategies, on a large scale, in other settings. Community-based neonatal health interventions have the potential to save lives and to take some pressure off health systems that are already strained. However, the affordability of such interventions in resource-poor areas remains in doubt. Strategies to ensure sustainable financing of such interventions for vulnerable populations are therefore needed.


Acknowledgements

We thank the many interviewees in Sylhet who generously gave their time, and the field and data-management staff who worked tirelessly on programme activities. We thank the members of the Projahnmo Technical Review Committee, the Bangladeshi Ministry of Health and Family Welfare and the Shimantik Executive Committee for their valuable help and advice.

Funding:

This research was funded by the Saving Newborn Lives Initiative of Save the Children – US (via a grant from the Bill & Melinda Gates Foundation) and the United States Agency for International Development (via cooperative agreements with the Johns Hopkins Bloomberg School of Public Health and the International Centre for Diarrhoeal Disease Research, Bangladesh, and the Saving Newborn Lives Initiative).

Competing interest:

None declared.

References

Share