Bulletin of the World Health Organization

Effect of Chiranjeevi Yojana on institutional deliveries and neonatal and maternal outcomes in Gujarat, India: a difference-in-differences analysis

Manoj Mohanan a, Sebastian Bauhoff b, Gerard La Forgia c, Kimberly Singer Babiarz d, Kultar Singh e & Grant Miller f

a. Sanford School of Public Policy, Duke University, 302 Towerview Drive, Durham NC 27708, United States of America (USA).
b. RAND Corporation, Arlington, USA.
c. The World Bank, Washington, USA.
d. Stanford Medical School, Stanford, California, USA.
e. Sambodhi Research and Communications Pvt Ltd, New Delhi, India.
f. School of Medicine and Freeman Spogli Institute for International Studies, Stanford University, Stanford, California, USA.

Correspondence to Manoj Mohanan (e-mail: manoj.mohanan@duke.edu).

(Submitted: 15 May 2013 – Revised version received: 14 October 2013 – Accepted: 30 October 2013 – Published online: 09 December 2013.)

Bulletin of the World Health Organization 2014;92:187-194. doi: http://dx.doi.org/10.2471/BLT.13.124644

Introduction

Each year, over 500 000 women in the world die from birth-related complications1 and more than 5 million neonates die before they reach 12 months of age.2 All but 2% of the neonatal deaths occur in developing countries.2 In the last decade, global efforts to reduce maternal and neonatal mortality have grown rapidly. Between 2003 and 2008, global spending on maternal, neonatal and child health more than doubled – although it remained relatively constant as a share of official development assistance.3,4 This allowed various new initiatives, including those focusing on skilled birth attendance and emergency obstetric care, to be supported.3,4

As no single intervention can address the range of causes of maternal death, facility-based intrapartum care – “institutional delivery” – is generally recommended as the best option for improving maternal health.5 Facilities that provide maternity care are likely to have the trained staff, infrastructure and standardized delivery protocols needed to manage most neonatal and obstetric complications.

In general, public hospitals in India lack the capacity and reach to serve many rural areas, which means that many poor women have no access to key health-care services.6,7 Recognizing this limitation, in January 2006 the state government of Gujarat launched the Chiranjeevi Yojana programme, a public–private partnership designed to increase institutional delivery rates.8 It was subsequently estimated that this programme had led to a 90% reduction in maternal deaths and a 60% reduction in neonatal deaths among beneficiaries in Gujarat.812 The programme received the Asian Innovations Award in 2006,13 had covered almost 800 000 deliveries by March 2012, and is generally perceived as a successful model that should be followed in other Indian states.14

However, perceptions of the programme’s success are based on the results of studies – simple cross-sectional investigations or before-and-after comparisons – that had severe limitations. These studies did not address the role of the self-selection of institutional delivery by pregnant women and took no account of the “background” increases in institutional deliveries that probably occurred over each study period.10

The Chiranjeevi Yojana programme covers the costs of deliveries – at designated private-sector hospitals – for women from “below-poverty-line” (BPL) households. BPL status – which is either determined by multidimensional means testing or designated by the relevant village authority – confers a variety of other benefits, including public subsidies for food grains, sugar, oil and fuel. The programme pays the designated private-sector hospitals 1600 Indian rupees – approximately 37 United States dollars (US$) – per delivery. In exchange, the programme expects the hospitals both to offer vaginal deliveries or caesarean sections to poor women free of charge and to reimburse at least some of the women’s travel costs. The hospitals can offer additional hospital services to beneficiaries for a separate fee. By 2010, the programme covered more than 800 private-sector hospitals and had helped finance more than 400 000 deliveries.15

The aim of the present study was to estimate the relationship between the Chiranjeevi Yojana programme and the probability of institutional delivery, the use of maternal and neonatal services provided by trained health workers, birth-related maternal complications and household spending for delivery. We used population-based samples and adjusted for time-invariant differences across the districts of Gujarat, changes in outcomes over time that were common across the state, and a range of household-level characteristics. By matching information about programme placement and timing to population-level data – rather than data from participating hospitals – we were also able to minimize the influence of self-selection into institutional delivery among pregnant women.

Methods

The Chiranjeevi Yojana programme was launched in five northern districts of Gujarat in January 2006 and then expanded to the rest of the state between December 2006 and January 2007. We use a multivariate difference-in-differences regression framework to identify changes in primary outcomes associated with the spread of the programme across the state. The 21 “late-implementing” districts served as a control for the five “early-implementing” districts of Banaskantha, Dahod, Kachchh, Panch Mahal and Sabarkantha in 2006 and vice versa in 2007.

Data sources and outcome measures

We relied on two sources of data for our study: our own household survey in 2010 and analogous data from the third wave of the District Level Household and Facility Survey (DLHS-3).

In our survey – conducted in August to September 2010 – we collected retrospective information from a state-wide sample of women who had had deliveries since January 2005. We used stratified three-stage cluster sampling (Appendix A, available at: http://www.cohesiveindia.org/IMATCHINE/CY-BWHO2013Appendix.pdf). Within each study village we conducted a brief census of all households. We then randomly selected 10 households that had reported at least one delivery since January 2005. We selected households with BPL index scores between 16 and 25 in the year 2002 based on original plans to use a “fuzzy” regression discontinuity design.16,17 Unfortunately, widespread inconsistencies in classification of the BPL status of households prevented such analyses. Of the 6002 sampled households, 5663 (94.4%) agreed to provide data and 5597 (93.3%) provided full data and were included in the final analysis.

Through household surveys modelled after the well-known Demographic and Health Surveys, we recorded maternal and household demographic and socioeconomic characteristics as well as detailed retrospective information about antenatal and postnatal care and deliveries for all births since 2005. Our primary outcomes were: the place of delivery – public facility, private facility or home; who assisted with the delivery – a trained health professional or a relative; the type of delivery – vaginal or caesarean; whether the mother had received antenatal care, postnatal care or both and, if so, whether such care had been provided by a trained health professional; whether the mother suffered birth-related maternal complications such as premature labour, excessive bleeding or loss of consciousness; whether the neonate had been admitted to a neonatal intensive-care unit; and how much the household had spent on the delivery – both the hospital fees and total costs including transportation and other expenses.

We conducted parallel analyses using publicly available data from the third wave of the District Level Household and Facility Survey (DLHS-3). Such surveys have been routinely commissioned – by the Indian Ministry of Health and Family Welfare – to monitor reproductive and child health throughout India. In Gujarat, DLHS-3 was conducted in 2007 and 2008 across each of the state’s 26 districts. We restricted our DLHS-3 sample to women whose last delivery – at the time of the survey – had been in or after 2005, yielding a final sample of 6484 households. Because the sample was designed to be representative at the district level, no attempt was made to identify households that had below-poverty-line status. However, the demographic characteristics of the DLHS-3 sample were reasonably comparable to those of our own survey (Table 1), and both measured similar delivery-related outcomes.

Data analysis

A multivariate difference-in-differences ordinary least-squares regression analysis was used to determine if changes in our primary outcomes were associated with the staggered introduction of the Chiranjeevi Yojana programme across Gujarat’s districts. We use the timing of births – as reported by mothers in the two surveys – together with mother’s district of residence to determine if the Chiranjeevi Yojana programme had been implemented when a delivery occurred.

Specifically, using birth-level observations, we regress our primary outcomes on an indicator variable denoting whether or not the Chiranjeevi Yojana programme was active. In all regressions we control for district and year fixed effects to account for unobserved interdistrict differences that did not vary over time and changes over time that were common to all districts;19 as well as maternal characteristics (mother’s age at marriage, age at delivery and level of education); and household characteristics (caste, religion, and wealth – BPL score, monthly household income in our data and a composite wealth index in the DLHS-3 data). We calculated robust standard errors that are clustered by block (the primary sampling unit) to relax the assumption of independent and identically distributed errors within districts.

Given our available data using a baseline of 0.56 (proportion of institutional deliveries), the minimum detectable effect size for test of proportions is 5 percentage points or more in the probability of institutional delivery (a change from 0.56 to 0.61). The summary statistics for all outcomes are shown in Table 2.

Results

Programme impact

Fig. 1 shows the unadjusted mean institutional delivery rates (and confidence intervals, CIs) in the early- and late-implementation districts – separately using data from our 2010 survey and DLHS-3 from 2007–2008. Both panels show background increases in institutional delivery rates over time across Gujarat that are unrelated to the Chiranjeevi Yojana programme. Both panels also show modest changes in institutional delivery rates over time, but there are otherwise no relative changes in either early- or late-implementation districts that coincide with the introduction of the Chiranjeevi Yojana programme. The divergence between early- and late-implementation districts in 2008 shown using DLHS-3 data (left-hand panel of Fig. 1) is not present in 2008 (right-hand panel of Fig. 1) and disappears when the DLHS-3 data are weighted to compensate for between-survey differences in key BPL scoring components, such as household assets, sanitation and literacy. (The results with reweighted DLHS-3 data are available from the corresponding author on request).

Fig. 1. Meana institutional delivery rates in Gujarat, India, 2005–2010
Fig. 1. Mean<sup>a</sup> institutional delivery rates in Gujarat, India, 2005–2010
a Unadjusted mean.
Note: the data were collected specifically for the present study (left-hand panel) or in the District Level Household and Facility Survey for 2007–2008 (right-hand panel). They are plotted separately for the “early-implementing” districts – i.e. Banaskantha, Dahod, Kachchh, Panch Mahal and Sabarkantha, which implemented the Chiranjeevi Yojana programme in January 2006 – and the “late-implementing” districts – i.e. the other 21 districts of Gujarat, which implemented the same programme in December 2006 or January 2007. Error bars indicate 95% confidence intervals, adjusted for survey design.

The relationship between the Chiranjeevi Yojana programme and each of the primary outcomes we investigated is reported in Table 2. In Appendix A we also report odds ratios obtained using logistic regressions rather than linear probability models and note that the two approaches yield comparable inferences throughout. The data collected through our study indicated that implementation of the programme was not associated with a statistically significant change in the probability of institutional delivery (2.42 percentage points; 95% CI: −5.90 to 10.74). The CIs also imply that any association with institutional delivery too small to be detected in our samples could be no larger than an increase of 10.7 percentage points. The analysis using data from the DLHS-3 also confirms our findings that the programme’s implementation was not associated with a statistically significant change in institutional deliveries (−3.08 percentage points; 95% CI: –9.12 to 2.96). The DLHS-3 estimates are also more precise, which suggests that any programme impact too small to detect in our analysis could have been no larger than an increase of 2.9 percentage points. We find similar null estimates both for probability of delivery in private maternity hospitals and for birth attendance by physicians or nurses.

The programme was also not associated with changes in the incidence of birth-related maternal complications, the use of antenatal and postnatal services or the use of neonatal intensive care. Our survey data suggests that 54% of the mothers in our sample suffered complications, including premature delivery, prolonged and obstructed labour, excessive bleeding, breech presentation, convulsions, hypertension, fever, incontinence or other birth-related problems after the programme had been implemented – and that the probability of these complications did not significantly change under the Chiranjeevi Yojana programme.

The reimbursement offered to designated health facilities by the Chiranjeevi Yojana programme is intended to cover the costs of delivery for poor women who choose to give birth in a health facility. Even if the programme has not increased institutional delivery rates, we would expect to see lower mean household expenditures on deliveries, given that the programme had paid providers over US$ 32 million as of 2012. However, analysis of our survey data indicated that implementation of the programme had no significant relationship either with the probability that households reported any delivery-related spending (2.69 percentage points; 95% CI: −1.10 to 6.48) or with mean hospital spending for delivery conditional on any spending (18.22% change; 95% CI: –9.91 to 46.34).

In general, the DLHS-3 data also indicated that the programme was not associated with significant changes in household spending for institutional deliveries. The single exception is a 21.2% reduction in mean spending (95% CI: 4.71 to 37.69) among households reporting any expenditure, but this loses statistical significance after adjustment for the multiple comparisons (data not shown). Even among the households of women who delivered in private hospitals, spending on the deliveries appeared not to have changed significantly under the programme.

Discussion

Our findings indicate that the Chiranjeevi Yojana programme was not associated with changes in the probability of institutional delivery (including delivery at private institutions), maternal morbidity or delivery-related household expenditure. These findings differ from those reported by previous evaluations suggesting substantial benefits of the Chiranjeevi Yojana programme, including a 27% increase in institutional deliveries, a 90% reduction in maternal deaths and a 60% reduction in neonatal deaths.8,9,12 These earlier studies did not address self-selection of women into institutional delivery, reporting inaccuracies by hospitals, or any increases in institutional deliveries over time that were unrelated to the programme. The programme was rolled out in a period when the economy of Gujarat was growing by over 10% per year, for example.

Our study also has important limitations. One is that because it was not a randomized, controlled evaluation, we cannot rule out the possibility of confounding. However, we note that our results are robust to the inclusion of a wide variety of control variables and that the staggered introduction of the Chiranjeevi Yojana programme does not appear correlated with pre-existing trend differences in institutional delivery rates (see analysis of DLHS-2 data in Appendix A). Analyses using two independent sources of data (our survey and DLHS-3) also yielded very similar results. Another limitation of our study is that it relied on respondents’ recall of primary outcomes for a period of up to 5 years. Poor recall quality in the household surveys is a potential concern, although we followed the methods of the well regarded Demographic and Health Surveys’ “fertility roster” to collect these retrospective data.20 The degree of recall error is also unlikely to be related to the implementation of the Chiranjeevi Yojana programme.

There are several possible explanations for observing no increase in the probability of institutional delivery associated with the Chiranjeevi Yojana programme. One is that the quality of services provided by private maternity hospitals is poor or, at least, is perceived to be poor by the local population. As a result, demand for institutional delivery may be low even if such delivery is provided free of charge. Another is that – despite the support of the programme – institutional deliveries in Gujarat remain associated with large transportation costs, informal payments or other expenses that make programme benefits small relative to the full cost of institutional delivery.21

Our finding of little or no association between the Chiranjeevi Yojana programme and the out-of-pocket costs of deliveries is more puzzling. Even if the programme failed to make institutional delivery more attractive for any women, it should have reduced the household expenses for the many poor women who still chose institutional delivery. Although we cannot fully explain this finding, we note media reports of poor women still being asked to pay fees for deliveries in health facilities that were participating in the programme.22,23 It seems possible that some providers are providing extra, chargeable services – or simply increasing side charges. If charges are being made for extra services, those services do not appear to have any discernible health benefits. Further research on this issue is needed.


Acknowledgements

We thank the Department of Health and Family Welfare, Government of Gujarat, for its support. We are grateful to Manveen Kohli, Pragya Pranjali, Razdan Rahman, Harshad Vaidya and the Sambodhi team for their support.

Funding:

Research discussed in this publication has been financed with funds provided by the UK Department For International Financing (DFID) to the International Initiative for Impact Evaluation, Inc. (3ie) and managed through the Global Development Network (GDN), as well as support from the World Bank. The findings, interpretations, and conclusions expressed here are those of the authors and do not necessarily reflect the views of any of the funders, the Board of Executive Directors of the World Bank or the governments they represent.

Competing interests:

None declared.

References

Share