Bulletin of the World Health Organization

Adaptation and implementation of local maternity dashboards in a Zimbabwean hospital to drive clinical improvement

J Crofts a, J Moyo b, W Ndebele b, S Mhlanga b, T Draycott a & T Sibanda c

a. School of Clinical Sciences, Southmead Hospital, University of Bristol, Bristol, BS10 5NB, England.
b. Mpilo Central Hospital, Bulawayo, Zimbabwe.
c. Whanganui Hospital, Whanganui District Health Board, Whanganui, New Zealand.

Correspondence to J Crofts (e-mail: jo.crofts@bristol.ac.uk).

(Submitted: 23 May 2013 – Revised version received: 17 October 2013 – Accepted: 20 October 2013 – Published online: 27 November 2013.)

Bulletin of the World Health Organization 2014;92:146-152. doi: http://dx.doi.org/10.2471/BLT.13.124347

Introduction

Global efforts to improve maternal and perinatal health continue. However, quality improvement demands quality measurement: we can only improve things we can measure. Moreover, if we wish to make rapid improvements in quality, we must make measurement easier, more timely and more understandable.1 The relevant information must be available to those who need it.2

In many countries, national maternal and perinatal mortality data are already collected locally in a variety of formats and are submitted to national and international bodies.3 This process often produces summary data that cover a long period of time. However, lengthy time lags are inevitable and make it difficult for both front-line staff and managers to identify and deal with local problems quickly enough to prevent further harm. This, we believe, is one of the crucial failings of otherwise commendable attempts to improve maternal and perinatal outcomes around the world. The Commission on Information and Accountability for Women's and Children's Health of the World Health Organization (WHO) has recognized that, in resource-poor settings, health outcome data are often out of date and of questionable quality and are “therefore, not timely enough for practical use by health planners and administrators”.4

In high-income countries, health-care quality is increasingly being monitored in local settings using performance score cards, such as “clinical dashboards”.5 In the United Kingdom of Great Britain and Northern Ireland, for example, the Royal College of Obstetricians and Gynaecologists has recommended the use of a maternity dashboard for all maternity units.2 The dashboard graphically presents changes over time in performance statistics and quality indicators by using a red–amber–green, traffic-light coding system to alert users to changes in the frequency of selected parameters. In high-income countries, maternity dashboards have been shown to be a feasible way of prospectively and continuously monitoring clinical outcomes. On one occasion, a maternity dashboard enabled an otherwise unrecognized adverse trend in a perinatal outcome to be detected. This trend was addressed by a training intervention that led to improvements in care and outcomes.6 Nonetheless, since 99% of maternal deaths occur in the developing world, the use of data for improving outcomes should not be restricted to high-income countries.

Local setting

The aim of this project was to assess the feasibility of adapting maternity dashboards for, and introducing them into, the Mpilo Central Hospital in Bulawayo, Zimbabwe, which is a public, tertiary referral hospital that handles around 10 000 births each year. In common with all Zimbabwean hospitals, Mpilo Central Hospital submits paper-based, quarterly maternity outcome statistics to the Ministry of Health and Child Welfare. The data, which are collated by “reproductive health officers”, include the numbers of births, caesarean sections, maternal deaths, admissions to special care baby units, stillbirths and neonatal deaths. However, as occurs in many other similar systems, the data are not readily available or accessible to local practitioners in a suitable form. Even if the data were available, however, reports are compiled only four times a year and it would still be at least 3 months – a period during which 2500 births would have taken place at Mpilo Central Hospital – before any adverse trends were apparent. Would it be feasible to review the data more frequently, perhaps at monthly intervals?

In November 2011, the Mpilo Central Hospital began running one-day, interdisciplinary, practical obstetric multiprofessional training (PROMPT) courses for all maternity staff on the labour ward. In addition, several tools were provided to improve the local work environment, including emergency boxes containing management protocols and the equipment required for managing emergencies. The primary function of maternity dashboards was to provide feedback on the effect of training. However, by identifying adverse trends in outcomes, they could also be used to highlight problems that required further investigation, for example, by clinical audit or root cause analysis. Subsequently, solutions could be developed and communicated partly or wholly via the PROMPT days.

Local maternity dashboards

The Royal College of Obstetricians and Gynaecologists has recently produced a prototype set of intrapartum clinical outcomes which are measurable and can be altered by best practice,2 in line with The good indicators guide.7 However, some important indicators, such as maternal death, were not included because they are rare in developed countries. Nevertheless, maternal death is still common in developing countries, especially in sub-Saharan Africa, and is therefore an essential indicator.

One of the authors, Thabani Sibanda, who is an obstetrician and medical statistician with experience in developing maternity dashboards,6 worked with the local reproductive health officer (a midwife), other midwives and doctors at Mpilo Central Hospital to devise a set of clinical indicators for pilot maternity dashboards that met both Zimbabwean national reporting requirements and local priorities. Two maternity dashboards were produced in Excel (Microsoft, Redmond, United States of America): an overview maternity dashboard and a maternal mortality dashboard (Box 1). The dashboard tool was designed to be user-friendly and all statistical formulae were embedded within it. Consequently, reproductive health officers, or other users, could input simple numerical data and the tool would perform all the calculations needed and present the results in a fully colour-coded display. The red, amber and green colour coding was an adaptation of the Shewhart control chart system:8 green indicated data points < 2 standard deviations from the target, amber indicated points ≥ 2 but < 4 standard deviations from the target and red, points ≥ 4 standard deviations from the target (Fig. 1).

Box 1. Clinical indicators used in pilot maternity dashboards, Zimbabwe, 2012

Overview maternity dashboard

Number of neonates born after 20 weeks’ gestation

Number of women who gave birth

Percentage of women who gave birth preterm (i.e. before 37 weeks’ gestation)

Percentage of women who died

Caesarean section rate

Assisted delivery rate

Percentage of women with a third- or fourth-degree perineal tear

Percentage of women who had a postpartum haemorrhage > 500 mL

Percentage of women with eclampsia

Percentage of neonates with an Apgar score < 7 at 5 minutes

Percentage of neonates admitted to the special care baby unit

Total stillbirth rate

Fresh stillbirth rate

Macerated stillbirth rate

Perinatal mortality rate

Maternal mortality dashboard

Percentage of women who died

Percentage of avoidable maternal deaths

Percentage of maternal deaths due to the third delay (i.e. delay in receiving adequate treatment at a health facility)

Percentage of maternal deaths that occurred in women not scheduled for antenatal care

Percentage of maternal deaths secondary to:

  • postpartum haemorrhage
  • pregnancy-induced hypertension or eclampsia
  • sepsis
  • ectopic pregnancy
  • abortion
  • placenta praevia
  • placental abruption
  • retrovirus infection
  • anaesthetic complications
  • other causes
Fig 1. Example of a maternity dashboard display, Zimbabwe, 2012
Fig 1. <b>Example of a maternity dashboard display, Zimbabwe, 2012</b>
Note: Green indicates data points < 2 standard deviations from the target; amber, data points ≥ 2 but < 4 standard deviations from the target; red, data points ≥ 4 standard deviations from the target for a given indicator.

It was important to choose targets for the different indicators carefully. For example, if the national maternal mortality ratio (MMR) for Zimbabwe in 2010 had been used as the target, the dashboard indicator would almost always be green because the MMR at Mpilo Central Hospital was significantly below the national rate. This may have resulted in complacency. A target of zero, on the other hand, would have been equally inappropriate, at least in the short-term. Use of the target in United Nations Millennium Development Goal 5a, which is to reduce the MMR by three quarters by 2015 with respect to 1990 (i.e. to 174 maternal deaths per 100 000 live births in Zimbabwe), would have resulted in repeated red indicators, even if improvement occurred. The development team felt this would be disheartening and decided to set a target that was 20% below the 2011 MMR in the hospital. This target was considered both achievable and desirable.

Changes after introducing maternity dashboards

Since March 2012, the reproductive health officer at Mpilo Central Hospital has prospectively input data for the two maternity dashboards on a monthly basis and has provided retrospective data to January 2010. In total, 28 months of outcome data were available in dashboard format at the time of writing. The use of dashboards in this setting appeared feasible.

The dashboards acted as a driver for change. They were presented during the introductory sessions of bimonthly training meetings at Mpilo Central Hospital and provided staff with the opportunity to review clinical outcomes, to understand where improvements were needed and to suggest ideas for improvement. In addition, the dashboards enabled the staff responsible for PROMPT training at the hospital to focus teaching on areas associated with indicators that were red for the majority of the time: for example, (i) an Apgar score less than 7, which is a risk factor for cerebral palsy; (ii) admissions to the special care baby unit; and (iii) stillbirths occurring during labour or delivery (i.e. fresh stillbirths). Subsequently, in an attempt to improve these outcomes, local training included sessions on monitoring the condition of the fetus during labour and on neonatal resuscitation.

In particular, the dashboards highlighted the large number of neonates being admitted to the special care baby unit, which led to a discussion between staff groups and a search for an explanation. It transpired that a specific practice had crept in: when paediatric junior doctors were not available to assess neonates before transfer to the special care baby unit, as required by local policy, the only way a midwife on the labour ward could ensure that neonates received the attention they needed from a paediatrician was to admit them to the unit. Staff were clearly aware that this situation was undesirable. An overcrowded special care baby unit would make it difficult to provide the right level of care to the right neonate. Moreover, exposing otherwise healthy babies to sick neonates increases risks in both groups and the unnecessary separation of neonates from their mothers impedes breastfeeding and bonding. In March 2012, the paediatrician in charge, the obstetrician and the midwife at the hospital began to reinforce the policy that neonates must be reviewed by a doctor on the labour ward before transfer to the special care baby unit. The proportion of neonates admitted to the unit fell from 23.1% in the first quarter of 2012 to 15.8% in the second. In addition, a review of dashboard data demonstrated that peaks in maternal deaths occurred in October and November in 2010, 2011 and 2012. Although this pattern may simply reflect a normal variation, its identification enabled hospital management to plan increases in staff training and availability for October and November 2013.

Lessons learnt

The main lessons learnt are summarized in Box 2. Initially, some staff at Mpilo Central Hospital were concerned about using maternity dashboards because they feared someone would be blamed whenever a poor outcome was highlighted. Indeed a recent literature review reported that, in general, clinicians can be reluctant to engage in improvement initiatives.9 In addition, implementation of initiatives can be limited by poor cooperation between professional groups and by resentment of policies imposed from outside or above.9 However, the simultaneous introduction of multidisciplinary PROMPT training and maternity dashboards appeared to overcome this initial reluctance, to break down boundaries between different professionals and to create a sense of local responsibility for local outcomes and a local desire for improvement.

Box 2. Summary of main lessons learnt

  • Implementation of maternity dashboards in a low-resource setting was feasible.
  • Use of a system that provided local clinical and managerial staff with outcome data resulted in locally driven actions to improve the quality of care.
  • Active participation of hospital management and midwifery staff was crucial for adapting and implementing maternity dashboards.

During our project, there was a debate about which outcome data should be collated. This is not unique. A recent study of maternity dashboards used in resource-rich settings identified 290 different clinical indicators in 96 clinical categories with up to 18 different definitions.10 Meaningful comparison of clinical outcomes between different health-care settings, districts or countries depends on agreeing on a standard set of clinical outcome measures that are relevant to both front-line staff and national policy-makers.11 In addition, data were difficult to collate for some performance indicators. In particular, it was not easy to identify cases of postpartum haemorrhage or eclampsia from the birth register. The reproductive health officer has now implemented a local notification system to identify these cases more accurately and to ensure they are included on the dashboard.

Staff appeared to like and understand the traffic-light colour-coding system, as has been reported in other maternity units.11 The use of red, amber and green works well on a computer screen and in projections. However, there was no colour-printing facility at Mpilo Central Hospital and dashboard results had to be printed out using a grey scale, which may have limited its effectiveness in highlighting areas of poor performance.

Implementation of the maternity dashboards at Mpilo Central Hospital was straightforward. It required only a local member of staff with a working knowledge of local outcome data who could enter data into the system and a computer with Excel software – such computers are readily available in Zimbabwe and the rest of the world. Although there were already people with computing skills, including familiarity with Excel, at Mpilo Central Hospital, the reproductive health officer had the greatest familiarity with the data and national reporting requirements and was regarded as the ideal person in the hospital to manage the dashboards. Moreover, she had a vested interest in the data, as hospital managers and government officials who needed data came directly to her. However, she had never used a computer, since she normally compiled paper reports. She was able to acquire the necessary computer skills on a 6-week, evening, computer training course (i.e. the International Computer Driving Licence) at a local college, at a cost of 120 United States dollars (US$). That she is now able to collect and enter data onto the maternity dashboards every month demonstrates the feasibility of an implementation strategy that involves empowering the appropriate local staff. The total cost of implementing the dashboards was low and comprised: the cost of the computer training course; US$ 400 for a laptop computer; US$ 35 for a back-up disc; and the cost of staff time for inputting data.

The Mpilo maternity dashboard was developed as part of a health partnership between Mpilo Central Hospital and North Bristol National Health Service Trust in the United Kingdom. The obstetrician primarily responsible for the design of the maternity dashboards had experience working in both Zimbabwean and United Kingdom health-care systems, which was invaluable for the development of these low-cost, automated dashboards. Although the dashboards were successfully implemented at Mpilo Central Hospital, the assistance of external health partners was required for the initial installation and for troubleshooting. Clearly the wider use of any form of dashboard would benefit from the development of formal training for those involved in its implementation.

Thoughts for the future

The Health Metrics Network was launched in 2005 to help countries improve their health information systems for use in evidence-based decision-making. Subsequently, the Commission on Information and Accountability for Women's and Children's Health was established to determine the most effective international institutional arrangements for ensuring global reporting, oversight and accountability on women's and children's health. The Commission covers 74 countries, 43 of which are in sub-Saharan Africa. In 2012, 40 of the 74 countries completed a detailed assessment for the Health Metrics Network.4 Although Zimbabwe did not complete the assessment, the findings were relevant to the country. Often data on the MMR were found to be questionable, imprecise and unrepresentative. There was also a lack of timeliness: data were summarized and returned to users months after collection. In our view, such data are of little practical use to front-line clinical staff. Furthermore, the assessment revealed that staff at health-care facilities did not know whether or not the data were actually used. This acted as a disincentive to the collation or even the collection of information.4

A WHO Country Accountability Framework assessment held in the United Republic of Tanzania in 2012 proposed several actions for the Zimbabwean Health Service: (i) strengthening data quality audits; (ii) improving access to up-to-date data in districts and health-care facilities; and (iii) establishing a national monitoring and evaluation committee to strengthen coordination between districts.12 We believe the compilation of dashboards by local staff would help improve the accuracy, timeliness and completeness of data.

One successful aspect of our project at Mpilo Central Hospital was that the maternity dashboards were primarily implemented by clinical staff who immediately understood that clinical data could be used to improve outcomes in their departments. The project was not a patient safety initiative imposed from above. Instead, it involved a bottom-up approach. However, the local nature of the project may also have had the disadvantage that key decision-makers outside the local area may have been unaware of it, which could have inadvertently limited its widespread adoption.

Historically, quality improvement programmes have been more successful in some organizations than others.13 Both the quality of the intervention and the implementation strategy can enhance clinical outcomes.14 Indeed, in a recent evaluation of WHO Patient Safety Solutions Aides Memoire, which are intended to serve as information tools for governments and health-care organizations on priority patient safety problems, many interviewees felt that the dissemination process used by WHO was not effective.15 The perception was that a greater emphasis had been placed on the aides-memoire themselves than on implementing a proactive strategy for dissemination. While it is encouraging that maternity dashboards were successfully introduced into one hospital in Zimbabwe, it would be unwise to immediately extend their use to a large number of other centres without a clear proactive dissemination plan that involves front-line health workers, hospital managers, ministries of health, governments and WHO country and regional offices.15

Conclusion

We believe this is the first report of the introduction of maternity dashboards in a low-resource setting. Active participation of the Mpilo Central Hospital board and senior medical and midwifery staff was crucial to its success, as was the early involvement and training of a reproductive health officer. Since the introduction of the dashboards, hospital staff have changed organizational systems and developed training to address areas that could be improved. When they occurred, improvements were displayed on the dashboards, which reinforced best practice.

Although the maternity dashboards were implemented in only one hospital, other units have already expressed an interest and there is a potential for expanding their use both within and outside Zimbabwe. Better Internet access makes it possible to link local units to regional, national and international maternity dashboards. However, to be successful, the more widespread use of maternity dashboards requires the participation of all key stakeholders, investment in computer hardware, training for local data managers and an integrated local training and monitoring system. The difficulties are not insurmountable. We believe the use of maternity dashboards should not be limited to high-resource settings. If implemented effectively, they can lead to improvements in both health care and health outcomes that will benefit mothers and infants around the world.


Acknowledgements

We thank Mpilo Central Hospital managers and clinical staff and all other contributors in Zimbabwe and the United Kingdom for their support in the implementation of maternity dashboards and PROMPT training at Mpilo Central Hospital, Bulawayo, Zimbabwe.

Funding:

The project received financial support from the Health Partnership Scheme, which is funded by the United Kingdom Department for International Development and managed by the Tropical Health and Education Trust. Funding was also received from the Laerdal Medical Foundation.

Competing interests:

Tim Draycott is a trustee and Joanna Crofts and Thabani Sibanda are members of the PROMPT Maternity Foundation, the charity that facilitated the introduction of in-hospital obstetric emergencies training. They do not receive any financial reward for their involvement in the charity.

References

Share