Improving quality and use of data through data-use workshops: Zanzibar, United Republic of Tanzania
Jørn Braa a, Arthur Heywood a & Sundeep Sahay a
a. Department of Informatics, University of Oslo, PO Box 1080, Blindern, 0316 Oslo 47, Norway.
Correspondence to Jørn Braa (e-mail: firstname.lastname@example.org).
(Submitted: 18 November 2011 – Revised version received: 29 January 2012 – Accepted: 31 January 2012.)
Bulletin of the World Health Organization 2012;90:379-384. doi: 10.2471/BLT.11.099580
Good health information systems are crucial for addressing health challenges and improving health service delivery in developing countries.1 However, the quality of the data produced by such systems is often poor and the data are not used effectively for decision-making.2 Although there has been increasing international attention to the need to develop strong health information systems, it has proved difficult to do so for several reasons, including fragmentation and lack of coordination of health programmes and insistence by international agencies on maintaining their own vertical systems3; lack of shared data standards4; unrealistic ambitions5; inability of system developers to handle complex organizational, social and cultural issues6; and problems of sustainability.7 The Health Metrics Network, established in 2005, has been instrumental in addressing the problem of fragmentation in health information systems through its technical framework,8 which promotes a data warehouse approach to information system integration,3,9 and in creating global consensus on the need for all actors to join forces and work towards integrated systems. Zanzibar, in the United Republic of Tanzania, provides an early example of this shift towards integration and the use of an integrated data warehouse/repository application to achieve it.
A key hypothesis in the experience reported in this paper is that data quality and data use are interrelated: poor quality data will not be used, and because they are not used, the data will remain of poor quality; conversely, greater use of data will help to improve their quality, which will in turn lead to more data use. This hypothesis was tested through data-use workshops, which sought to encourage data use and enhance data quality by promoting systematic peer review, building teamwork and stimulating self-assessment, using indicators to measure targets.
Background and methods
Zanzibar consists of two islands, each making up a health zone; one island comprises six districts and the other four. As part of a health system strengthening project,10 in 2005 the Health Management Information System (HMIS) Unit of the Ministry of Health, with support from the Danish International Development Agency, launched a process aimed at strengthening the HMIS, improving data reporting and implementing the District Health Information Software (DHIS) (Fig. 1). DHIS is a software tool for collection, validation, analysis and presentation of aggregate statistical data managed in multiple data sets and tailored to support integrated health information management activities. It is designed to serve as a district-based country data warehouse that addresses both local and national needs. Version 2 of the program is an open-source web-based software package available free of charge (www.DHIS2.org).11 It is being used by national health management information systems in numerous countries in Africa and Asia.
Fig. 1. District Health Information Software screenshot and example of data presentation
Quarterly data-use workshops were held in which district health management team (DHMT) members (roughly seven per district) presented their district’s routine data to their peers from other districts. The workshops lasted approximately 5 days each and were facilitated by external facilitators from the Health Information Systems Programme, supported by the Zanzibar HMIS Unit and selected health zone staff. The workshops began in 2005 and have continued since then. Most workshops are now being run by the HMIS Unit without outside help.
During the workshops, each district or programme presents and assesses its own data using standardized analysis templates based on the Millennium Development Goals and local strategic plans, after which their peers discuss and critique these presentations. The workshops encourage self-assessment and provide an opportunity to compare presentations, identify common issues relating to data quality and health services performance, promote local involvement and improve data quality. They also contribute direct feedback to HMIS planners for the revision of indicators and data sets and to software developers for the design of new functionalities, reports and other features.
As participants became more familiar and proficient with data-handling processes (e.g. indicators, analysis, display) and the tools (e.g. DHIS, Excel pivot tables and graphs), the length of their participation in workshops became shorter. At the same time, as more programmes started to use DHIS, more programme managers participated and the overall workshop duration increased.
Improvements were noted in the following areas as a result of the data-use workshops:
Forms were simplified on the basis of revised indicator and data sets, dramatically reducing the number of data elements collected and thus the workload of facility staff. This simplification was achieved largely because workshop participants realized that it was unnecessary to disaggregate some data (e.g. by age, sex or uncommon diseases) as the disaggregated data were not being used. Similarly, duplication of data collection by different programmes was virtually eliminated (for example, the Reproductive and Child Health Programme stopped collecting data on human immunodeficiency virus [HIV] infection, immunization and malaria) and data gaps were filled. Changes were agreed collectively through improved communication fostered by the workshops and supported by strong leadership from the Ministry of Health.
Data submission improved considerably, with districts reporting more regularly on most data forms. The process began modestly, focusing on a couple of programmes, but other programmes gradually saw the value of using the national HMIS rather than their own parallel data collection systems, and more programmes (and hospitals, including the national referral hospital) were added. Indicator set changes, including some reductions in indicators, were negotiated with programmes jointly each year through the HMIS Unit, the Health Information Systems Programme and the two health zones.
The workshops provided a stimulus for integration of the previously separate data sets and databases of primary health-care units, hospitals and programmes, and allowed district health management team members to gain a better idea of the roles played by different actors, which improved practical collaboration. Integration of programme data into a single DHIS database – with one national data set covering the Millennium Development Goals indicators, poverty reduction and national strategic plan indicators and programme-specific indicators – was a major achievement. Integration was a slow process, however, as some externally funded programmes were initially reluctant to share “their” data and did not trust the quality or timeliness of the national database.
Data quality improved dramatically, thanks to increased use of quality checks (for timeliness, correctness, consistency and completeness of data) at the facility level, use of computer checks by districts and practical experience gained under supervision during workshops. Mistakes were identified by participants when data were presented during workshops, sometimes leading to heated discussion of quality issues, which made a strong impression on participants.
Data analysis and interpretation
At the start of the process, most district health management team staff did not think in terms of indicators, and their presentations focused on raw data. As information officers and programme managers became more competent in using the HMIS, data analysis tools (targets and indicators) became more widely used and understood, which strengthened self-assessment and “epidemiological thinking”. The link between plans, targets and indicators was emphasized, which helped to increase the use of indicators at local levels and the analysis of coverage and quality of service delivery. Fig. 1 shows a screenshot of the DHIS dashboard displaying analytical graphs. Some examples of improved local information use are summarized in Table 1.
Table 1. Examples of improved information use resulting from project for strengthening the Health Management Information System (HMIS) in Zanzibar, the United Republic of Tanzania, 2005–2008
District health management team members honed their ability to solve problems using HMIS data as they gained greater appreciation of the value of improved data quality and felt more competent to perform HMIS tasks and apply epidemiological approaches to daily data management issues.
Team work improved considerably as the district health management teams shared information about service delivery and used the HMIS to monitor and evaluate progress towards targets set in their district annual plans. Leaders became more confident in making evidence-based decisions to improve quality of care based on collective values developed through the data-use workshops. While a “culture of information” was not fully established, significant strides were made in that direction.
Practical computer skills
Workshop participants improved their computer skills in using DHIS for analysis, presentation and dissemination and also enhanced their knowledge of basic hardware and software maintenance, virus protection and backups. Software developers benefited from insights into new requirements and gained a better understanding of weaknesses related to local context and configuration.
DHMT members’ presentation skills were initially weak, as they were unused to drawing graphs, using PowerPoint, engaging in debate or offering constructive criticism. These skills improved dramatically as a result of the workshops, especially when standardized templates for presentations were developed. Local HMIS Unit and health zone personnel acquired sufficient skills to run the workshops without outside facilitators. Box 1 summarizes the lessons learnt from this experience.
Box 1. Summary of main lessons learnt
- The outcomes of the data-use workshops demonstrate and validate our hypothesis that the more data are used, the more data quality will improve, leading to significant innovations in the use of information and breaking the vicious cycle of non-use and poor quality of data.
- An integrated framework for HMIS, using a national data warehouse framework, provides an enabling environment in which actors, health programmes and systems can “speak to each other”, which is the foundation for improving health systems.
- Regular data-use workshops, with self-assessment and peer critique and discussion of the data presented, provide a powerful means of building a strong evidence base for HMIS improvements.
The data-use workshops approach developed in Zanzibar helped to strengthen the United Republic of Tanzania’s HMIS and thereby its overall health system, providing a concrete example of health information systems strengthening. The approach has been applied in other countries, such as in Kenya and Rwanda, where it has become an institutionalized part of quarterly review processes.
- AbouZahr C, Boerma T. Health information systems: the foundations of public health. Bull World Health Organ 2005; 83: 578-83 pmid: 16184276.
- Mavimbe JC, Braa J, Bjune G. Assessing immunization data quality from routine reports in Mozambique. BMC Public Health 2005; 5: 108- doi: 10.1186/1471-2458-5-108 pmid: 16219104.
- Sæbø J, Kossi E, Hodne Titlestad O, Rolland Tohouri R, Braa J. Comparing strategies to integrate health information systems following a data warehouse approach in four countries. Info Technol Develop 2011; 17: 42-60 doi: 10.1080/02681102.2010.511702.
- Braa J, Hanseth O, Heywood A, Mohammed W, Shaw V. Developing health information systems in developing countries: the flexible standards strategy. MIS Quart 2007; 31: 381-402.
- Heeks R. Failure, success and improvisation of information systems projects in developing countries. Inf Soc 2002; 18: 101-12 doi: 10.1080/01972240290075039.
- Aqil A, Lippeveld T, Hozumi D. PRISM framework: a paradigm shift for designing, strengthening and evaluating routine health information systems. Health Policy Plan 2009; 24: 217-28 doi: 10.1093/heapol/czp010 pmid: 19304786.
- Kimaro HC, Nhampossa JL. Analyzing the problem of unsustainable health information systems in less-developed economies: case study from Tanzania and Mozambique. Info Technol Develop 2005; 11: 273-98 doi: 10.1002/itdj.20016.
- Health Metrics Network [Internet]. Framework and standards for country health information systems. 2nd ed. Geneva: World Health Organization; 2008. Available from: http://www.who.int/healthmetrics/documents/framework/en/index.html [accessed 10 February 2012].
- Braa J. A data warehouse approach can manage multiple data sets. Bull World Health Organ 2005; 83: 638-9 pmid: 16184286.
- Sheik YH, Titlestad OH. Implementing health information systems in Zanzibar: using internet for communication, information sharing and learning. In: Proceedings of the IST-Africa 2008 Conference and exhibition; 2008 May 7–9; Windhoek, Namibia.
- District Health Information Software 2 [Internet]. Oslo: Health Information Systems Programme; 2012. Available from: http://www.hisp.uio.no/ [accessed 15 March 2012].
- Couple year protection rate [Internet]. Oslo: Health Information Systems Programme; 2012. Available at: http://indicators.hst.org.za/healthstats/231/data [accessed 15 March 2012].
- Monitoring emergency obstetric care: a handbook. Geneva: World Health Organization; 2009. Available from: http://www.unfpa.org/webdav/site/global/shared/documents/publications/2009/obstetric_monitoring.pdf [accessed 15 March 2012].