External quality assessment of Giemsa-stained blood film microscopy for the diagnosis of malaria and sleeping sickness in the Democratic Republic of the Congo
Pierre Mukadi a, Philippe Gillet b, Albert Lukuka a, Benjamin Atua c, Nicole Sheshe d, Albert Kanza d, Jean Bosco Mayunda d, Briston Mongita e, Raphaël Senga f, John Ngoyi g, Jean-Jacques Muyembe a, Jan Jacobs b & Veerle Lejon h
a. Institut National de Recherche Biomédicale, Kinshasa, Democratic Republic of the Congo.
b. Department of Clinical Sciences, Institute of Tropical Medicine, Antwerp, Belgium.
c. Programme national de lutte contre le paludisme, Kinshasa, Democratic Republic of the Congo.
d. Programme national de lutte contre la tuberculose, Kinshasa, Democratic Republic of the Congo.
e. Division provinciale de la santé, Kisangani, Democratic Republic of the Congo.
f. Division provinciale de la santé, Goma, Democratic Republic of the Congo.
g. Division provinciale de la santé, Mbuji-Mayi, Democratic Republic of the Congo.
h. Institut de Recherche pour le Développement, UMR 177 IRD-CIRAD INTERTRYP, Campus International de Baillarguet TA A-17/G, 34398 Montpellier Cedex 5, France.
Correspondence to Veerle Lejon (e-mail: email@example.com).
(Submitted: 10 September 2012 – Revised version received: 27 February 2013 – Accepted: 07 March 2013 – Published online: 29 April 2013.)
Bulletin of the World Health Organization 2013;91:441-448. doi: http://dx.doi.org/10.2471/BLT.12.112706
With 97% of its population living in areas of stable malaria transmission, the Democratic Republic of the Congo is among the countries with the highest malaria burden in Africa.1,2 For diagnosing malaria, the microscopic detection of Plasmodium in thick blood films remains the gold standard, although reliable rapid diagnostic tests are increasingly available.3 In addition, microscopic examination of stained blood films also enables the detection of other pathogens, including Trypanosoma brucei ssp., some of which cause human African trypanosomiasis. Half of all people at risk of African trypanosomiasis live in the Democratic Republic of the Congo, where almost 80% of all reported cases have occurred.4
The microscopic detection of Plasmodium and Trypanosoma requires technical skill, thorough training, maintenance of expertise and regular handling of samples containing the parasites. The quality of microscopy, however, often remains inadequate because of poor-quality equipment, insufficient training or a lack of quality assurance.5 In 2002, the World Health Organization (WHO) launched an external quality assessment programme to improve the proficiency of microbiological testing for diseases prone to cause epidemics in the WHO African Region, including malaria.6,7
In 2010, the first, national, large-scale external quality assessment in the Democratic Republic of the Congo was carried out in four of the country’s 11 provinces.8 It revealed the poor quality of malaria microscopy. In this paper, we present the results of the second, nationwide external quality assessment of stained blood film microscopy. Since trypanosomiasis is widespread in the Democratic Republic of the Congo and accurate diagnosis is important, the assessment included a slide with trypanosomes. In addition, given the increasing use of rapid diagnostic tests for malaria,2 participants in the quality assessment were questioned about use of these tests.
The data obtained during the second external quality assessment of stained blood film microscopy in the Democratic Republic of the Congo demonstrate the benefits of such assessments, show they are feasible when carried out in collaboration with vertical, disease-oriented programmes and highlight the difficulty of diagnosing human African trypanosomiasis.
The second external quality assessment was organized by the Institut National de Recherche Biomédicale in Kinshasa, which is the reference laboratory for the National Malaria Control Programme (Programme national de lutte contre le paludisme) in the Democratic Republic of the Congo. Between July and October 2011, 356 clinical laboratories in the country’s 11 provinces were contacted. The laboratories had either subscribed to and participated in the previous external quality assessment, which was also organized by the Institut National de Recherche Biomédicale,8 or were part of the network of the National Malaria Control Programme or the National Tuberculosis Control Programme (Programme national de lutte contre la tuberculose). The number of human African trypanosomiasis cases in each province in 2011 is shown in Fig. 1.9 No data were available for Sud-Kivu because of the ongoing war and the number of cases was assumed to be under 100.
Fig. 1. Laboratories participating in the external quality assessmenta and human African trypanosomiasis cases, by province, Democratic Republic of the Congo, 2011
The samples used in the external quality assessment comprised four Giemsa-stained, thick and thin blood films (Table 1), prepared as previously described.8 All blood samples were prepared using ethylenediaminetetraacetic acid (EDTA) and were left over from samples drawn for routine clinical care from patients presenting at the Institut National de Recherche Biomédicale. Slides containing trypanosomes were prepared from rodent blood infected with Trypanosoma brucei brucei and diluted in human EDTA blood. Thick and thin blood films were placed on a single slide (Menzel-Gläzer, Braunschweig, Germany). After fixation of the thin blood films with methanol, the films were treated with Giemsa stain (Merck, Darmstadt, Germany). For conservation, the blood films were covered with Entellan (Merck, Darmstadt, Germany) and a coverslip. Parasite density was determined at 1000-times magnification by six expert microscopists from the Institut National de Recherche Biomédicale and the Institute of Tropical Medicine in Antwerp, Belgium, and the reference value was taken to be the mean density ± 2 standard deviations (SDs). For all slides, the presence or absence of Plasmodium species and the species present were evaluated using real-time polymerase chain reaction methods.10,11 The slides were sent to coordinators in each province who transported them to participating laboratories along with a teaching sheet explaining the procedure that should be used to determine parasite density. On detecting a malaria infection, participants were to report the Plasmodium species present, the developmental stage of the parasite and parasite density, expressed as trophozoites per µl. Other findings were to be reported as comments.
Table 1. Sample analysis, external quality assessment of laboratories, Democratic Republic of the Congo, 2011
Participants were also asked to complete a questionnaire on: (i) their use of microscopy and rapid diagnostic tests for malaria diagnosis; (ii) the number of tests performed each month; (iii) the proportion of test results that were positive; and (iv) the number of staff performing diagnostic tests and their training. In addition, information was requested about the type and brand of the rapid diagnostic tests used and occurrence of shortages.
The results of the microscopic analysis were categorized as: without errors; with minor errors; or with major errors (Table 1). The distinction between a minor and a major error was based on the potential effect the error could have on the patient's diagnosis and clinical management. A parasite density less than 2 SDs from the mean (reference) density, as determined by the expert microscopists, was considered to be without error. A density that lay outside this range was considered a major error if it was more than 10 times or less than one 10th the reference value; all other densities outside the range were considered minor errors. A slide analysis was considered correct if no error or a minor error was made and incorrect if a major error was made (Table 1). The target standard for the performance of the slide analysis was a correct result for each of the four slides. Data were entered into Excel spreadsheets (Microsoft, Redmond, United States of America) and analysed using Stata version 10.0 (StataCorp. LP, College Station, USA). Differences between proportions were tested for significance using the χ2 or Fisher exact test and differences between two groups of continuous variables were tested for significance using the Mann–Whitney rank–sum test. Trends in proportions were assessed using the χ2 test for trends.
Responses were received from 277 of the 356 (77.8%) laboratories contacted. Of the 277 responders (Fig. 1), 11 (4.0%) were provincial reference laboratories, 109 (39.4%) were general hospitals, 143 were health centres (51.6%) and 14 (5.1%) were private laboratories. Nationally, coverage was 91.7% for provincial reference laboratories (11 of 12), 27.7% for hospitals (109 of 393) and 1.7% for health centres (143 of 8266). Coverage for private laboratories was not known. The median time taken for the quality assessment samples to arrive at participating laboratories was 27 days (range: 1–109). Reports were returned with a median delay of 7 days (range: 1–83).
Table 1 summarizes the results of the analysis of the four external quality assessment samples at participating laboratories. Slide 1, which contained Plasmodium falciparum gametocytes only, was read without errors by 44.8% of participants, whereas 37.7% made a minor error and 17.5% made a major error. Slide 2, which contained no parasites, was scored as negative by 81.0% of participants but the remaining 19.0% made the major error of reporting the presence of Plasmodium. Slide 3, which contained trypanosomes, was reported correctly by 44.4% of participants. In addition, 5.2% reported the presence of trypanosomes but also malaria infection, which was considered a minor error. However, 50.4% missed the diagnosis of trypanosomiasis. Of the 136 participating laboratories that did not report trypanosomes, 51.5% made no comment, 44.9% reported the presence of Plasmodium, 1.8% could not identify the parasite or reported the presence of an unusual parasite form, 0.6% suggested that Loa loa microfilaria were present and 0.6% reported fibres on the slide. Slide 4 contained P. falciparum trophozoites at a reference density of 113 530 per µl. Only 13.7% of participating laboratories recorded a parasite density less than 2 SDs from the mean (i.e. 63 000 to 164 000 per μl), which was considered without error; minor errors were made by 59.4% and major errors, by 26.9%. The trophozoite densities reported ranged between 32 and 24 935 064 per µl (Fig. 2). The most common error, made by 49.1%, was to report a parasite density below 63 000 per µl. Finally, 11.1% of participants estimated the density to be less than one 10th the reference value (i.e. < 11 353 per µl), whereas 0.7% estimated it to be more than 10 times the reference value (i.e. > 1 135 300 per µl).
Fig. 2. Laboratory measurement of Plasmodium falciparum trophozoite density on test slides, external quality assessment, Democratic Republic of the Congo, 2011
Results for all four slides were received from 263 participating laboratories; 92 (35.0%) reported a correct result (i.e. no error or a minor error) on all slides (Fig. 3). However, 3.0% reported no correct results and 11.0% reported only one correct result. Significantly fewer laboratories reported a correct result for slide 3, which contained trypanosomes: 50.2% reported a correct result compared with 73.4 to 82.1% for the other slides (P = 0.02).
Fig. 3. Venn diagram of laboratoriesa reporting correct results for individual test slides and their combination, external quality assessment, Democratic Republic of the Congo, 2011
Of the 271 laboratories that responded, 257 (94.8%) reported that they usually diagnosed malaria by thick blood film microscopy. The median number of slides processed was 159 per month (interquartile range, IQR: 86–352; maximum: 2532), by a median of 2 (IQR: 1–3) laboratory technicians. The median proportion that tested positive was 51.9% (IQR: 31.0–70.5). Only 100 of the 271 (36.9%) laboratories had taken part in training on malaria microscopy; the median time since training was 3 years (IQR: 1–5).
Rapid diagnostic tests for malaria were used by 44.3% (120/271) of the participating laboratories. The median number of tests performed was 160 per month (IQR: 36–188; maximum: 1283), by a median of 4 (IQR: 2–6) people. The median proportion that tested positive was 40.0% (IQR: 24.2–58.0). Overall, 64 of the 120 (53.3%) laboratories had taken part in training on rapid diagnostic tests, mostly in 2010, and the median length of experience was 8 months (IQR: 6–12). In 13.3% of the laboratories, rapid tests had replaced microscopy for malaria diagnosis. Stock disruption was reported by 30.8%. In addition, 29.2% of the laboratories used rapid diagnostic tests during night shifts, 7.5% used them in emergencies and 2.5% used them for ensuring transfusion safety, whereas 35.0% did not specify the circumstances in which they used the tests. The most common test, used by 75% of the laboratories, was an SD Bioline malaria rapid test (Standard Diagnostics Inc., Hagal-Dong, Republic of Korea), followed by the Paracheck Pf (Orchid Biomedical Systems, Goa, India), used by 14.2%.
Factors affecting analytical performance
Whether a laboratory regularly used microscopy or rapid diagnostic tests for malaria diagnosis had no significant effect on the proportion that reported a correct result for all four slides: 85 of 240 laboratories that used microscopy reported four correct results compared with 2 of 11 laboratories that used only rapid tests (P = 0.3). Moreover, laboratories that had taken part in training did not have better results than those that had not (P = 0.09). There was no significant difference between laboratories that had four correct results and other laboratories in the median number of slides processed per month: 166 per month (IQR: 88–438) versus 152 per month (IQR: 83–286), respectively (P = 0.2). In particular, laboratories that processed more than 300 slides per month did not perform significantly better than those that processed fewer (P = 0.052). Nor did higher-level referral centres perform significantly better: 54.5% of provincial reference laboratories correctly read all four slides, compared with 42.3% of hospitals and 30.6% of health centres (P = 0.08 for trend). However, laboratories that were participating in an external quality assessment for the second time did perform significantly better: 64 of 152 (42.1%) laboratories participating for the second time reported four correct results compared with 28 of 111 that were participating for the first time (25.2%; P = 0.005). Finally, 110 of 193 (57.0%) laboratories in provinces with more than 100 African trypanosomiasis cases annually (Fig. 1) correctly reported trypanosomes on slide 3, compared with 24 of 77 (31.2%) laboratories in provinces with fewer cases (P < 0.001).
There were significantly fewer serious errors in the second external quality assessment than in the first.8 In the present quality assessment, 51 of 269 (19.0%) laboratories reported false-positive results for the slide with no parasites, compared with 58 of 174 (33.3%) in the first assessment (P < 0.001). Major errors in parasite density estimates were also less frequent in this assessment: 32 of 186 (17.2%) laboratories compared with 23 of 44 (52.3%) in the first external quality assessment (P < 0.001).
Overall, the quality of Giemsa-stained blood film microscopy observed in the present external quality assessment was poor. The principle serious errors were: not diagnosing trypanosomiasis; not recognizing P. falciparum gametocytes; diagnosing malaria from a slide with no parasites; and substantial quantitative errors in parasite density estimates.
The present assessment had several limitations. The diagnostic competence of the participating laboratories may have been overestimated and might not be representative of all laboratory staff involved because the most experienced staff may have read the slides for the external quality assessment.12 In addition, the test slides were stained before receipt and staining quality is known to be poor in routine laboratory settings.8 Another limitation stems from the estimated distribution of human African trypanosomiasis cases in different provinces. The disease is highly focal and, in particular, the risk varies greatly across Orientale, Équateur, Katanga and Maniema provinces, which contain large areas where there is no risk.13 Consequently, a laboratory may serve a population that lives either completely inside or outside an area where there is a disease risk and, therefore, its experience with the disease may not be typical of the province in which it is located.
A major strength of this external quality assessment was the synergistic collaboration with existing government networks and vertical programmes, in particular with the national tuberculosis control programme.14 We found that the quality assessment could be carried out without the need for new structures or organizations. Working with existing networks and programmes is less expensive and more sustainable. Moreover, over the long term, it is probably more feasible to test diagnostic proficiency by external quality assessment than by using a quality control system to cross-check routine blood slides, which can result in a heavy workload.15 Despite the logistic challenges of organizing external quality assessments in poorly accessible areas of countries lacking in infrastructure, such as the Democratic Republic of the Congo, the response time was not much longer than that reported by WHO for quality assessments of national public health laboratories in other parts of Africa.7
Another advantage of external quality assessments is that they are usually conceived as didactic programmes: they often include a feed-back report that explains the background to and pitfalls of testing and refreshes knowledge of state-of-the-art techniques. The present quality assessment involved the distribution, at little cost, of a report on the previous external quality assessment and a teaching sheet, both of which helped encourage participation. However, external quality assessment programmes do not replace training. Since 2011, the National Malaria Control Programme in the Democratic Republic of the Congo has updated technical guidance on malaria diagnosis. The education of trainers has started at the provincial level and, to date, training has been carried out at over 100 health-care facilities in four provinces. Unfortunately, the lack of infrastructure in the country is a limitation. Nevertheless, our findings show that performance can be improved: (i) laboratories that participated in an external quality assessment for a second time performed better; (ii) there were fewer false-positive results than in the previous assessment; and (iii) the number of major quantitative errors in estimating parasite density was lower than previously. These observations confirm findings in the United Kingdom.12
External quality assessments also provide the opportunity to introduce laboratories to new or rarely-seen parasites.12 As observed in WHO external quality assessments, correctly identifying blood parasites other than P. falciparum can be a problem.7 That half of participating laboratories in our assessment did not recognize the presence of trypanosomes is a concern, especially since the diagnosis of African trypanosomiasis in the Democratic Republic of the Congo will increasingly shift from active case-finding by mobile teams to integration into the horizontal health-care system.4 In contrast, ability to identify trypanosomes was relatively good in the slides included in the external quality assessments performed by WHO7 and in the United Kingdom12. However, most participants in WHO’s assessments were national reference laboratories, where training and reference documentation may be better.
In WHO’s assessments, an estimated parasite density in the target range of 0.5 to 1.5 times the reference value was regarded as acceptable. This range corresponds approximately to our definition of no error. Overall, 13.7% of laboratories in our assessment made no error in their parasite density estimate compared with 51% that reported an acceptable value in the WHO assessments.7 In both the present and previous external quality assessment in the Democratic Republic of the Congo,8 parasite density tended to be underestimated. One reason may be that density has been expressed in parasites per µl in the country only since 2010. The distribution of teaching sheets during the current external quality assessment may explain why there were fewer errors in parasite density estimates than in the previous assessment.
In addition, the proportion of false-positive results for Plasmodium in the current assessment (i.e. 19%) was lower than previously but higher than the 8 to 15% and the 2% reported, respectively, by two external quality assessments carried out in settings where malaria is not endemic.12,16 Although the laboratories in our assessment should have encountered Plasmodium more often, they were probably less familiar with external quality assessments and expected the slides in a malaria microscopy assessment to contain parasites. On the other hand, the proportion of false-positive results was much lower than that observed in a study of the sensitivity and specificity of rapid diagnostic tests and malaria microscopy carried out in 2011 in the Democratic Republic of the Congo, in which the false-positive rate for routine microscopy was 70.6%.17 The difference may be due to the poor quality of Giemsa staining and of the reagents used under routine conditions8,18 or to the possibility that better laboratories are more likely to participate in external quality assessments.
In contrast to previous and other external quality assessments, we found no clear association between analytical performance and how regularly a laboratory handled infected samples. In areas where malaria is endemic, laboratory staff may acquire the necessary expertise even if they only assess 40 slides per month. The difference in performance between the current and previous external quality assessments in the Democratic Republic of the Congo could be due to differences in participants or to an overall improvement.
The results of the questionnaire were similar in current and previous assessments. The lack of training remains of concern. Notably, the use of rapid diagnostic tests has almost doubled since 2010, when they were used by only 24.7% of participants. However, there seems to be no clear strategy for deciding when to use them. The upward trend in the Democratic Republic of the Congo appears to reflect that observed throughout Africa.2
In conclusion, the overall quality of stained blood film microscopy in the Democratic Republic of the Congo was poor. In particular, the failure to recognize trypanosomes in a country where sleeping sickness is endemic is concerning. Since the use of rapid diagnostic tests has risen, relevant external quality assessments are increasingly important. Participating in an external quality assessment appeared to improve a laboratory’s performance but this needs to be confirmed by longer follow up. Although organizing quality assessments and training is challenging in vast, resource-poor countries, it is feasible when carried out in collaboration with existing government networks and vertical programmes. Moreover, external quality assessments can provide opportunities for educational activities, such as distributing teaching sheets, and the involvement of major stakeholders can increase awareness of the importance of these assessments, ensure they are sustainable and create opportunities for continued funding.
John Ngoyi is also affiliated with the Institut Supérieur des Techniques Médicales, Mbuji-Mayi, Democratic Republic of the Congo and Jean-Jacques Muyembe is also affiliated with the Faculté de Médecine, Université de Kinshasa, Kinshasa, Democratic Republic of the Congo. Veerle Lejon's research for this study was conducted in part at the Department of Clinical Sciences of the Institute of Tropical Medicine in Antwerp, Belgium.
This study was funded by the Directorate General for Development Cooperation of the Belgian Government through Institutional Collaboration INRB-ITM (Network Program on Laboratory Quality Management, Project 3.21). The funders had no role in the study design, data collection and analysis, the decision to publish or preparation of the manuscript.
- Hay SI, Okiro EA, Gething PW, Patil AP, Tatem AJ, Guerra CA, et al., et al. Estimating the global clinical burden of Plasmodium falciparum malaria in 2007. PLoS Med 2010; 7: e1000290 http://dx.doi.org/10.1371/journal.pmed.1000290 pmid: 20563310.
- World malaria report: 2012. Geneva: World Health Organization; 2012.
- Abba K, Deeks JJ, Olliaro P, Naing CM, Jackson SM, Takwoingi Y, et al., et al. Rapid diagnostic tests for diagnosing uncomplicated P. falciparum malaria in endemic countries. Cochrane Database Syst Rev 2011CD008122 pmid: 21735422.
- Simarro PP, Jannin J, Cattand P. Eliminating human African trypanosomiasis: where do we stand and what comes next? PLoS Med 2008; 5: e55 http://dx.doi.org/10.1371/journal.pmed.0050055 pmid: 18303943.
- malERA Consultative Group on Diagnoses and Diagnostics. A research agenda for malaria eradication: diagnoses and diagnostics. PLoS Med 2011; 8: e1000396 http://dx.doi.org/10.1371/journal.pmed.1000396 pmid: 21311583.
- Policy and procedures of the WHO/NICD microbiology external quality assessment programme in Africa. Geneva: World Health Organization; 2007 (WHO/CDS/EPR/LYO/2007.3).
- Frean J, Perovic O, Fensham V, McCarthy K, von Gottberg A, de Gouveia L, et al., et al. External quality assessment of national public health laboratories in Africa, 2002–2009. Bull World Health Organ 2012; 90: 191-9A http://dx.doi.org/10.2471/BLT.11.091876 pmid: 22461714.
- Mukadi P, Gillet P, Lukuka A, Atua B, Kahodi S, Lokombe J, et al., et al. External quality assessment of malaria microscopy in the Democratic Republic of the Congo. Malar J 2011; 10: 308 http://dx.doi.org/10.1186/1475-2875-10-308 pmid: 22008378.
- Kande Betu-Ku-Mesu V. Rapport annuel 2011. Kinshasa: Programme Nationale de lutte contre la trypanosomiase humaine Africaine; 2012. French.
- Cnops L, Van Esbroeck M, Bottieau E, Jacobs J. Giemsa-stained thick blood films as a source of DNA for Plasmodium species-specific real-time PCR. Malar J 2010; 9: 370 http://dx.doi.org/10.1186/1475-2875-9-370 pmid: 21176207.
- Cnops L, Jacobs J, Van Esbroeck M. Validation of a four-primer real-time PCR as a diagnostic tool for single and mixed Plasmodium infections. Clin Microbiol Infect 2011; 17: 1101-7 http://dx.doi.org/10.1111/j.1469-0691.2010.03344.x pmid: 20718798.
- Kettelhut MM, Chiodini PL, Edwards H, Moody A. External quality assessment schemes raise standards: evidence from the UKNEQAS parasitology subschemes. J Clin Pathol 2003; 56: 927-32 http://dx.doi.org/10.1136/jcp.56.12.927 pmid: 14645352.
- Simarro PP, Cecchi G, Franco JR, Paone M, Diarra A, Ruiz-Postigo JA, et al., et al. Estimating and mapping the population at risk of sleeping sickness. PLoS Negl Trop Dis 2012; 6: e1859 http://dx.doi.org/10.1371/journal.pntd.0001859 pmid: 23145192.
- Sarkinfada F, Aliyu Y, Chavasse C, Bates I. Impact of introducing integrated quality assessment for tuberculosis and malaria microscopy in Kano, Nigeria. J Infect Dev Ctries 2009; 3: 20-7 http://dx.doi.org/10.3855/jidc.101 pmid: 19749445.
- Klarkowski DB, Orozco JD. Microscopy quality control in Médecins Sans Frontières programs in resource-limited settings. PLoS Med 2010; 7: e1000206 http://dx.doi.org/10.1371/journal.pmed.1000206 pmid: 20126251.
- Thomson S, Lohmann RC, Crawford L, Dubash R, Richardson H. External quality assessment in the examination of blood films for malarial parasites within Ontario, Canada. Arch Pathol Lab Med 2000; 124: 57-60 pmid: 10629133.
- Muhindo HM, Ilombe G, Meya R, Mitashi PM, Kutekemeni A, Gasigwa D, et al., et al. Accuracy of malaria rapid diagnosis test Optimal-IT(®) in Kinshasa, the Democratic Republic of Congo. Malar J 2012; 11: 224 http://dx.doi.org/10.1186/1475-2875-11-224 pmid: 22770130.
- Mori M, Ravinetto R, Jacobs J. Quality of medical devices and in vitro diagnostics in resource-limited settings. Trop Med Int Health 2011; 16: 1439-49 http://dx.doi.org/10.1111/j.1365-3156.2011.02852.x pmid: 21955331.