Bulletin of the World Health Organization

The impact of new tuberculosis diagnostics on transmission: why context matters

Hsien-Ho Lin a, David Dowdy b, Christopher Dye c, Megan Murray d & Ted Cohen e

a. Institute of Epidemiology and Preventive Medicine, National Taiwan University, 17 Xuzhou Road, 100 Taipei, Taiwan, China.
b. Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, United States of America (USA).
c. HIV/AIDS, Tuberculosis, Malaria and Neglected Tropical Diseases Cluster, World Health Organization, Geneva, Switzerland.
d. Department of Epidemiology, Harvard School of Public Health, Boston, USA.
e. Division of Global Health Equity, Brigham and Women's Hospital, Boston, USA.

Correspondence to Hsien-Ho Lin (e-mail: hsienho@ntu.edu.tw).

(Submitted: 18 December 2011 – Revised version received: 29 June 2012 – Accepted: 02 July 2012 – Published online: 16 July 2012.)

Bulletin of the World Health Organization 2012;90:739-747A. doi: 10.2471/BLT.11.101436

Introduction

New tools for the diagnosis of tuberculosis have been developed over the past 10 years. All of them offer improved diagnostic accuracy over traditional sputum smear microscopy and several promise to detect the most diagnostically challenging types of tuberculosis, including multidrug-resistant tuberculosis and tuberculosis in patients with human immunodeficiency virus (HIV) infection.13 One such tool, the Xpert MTB/RIF assay, is an automated nucleic acid amplification test that has greater than 70% sensitivity in detecting smear-negative, culture-positive tuberculosis and can be used nearer to the point of care by individuals with little training.4,5 Other new tools, including fluorescent microscopy, loop-mediated isothermal amplification assays and line probe assays, also have promising applications.68 Evidence on the accuracy of these new diagnostics has rapidly accumulated from clinical validation and field demonstration studies. These data have been used to support the implementation and rollout of new diagnostic tools in several low-income countries with the hope of reducing the burden of tuberculosis and providing cost-effective screening options for policy-makers.3

For a novel diagnostic test to have population-level impact on the control of tuberculosis, improved accuracy is necessary but not sufficient. Tuberculosis transmission depends directly on the length of time patients remain infectious before they are treated. A diagnostic tool that improves detection only late in the course of disease may prevent death, but its impact on transmission will be small. The expected epidemiological impact of implementing a new diagnostic test will also vary with the rates of case detection and treatment success in the existing tuberculosis control system. If deployed in a clinic where tuberculosis cases are already being detected with high sensitivity (e.g. through the use of high-quality diagnostics), the new tool will have a much lower incremental impact than if it is deployed in a clinic where tuberculosis is frequently missed. Similarly, if the diagnosis of tuberculosis is not followed by prompt and effective treatment, the impact of a new diagnostic test will be small.

For tuberculosis control to be effective, every patient must navigate a complex diagnostic pathway.9,10 This pathway, which begins only when an individual presents at a local health centre, involves a series of steps: a clinician suspects tuberculosis, orders the appropriate diagnostic test, receives the results of this test, makes the appropriate diagnosis, places the patient on an effective treatment regimen and ensures the completion of therapy. The accuracy of a diagnostic test affects this pathway at a critical moment, but the loss of patients at any step along the diagnostic pathway, which can be substantial, limits the epidemiologic impact of any new diagnostic tool.9

We hypothesized that several contextual factors will affect the capacity of a new diagnostic tool to reduce the burden of tuberculosis. To evaluate and quantify the relative influence of these factors on the potential epidemiologic impact of a new diagnostic test, we developed a mathematical model of a tuberculosis epidemic that captures the complex patient pathway involved in tuberculosis diagnosis and treatment.

Methods

Using an age-structured epidemic model of tuberculosis transmission, we compared the potential impact of the existing diagnostic algorithms using smear microscopy with that of a hypothetical new tool that replaced smear microscopy in the algorithm. The existing standard approach requires performing three initial sputum smear examinations in people suspected of having pulmonary tuberculosis. For those whose smear is negative, the diagnosis is based on a chest X-ray, a trial of broad-spectrum antibiotics and clinical judgment. The new tool was assumed to have a sensitivity of 100% for highly infectious cases (defined as smear-positive under ideal laboratory conditions) and of 70% for less infectious cases (defined as smear-negative).4,5 We constructed the model to reflect an existing diagnostic algorithm to make it possible to assess the influence of operational factors on the epidemiologic impact of the new tool. The operational factors considered in the analysis were: (i) access to tuberculosis care, defined by the proportion of the population able to enter the health-care system upon contracting tuberculosis; (ii) duration of patient delay, defined as the time from the onset of symptoms until entry into the health-care system; (iii) diagnostic default, defined as the loss of a patient to follow-up before completion of the diagnostic procedure; (iv) sensitivity of the secondary diagnostic tool (if the initial test result is negative); (v) initial default, defined as the loss of a patient to follow-up after being informed of a positive test result but before treatment initiation; (vi) treatment success, defined by treatment completion and cure.

Expanded epidemic transmission model

Ours is a compartmental model that follows conventions adopted for previous epidemic models of major anti-tuberculosis interventions11,12 and prior models of tuberculosis diagnostics.1316 The model divides the population into compartments based on tuberculosis disease status (Fig. 1), HIV status (positive/negative) and age. Since the diagnostic pathway begins after the onset of disease, we further expanded disease states to incorporate important steps in the diagnostic pathway (Fig. 1). This expanded model enables detailed study of the diagnostic process in the context of a transmission modelling framework. It makes it possible to explicitly investigate the impact of increased diagnostic sensitivity as one step in this pathway (i.e. by changing the relative size of flows to “test+” versus “test−:” f2 in Fig. 1 and Fig. A1 in Appendix A, available at: https://www.dropbox.com/s/ts9yt1uop8006wv/Appendix%20A.docx.zip), as well as the relative impact of other factors along the diagnostic pathway (e.g. reduced loss to follow-up resulting from more rapid diagnosis: f1 in Fig. 1 and Fig. A1 in Appendix A).

Fig. 1. Graphical representation of the expanded epidemic model used to study the impact of new tuberculosis diagnostics on transmission
Fig. 1. Graphical representation of the expanded epidemic model used to study the impact of new tuberculosis diagnostics on transmission
f1, relative impact of other factors along the diagnostic pathway; f2, impact of increased diagnostic sensitivity as one step in this pathway.Note: Part A shows a structure similar to that of previous tuberculosis transmission models. This figure does not show additional model stratification based on age and human immunodeficiency virus infection status. Part B shows expansion of the simple standard tuberculosis transmission model, through addition of more patient diagnostic pathways, to investigate the effect of a diagnostic tool within the operational context. See Appendix A for a detailed description of model compartments and rate transitions between compartments.

We used a Bayesian procedure to calibrate the expanded model to the observed epidemiological characteristics of tuberculosis and HIV infection in the United Republic of Tanzania, a country with a high burden of both diseases and a relatively low prevalence of multidrug-resistant tuberculosis.1719 In brief, we performed 100 000 simulations with parameter values selected randomly from prior distributions of input parameters. A likelihood score was computed for each resulting simulation based on the fit to tuberculosis incidence estimated by the World Health Organization. These 100 000 simulations were resampled (with replacement) 100 000 times, with the probability of selection proportional to the likelihood of each curve. The posterior mode curve (i.e. the curve that was resampled most frequently) was deemed as the best fitting curve after calibration. We present this modal curve in the main analysis (Fig. 2 and Fig. 3). Key model parameters are given in Table 1 (available at: http://www.who.int/bulletin/volumes/90/10/11-101436) and model assumptions are listed in Box 1. Appendix A describes in detail the model structure, the prior and posterior ranges of all model parameters and the calibration procedure.

Fig. 2. Incidence of tuberculosis (all forms) in the United Republic of Tanzania based on WHO estimates and projected incidence based on the calibrated epidemic model
Fig. 2. Incidence of tuberculosis (all forms) in the United Republic of Tanzania based on WHO estimates and projected incidence based on the calibrated epidemic model
WHO, World Health Organization. Note: The green solid line and green dashed lines represent the point estimate and 95% confidence limits of tuberculosis incidence from 1997–2010 as reported by WHO. The black solid line is the most frequently observed trajectory (i.e. the mode) from 100 000 resamples of simulated tuberculosis epidemic curves after calibrating to WHO estimates; this line represents the model output most consistent with the observed trend in tuberculosis incidence. The grey lines represent 1000 random samples from the 100 000 resamples of calibrated tuberculosis epidemic curves. These modelled trajectories (black and grey lines) suggest that the calibration procedure captured the main features of tuberculosis epidemiology in the United Republic of Tanzania.
Fig. 3. Projected incidence, prevalence and mortality trends for pulmonary tuberculosis and annual risk of latent tuberculosis under three diagnostic scenariosa
Fig. 3. Projected incidence, prevalence and mortality trends for pulmonary tuberculosis and annual risk of latent tuberculosis under three diagnostic scenarios<sup>a</sup>
a Scenario I: sputum smear microscopy under the reference case operational context (corresponding to the mode values of operational parameters in Table 2).b Scenario II: use of the new tool (with 70% sensitivity for smear-negative disease) to replace sputum smear microscopy in Scenario I.c Scenario III: use of the new tool in combination with other interventions that shorten the average patient delay and increase access to care and treatment success rate (with parameter values corresponding to the 90% posterior limits in Table 2).

Box 1. Model assumptions

i. Sputum smear as initial baseline test is replaced by novel diagnostic.

ii. There is a single diagnostic attempt during the patient's disease course with no repeat diagnostic attempts.

iii. There is no multidrug-resistant tuberculosis.

iv. The incidence of HIV infection in the target country stays at the 2009 level during the period of analysis.

v. Individuals show homogeneous mixing in the population.

Outcome measures and analytic methods

Using the calibrated model for the United Republic of Tanzania, we projected the impact of the new, more sensitive tool over that of the existing diagnostic algorithm using smear microscopy, under the assumption that the incidence of HIV infection would remain the same as in 2009. We estimated the average annual decline in pulmonary tuberculosis incidence, prevalence and mortality, and in the annual risk of tuberculosis infection (ARTI) over the ensuing 10 years under the base case scenario of smear microscopy and under the alternative scenario, namely, use of the new diagnostic tool in 2010. We quantified the impact of the new tool by calculating the absolute difference in the annual rate of decline in pulmonary tuberculosis incidence under the base case scenario and the alternative scenario. We also investigated the impact of the new tool on the duration of infectiousness, which directly affects the rate of disease transmission.

To investigate the influence of operational factors on the epidemiologic impact of the new tool, we conducted one-way sensitivity analyses by changing one operational parameter at a time and computing the absolute difference between the average annual rate of decline in pulmonary tuberculosis incidence observed with the new diagnostic tool and that observed with sputum smear microscopy. The ranges of variation of operational parameters were based on the 90% posterior intervals from the 100 000 resamples after Bayesian calibration (Table 2). We also investigated the independent influence of each operational factor on the impact of the new tool by computing the partial rank correlation coefficient from 1000 simulations from the joint prior distribution of all input parameters. The partial rank correlation coefficient measures the correlation between an input parameter and the impact of a new tool while adjusting for the influence of all other input parameters.27 We conducted uncertainty analysis on the absolute difference in the annual rate of decline in pulmonary tuberculosis incidence using 1000 random samples from the 100 000 posterior resamples. The 95% credible intervals came from the 25th and the 975th values among the 1000 simulations. We used Berkeley Madonna version 8.3.18 (University of California at Berkeley, Berkeley, United States of America) for model simulations and R version 2.11.1 (The R Foundation for Statistical Computing, Vienna, Austria) for statistical analysis.28,29

Results

Under the base case scenario of smear microscopy, the incidence of pulmonary tuberculosis would continue to decline in the United Republic of Tanzania because of sustained case detection and treatment. Over a 10-year horizon, pulmonary tuberculosis incidence, prevalence and mortality and ARTI would decline by an annual average of 3.94%, 3.31%, 4.41%, and 3.11%, respectively (Fig. 3). We projected that with use of the new diagnostic tool the burden of pulmonary tuberculosis would decline faster than with continued use of smear microscopy. Under a reference case operational context (corresponding to the modal values of the operational parameters in Table 2), the projected average annual declines in pulmonary tuberculosis incidence, prevalence and mortality and in ARTI would be 4.25%, 5.07%, 5.27% and 3.89%, respectively (Fig. 3). Thus, the new tool is associated with an additional average annual decline in the incidence of pulmonary tuberculosis of 0.31 percentage points (0.04–0.42), and with additional average annual declines in prevalence, mortality and ARTI of 1.77 percentage points (95% credible interval, CI: 0.33–2.30), 0.86 percentage points (95% CI: 0.09–1.97) and 0.78 percentage points (95% CI: 0.12–1.07), respectively. Owing to improved sensitivity and early case detection, the new tool reduces the period of infectiousness from 10.4 months to 9.1 months on average.

One-way sensitivity analyses suggested that the additional annual decline in the incidence of pulmonary tuberculosis is strongly influenced by the sensitivity of the existing diagnostic strategies used to retest those whose initial test results are negative (Fig. 4). When the existing diagnostic strategy was highly sensitive for smear-negative cases (e.g. X-ray or clinical suspicion), the improvement offered by the new, more accurate diagnostic tool was less impressive. A long delay in seeking medical attention after the onset of symptoms also reduced the epidemiologic impact of the new diagnostic tool (Fig. 4) because of prolonged transmissibility. Variations in other operational factors (e.g. level of access to tuberculosis diagnosis and treatment, rate of diagnostic default among patients testing positive, rate of initial (i.e. post-diagnostic) default and treatment success rate) also affected the impact of the new tool on tuberculosis incidence in the direction anticipated (Fig. 4). The results of the partial rank correlation coefficient analysis were consistent with those of the one-way sensitivity analysis (Fig. 4).

Fig. 4. Sensitivity analysis on the influence of operational factors on the impact of a sensitivity diagnostic tool on annual decline in pulmonary tuberculosis incidence
Fig. 4. Sensitivity analysis on the influence of operational factors on the impact of a sensitivity diagnostic tool on annual decline in pulmonary tuberculosis incidence
a Access to and sensitivity of existing diagnostic strategies for smear-negative cases.b Lost to follow-up before completion of diagnostic process.Note: Part A shows the results of the one-way sensitivity analysis of additional annual decline in pulmonary tuberculosis incidence using the values of operational factors from posterior distributions after calibration. The vertical line corresponds to the estimate using the mode values in Table 2, and “High” and “Low” correspond to the estimates using the high and low values in Table 2.Part B shows the partial rank correlation coefficients obtained with 1000 random simulations from joint prior distribution of all input parameters. Coefficients of the top 10 influential input parameters and those of all other operational parameters are presented.

Following the one-way sensitivity analysis, we attempted to identify the types of operational settings in which the diagnostic tool was expected to deliver the greatest average annual decline in the incidence of pulmonary tuberculosis. To achieve this we sequentially tuned the operational parameters to the values that yielded the greatest impact on disease incidence, beginning with the parameter shown to exert the greatest influence in the one-way sensitivity analysis. We found that the greatest additional annual decline in the incidence of pulmonary tuberculosis (0.57%) would be achieved if the new diagnostic tool were used in settings that fulfilled all of the following conditions: prompt care seeking after symptom onset, an existing diagnostic strategy with poor sensitivity and full (99%) access to tuberculosis care (with parameter values corresponding to the 90% posterior limits in Table 2).

In addition to assessing the impact of a new, more sensitive diagnostic tool, we conducted iso-effect analyses to identify how a similar incremental benefit could be attained with other interventions. To this end we increased the standard diagnostic strategy’s sensitivity for smear-negative cases from 0% to 70%. Under the best-fitting condition of the calibrated model, it would require any of the following to obtain the same effects as those observed with the new diagnostic tool: shortening the time to care seeking after symptom onset from 2 to 1.2 months; increasing access to tuberculosis care from 83% to 88% of cases; increasing the treatment success rate from 70% to 87%, or decreasing the fraction of initial (i.e. post-diagnostic) defaults from 23% to 16%. We further investigated the impact of joint interventions that included the following components: (i) a new diagnostic tool with 70% sensitivity for smear-negative cases; (ii) a strategy for shortening care-seeking delay after symptom onset to between1.3 and 1.7 months; (iii) a strategy for increasing access to tuberculosis care to 99%; (iv) a strategy for increasing the treatment success rate to 96% (Table 2). As expected, the combined intervention would have a greater impact on annual decline than any single intervention (Fig. 3).

Discussion

Using an expanded tuberculosis epidemic model that identifies additional stages along a diagnostic pathway, we found the impact of a new, more accurate diagnostic tool on tuberculosis epidemiology to be substantially affected by contextual factors unrelated to tool performance. In particular, the epidemiologic impact of such a tool is greatest in settings where access to tuberculosis care is good (i.e. where symptomatic tuberculosis patients who seek care are likely to obtain it) but existing diagnostic strategies have poor sensitivity (e.g. poor access to chest X-ray for smear-negative cases). Since this may be the scenario in many community-level health centres, it is important to develop diagnostic tests that can be deployed at this level. Furthermore, new diagnostics will have a lesser population-level impact than expected if they are implemented in poorly accessible reference laboratories already equipped with sensitive tools (e.g. culture).

Our projections urge caution, because diagnostic pathways are fraught with inevitable inefficiencies that can make the incremental impact of new diagnostic technologies lower than anticipated. We specifically found that a diagnostic test with 100% sensitivity for smear-positive tuberculosis and 70% sensitivity for smear-negative tuberculosis would reduce the average annual incidence of pulmonary tuberculosis by a mere 0.31 percentage points in the United Republic of Tanzania. Our findings strongly indicate the need for a comprehensive approach that emphasizes the importance of the operational context of diagnosis and treatment when deciding how to implement and scale up a new diagnostic tool.

Our model illustrates the critical importance played by the duration of infectiousness in tuberculosis transmission. The level of tuberculosis transmission that occurs before patients present to the health-care system varies widely between settings,30 but it limits the degree to which a new, improved diagnostic test deployed in a clinic-based setting (i.e. without concomitant active case-finding efforts) can reduce tuberculosis incidence. The potential capacity of the new tool to reduce incident disease is eroded when patients delay in seeking care. Furthermore, symptom onset may not be a good indicator of the onset of infectiousness. Prevalence studies have shown that as many as two thirds of previously undiagnosed tuberculosis patients reported no symptoms31 and that sputum culture results can be positive in asymptomatic patients.32 Thus, decisions to scale up new tuberculosis diagnostics should take into account the results of studies, critically needed, on the fraction of secondary infections that occur before symptomatic patients seek health care and on the impact of the new diagnostics on the time between symptom onset and health care seeking and between initial contact with the health-care system and initiation of tuberculosis therapy.

The epidemiologic impact of a new diagnostic tool will also depend on the accessibility and sensitivity of the existing strategy for diagnosing sputum-smear-negative cases. Few studies have reported on the sensitivity of the strategies commonly employed to diagnose tuberculosis in sputum-smear-negative cases, such as chest X-ray or empirical treatment with antibiotics.33 Even fewer data are available on access to and use of these strategies when indicated and on the sensitivity of clinical suspicion for the diagnosis of sputum-smear-negative pulmonary tuberculosis. The importance of operational factors (Fig. 4) points to the need to better understand the organization of the health system and to accurately measure each step of the diagnostic pathway when projecting the overall impact of a new diagnostic.9,34,35

We modelled a new diagnostic tool with improved sensitivity for illustrative purposes only. Introducing such a tool could affect the diagnostic process in several ways. For example, a new test could increase patients’ faith in the health-care system and lower physicians’ threshold for considering a diagnosis of tuberculosis, which would result in reduced patient and health system delays. A new test with a short turnaround time will reduce the need for multiple visits to the health system and thereby decrease diagnostic default. For example, the result from the new Xpert MTB/RIF assay can be obtained within two hours. Thus, patients can get their test results on the same day. These additional effects of implementing a new tuberculosis diagnostic can have a much greater population impact than simply improving diagnostic sensitivity. Indeed, one could consider the “combined intervention” in our model (Fig. 3) to be a representation of the potential impact of a diagnostic test that not only has greater sensitivity, but that also reduces patient delays and increases access to care through improvements in provider behaviour. This highlights the importance of collecting information on changes in health system practices and patient behaviour following the implementation of a new diagnostic tool. An impact assessment framework that comprehensively evaluates the impact of a new diagnostic at the patient level, the health system level and the population level will serve as a useful guiding tool for policy-makers.35,36

Our epidemic model follows the standard epidemic model of tuberculosis transmission to evaluate the impact of new diagnostics on disease transmission. Unlike most previous epidemic models, ours includes the patient diagnostic pathway and the operational components of the diagnostic process. Thus, it allows the impact of a new tool to be explicitly examined in relation to other potentially important operational factors. Although our statistical approach accounts for uncertainty in parameter values, it does not account for uncertainty in model structure. For example, in our model, we classify tuberculosis patients as either highly infectious (sputum-smear-positive) or less infectious (sputum-smear-negative) and assume that the new tool has a fixed sensitivity within each state. If, however, the degree of infectiousness is a continuum and the smear-negative category includes some patients who are more infectious than others within the same category, our model could underestimate the impact of the new tool. We also did not vary the diagnostic specificity of the new tool in our analysis. A tool with poor specificity, even if highly sensitive, could undermine the system by overloading facilities with individuals who do not require anti-tuberculosis treatment. A formal cost-effectiveness analysis that takes into account various aspects of impact and the associated costs of a new tool would be valuable to policy-makers.9,36

In conclusion, our model-based quantitative analysis suggests that novel diagnostic tests can have substantial population-level impact on tuberculosis epidemics, but only in certain contexts or in combination with interventions geared towards promoting earlier health-care seeking after symptom onset to reduce the length of the infectious period. It is therefore critical to consider contextual factors such as the sensitivity of existing tests, the degree of access to tuberculosis care and the average time that transpires before care-seeking, in determining when and where novel diagnostics should be scaled up. Although we focused our analysis on diagnostics for tuberculosis screening, the central notion – that implementing diagnostic tools with impressive operating characteristics (e.g. high sensitivity and specificity) does not automatically translate into a large epidemiological impact – should also apply in the case of new, improved tools for diagnosing other infectious diseases.


Acknowledgements

We thank Chen-Yuan Chiang, Donald Enarson, Ivor Langley, Gillian Mann and Bertie Squire for valuable discussion during the conduct of the study. We thank Kimberly Powers for technical assistance on model simulations.

Funding:

This research was supported, in part, by a United States Agency for International Development (USAID) Cooperative Agreement (TREAT TB – Agreement No. GHN-A-00-08-00004-00). The contents are the responsibility of the authors and do not necessarily reflect the views of USAID.

Competing interests:

None declared.

References

Share