Bulletin of the World Health Organization

Use of data from registered clinical trials to identify gaps in health research and development

Roderik F Viergever a, Robert F Terry b & Ghassan Karam c

a. Department of Primary and Community Care, Radboud University Nijmegen Medical Centre, Nijmegen, Netherlands.
b. Department of Public Health, Innovation, Intellectual Property and Trade, World Health Organization, 20 avenue Appia, Geneva 27, Switzerland.
c. Department of Ethics, Equity, Trade and Human Rights, World Health Organization, Geneva, Switzerland.

Correspondence to Robert F Terry (e-mail: terryr@who.int).

(Submitted: 23 October 2012 – Revised version received: 01 February 2013 – Accepted: 06 February 2013 – Published online: 03 April 2013.)

Bulletin of the World Health Organization 2013;91:416-425C. doi: http://dx.doi.org/10.2471/BLT.12.114454

Introduction

More than two decades ago it was shown that only 5% of the world’s resources for health research and development (R&D) were spent on the health problems of developing countries, which then represented 93% of the world’s burden of preventable mortality.1,2 The lack of a rational link between the health R&D that was needed and that which was being conducted resulted in the existence of “neglected populations”.3 This mismatch, which still exists, had and has two main causes. First, the distribution of R&D funding has been – and remains – largely determined by market forces rather than by a more equitable system that is based on health needs.4,5 Second, even when funding for health R&D is distributed by philanthropic or governmental donors, many high-burden diseases and priority areas of R&D can remain badly underfunded.6 This indicates a lack of appropriate mechanisms for the prioritization and coordination of such R&D.7 To start addressing these problems, a sense of agreement on a common R&D agenda will have to grow among funders of health R&D – something that, to date, has proven difficult to achieve.7 As a first step towards such a common agenda, the current composition of the “global landscape” of health R&D needs to be explored so that the gaps in this landscape and neglected populations can be identified. If we are to change how we spend our money on health R&D, we first need to know how we are spending it now.

Unfortunately, we know very little about what health R&D is being conducted, where and how it is being conducted, and who is conducting it.8 Databases of registered clinical trials may offer a new resource for gaining insight into the health R&D “landscape”. In the past decade, trial registration has become broadly accepted as an ethical and scientific responsibility.916 Enforcing regulations, policies and legislation has been crucial to the success of trial registration. There has been relevant national legislation,12 the editors of many medical journals have made trial registration a prerequisite for the publication of trial results,9,1315 such registration may also now be a prerequisite for the ethical approval of a trial’s protocol11,17 and a self-regulating pharmaceutical industry has also promoted trial registration.16 On several continents, many publicly accessible, online registries have been established to allow investigators to register their clinical trials.18 In 2005, the International Clinical Trials Registry Platform (ICTRP) was established by the World Health Organization (WHO) to create a platform for linking these clinical trial registries and provide a single point of access to information on all clinical trials conducted globally.11 Over the last 8 years, the ICTRP has grown into a platform that combines data from 15 different clinical-trial registries, both national and regional, and offers access to more than 200 000 registered records of clinical trials.

This study was conducted to explore what can be learnt from the clinical trial records available on the ICTRP database about the current composition of the “global landscape” of health R&D. We were especially interested in the distribution of trials across different diseases and countries and the identification of any major gaps in the “landscape”.

Methods

Study sample

By using an automated random sampling function that is available as part of the ICTRP’s data management system, we randomly selected from the ICTRP database 5% of all the records for interventional clinical trials that were registered as actively recruiting participants on 10 August 2012. A 5% sample was considered to be sufficient to produce results that could give a general view, but not too large to hamper the manual extraction of relevant data. For trials that were registered in more than one registry, we included only the record with the earliest registration date.19 We excluded trials that, according to the ICTRP’s records, were only observational in nature.

Data extraction

Registry name, date of registration, age and sex inclusion criteria, target sample size, study design, study type, study phase and the countries of recruitment for each record were downloaded from the ICTRP and imported into an Excel (Microsoft, Redmond, United States of America) database on 10 August 2012. We manually reviewed the health condition or problem studied, the intervention and the primary sponsor by examining the registered record, and we then coded the data as described in the next section.

Data coding and classifications

We coded the health conditions or problems studied in each selected trial according to table C3 of the Global burden of disease: 2004 update.20

We categorized the countries in which the subjects of trials were recruited as high-, upper-middle-, lower-middle- or low-income according to the World Bank’s groupings, which are based on gross national incomes per capita.21 We also identified the WHO region to which each country belonged using the current WHO classification of Member States.22 If a trial was recruiting participants in multiple countries that belonged to the same income group or same WHO region, we counted the group or region only once.

We divided primary sponsors (i.e. the individual, organization, group or other legal entity that was responsible for initiating, managing and/or financing a trial) into nine categories: collaborative groups of researchers or doctors; contract research organizations; foundations; government institutions; industries; individuals registered as sponsors; research institutes; universities or hospitals; and “other”. We then classified trials as having an industrial primary sponsor, a non-industrial primary sponsor (including collaborative groups, foundations, governments, research institutes and universities or hospitals) or another type of sponsor (including individuals registered as primary sponsors, contract research organizations and “other” sponsors).

All data were extracted and coded by one author (RFV) and, if ambiguous, discussed with another author (RFT).

Data analysis

For each health condition or problem studied and for each of the categories used for the countries of recruitment, the number of trials detected in the 5% sample was extrapolated to estimate the total number of actively recruiting, interventional trials with the same characteristic that were registered on the ICTRP. The Wilson score interval23 was used to calculate 95% confidence intervals for each estimate.

Whenever possible, for each health condition or problem studied, we mapped the estimated total number of related trials on the ICTRP against the corresponding burden of disease in disability-adjusted life years (DALYs).20,24 Additionally, we divided the estimated total number of related trials by the corresponding burden of disease in DALYs to give an estimate of the total number of trials per million DALYs for each health condition. Burden-of-disease data were not available for all of the health conditions that were being investigated in the selected trials.24 In addition, the subcauses of injuries were ignored in these calculations because the sources of the injuries were not included in the majority of the records pertaining to injuries. Among the health conditions and problems, we also excluded residual (“other”) categories, several overarching categories (i.e. skin disorders, endocrine disorders and “other neoplasms”) and a small number of specific diseases for which uncertainties in the burden-of-disease estimates were large (e.g. chlamydia, gonorrhoea, neonatal infections, polio, all congenital anomalies, all oral diseases and Chagas disease in low-income countries). Trials that recruited participants with malignant neoplasms in general were redistributed proportionally over all of the disease codes for such neoplasms, in a similar approach to that taken by the authors of the Global burden of disease: 2004 update.20

We expressed estimates of the numbers of trials in the ICTRP database that were recruiting in countries in each income group and WHO region as the numbers of trials per capita. For this, we estimated the sizes of the relevant national populations in the year 2012 using the World Bank’s database of health, nutrition and population statistics.25 For each income group and WHO region, we divided the number of trials per capita by the corresponding total burden of disease in DALYs per capita to obtain an estimate of the total number of trials per million DALYs for each category used for the countries of recruitment.

We derived all burden-of-disease data – which were standard DALYs with time discounting and age-weighting – from the most recently published results of WHO’s Global Burden of Disease study.20,24

We used Z-tests23 to compare the proportions of trials whose primary sponsor was industrial with the corresponding proportions of trials with non-industrial primary sponsors.

All of the data analysis was conducted using the Excel software package.

Results

On 10 August 2012, 2381 clinical trials that were registered as interventional and actively recruiting were randomly selected from the ICTRP database (Fig. 1). Baseline information on registry name, intervention type, year of registration, sponsorship, target sample size, study phase and inclusion criteria for sex and age of participants is presented in Table 1.

Fig. 1. Flowchart of the sampling of the records of interventional and actively recruiting trials in the International Clinical Trials Registry Platform (ICTRP), 2012
Fig. 1. Flowchart of the sampling of the records of interventional and actively recruiting trials in the International Clinical Trials Registry Platform (ICTRP), 2012

Health conditions or problems studied

The health condition or problem studied could be classified for 2195 of the 2381 selected trials. The most common focus of investigation – both in terms of the absolute number of trials and the number of trials per million DALYs caused by the condition or problem – was on noncommunicable diseases (52.4), followed first by communicable, maternal, perinatal and nutritional conditions (7.4) and then by injuries (6.0) (Table 2, available at: http://www.who.int/bulletin/volumes/91/6/12-114454, and Fig. 2). The estimated total number of trials registered on the ICTRP for each health condition or problem was mapped against the global burden of the condition or problem (Fig. 3).

Fig. 2. Health problems being investigated by trials registered in the International Clinical Trials Registry Platform (ICTRP), 2012
Fig. 2. Health problems being investigated by trials registered in the International Clinical Trials Registry Platform (ICTRP), 2012
DALY, disability-adjusted life year.
Note: Only interventional and actively recruiting trials were investigated. The health problems are split according to both the estimated numbers of trials on the ICTRP (lefthand chart) and the burden of disease that they cause globally (righthand chart). Confidence intervals were calculated for the estimates but have been omitted from the figure, for clarity.
Fig. 3. Estimated number of trials in the International Clinical Trials Registry Platform investigating a specific health problem and the burden of disease posed by that problem, 2012
Fig. 3. Estimated number of trials in the International Clinical Trials Registry Platform investigating a specific health problem and the burden of disease posed by that problem, 2012
DALY, disability-adjusted life year; AIDS, acquired immunodeficiency syndrome; HIV, human immunodeficiency virus.
Note: Only interventional and actively recruiting trials were included in the analysis. Data in the grey area is for health problems that have 400 or fewer registered trials and a global burden of disease of less than 20 million. A full list of registered trials by health problem is shown in Table 2 (available at: http://www.who.int/bulletin/volumes/91/6/12-114454). Only trials investigating specific health problems were included in this figure; overarching categories and subcategories of health problems were excluded. Confidence intervals were calculated for the estimates but have been omitted from the figure, for clarity.

Countries of recruitment and sponsorship

Information on countries of recruitment was available for 2377 of the 2381 selected trials. Trials were found to recruit most often in high-income countries – absolutely, per capita and proportionally to the burden of disease in these countries – followed first by upper-middle-income countries, then by lower-middle-income countries and finally by low-income countries (Table 3 and Fig. 4). Trials recruited most often were in WHO’s European Region and the Region of the Americas (Table 3 and Fig. 5).

Fig. 4. Estimated numbers of trials in the International Clinical Trials Registry Platform recruiting participants in low-, lower-middle-, upper-middle- and high-income countries, 2012
Fig. 4. Estimated numbers of trials in the International Clinical Trials Registry Platform recruiting participants in low-, lower-middle-, upper-middle- and high-income countries, 2012
DALY, disability-adjusted life year.
Note: Only interventional and actively recruiting trials were included in the analysis. For illustration, the burdens of disease in countries in the same income groups are also presented. The error bars on the estimates of trial numbers indicate 95% confidence intervals.
Fig. 5. Estimated numbers of trials in the International Clinical Trials Registry Platform recruiting in each of WHO’s regions, 2012
Fig. 5. Estimated numbers of trials in the International Clinical Trials Registry Platform recruiting in each of WHO’s regions, 2012
DALY, disability-adjusted life year; WHO, World Health Organization.
Note: Only interventional and actively recruiting trials were included in the analysis. For illustration, the burdens of disease in countries in the same regions are also presented. The error bars on the estimates of trial numbers indicate 95% confidence intervals.

We were able to determine country of recruitment and classify the primary sponsor as non-industrial or industrial for 2253 of the 2381 selected trials. Trials with non-industrial primary sponsors recruited more often in low-income countries than trials with industrial primary sponsors (odds ratio, OR: ∞; Z = 2.0; P = 0.0464), whereas trials with industrial primary sponsors recruited more often in lower-middle-income (OR: 4.0; Z = 7.2; P < 0.0001), upper-middle-income (OR: 2.0; Z = 5.0; P < 0.0001) and high-income countries (OR: 2.2; Z = 4.0; P = 0.0001) (Table 4). Trials with industrial primary sponsors were more likely to have multi-country recruitment [222 (44.8%) of 495] than trials with non-industrial primary sponsors [73 (4.1%) of 1758] (OR: 18.8; Z = 23.7; P < 0.0001).

Discussion

The global monitoring of health R&D requires analyses of the inputs (e.g. investments),2,5,6 processes (e.g. analyses of the R&D “pipeline”)26,27 and outputs (e.g. publications28 or products such as medicines)4 of R&D. Such “triangulation” of different sources of information is essential if we are to obtain a complete picture of what health R&D is being conducted, where and how it is being conducted, and who is conducting it. The increasing public availability of information on clinical trials provides an additional source of information for analysing current processes in health R&D at global, regional or country levels. Evaluations of registered trial data have recently been used to shed light on national clinical trial portfolios29,30 and specific research areas.3134 This type of evaluation has several strengths: all trials should be registered, even if their final results are never published; registered records contain information that is complementary to that in any published articles on the trials;35 databases of registered trials can provide insight into currently ongoing R&D; and their standardized and searchable format makes databases of registered trials suitable for aggregate analysis.36 For the purpose of obtaining a comprehensive global picture of all ongoing clinical trials, the ICTRP is an unmatched resource of information since it provides access to data from all of the major clinical trial registries around the world that meet the relevant standards of WHO’s registry criteria.37

The results of this study show that, at least on a global scale, there is little correlation between the burden of disease attributable to a particular health condition or problem and the amount of clinical trial research being conducted on that health problem. This finding confirms the mismatch – between health R&D need and relevant health R&D – that has previously been observed using alternative R&D metrics, such as R&D investments and R&D outputs.14,6,33,38 A consequence of this mismatch is the existence of several populations that are neglected with respect to health R&D.3 In particular, health R&D currently does not adequately meet the needs of populations in lower-income countries.3,39 In general, communicable, maternal, perinatal and nutritional conditions – which cause a much higher proportion of the burden of disease in lower-income countries than in high-income countries20 – currently receive much less attention, in terms of clinical trial research, than noncommunicable diseases. In addition, clinical trials recruit much less often in lower-income countries than in higher-income countries. For health conditions or problems that cause a large burden in both lower- and higher-income countries, it is important that populations in lower-income countries be included in clinical trial research so that their specific R&D needs can be addressed.3

There are several limitations in using registered trial data for identifying gaps in the health R&D “landscape”. No account is taken of research other than that conducted within the context of a clinical trial. Since a registry for systematic reviews has recently been established40 and the creation of a registry for observational research has been widely advocated,41,42 evaluations of the health R&D “landscape” may soon broaden in scope. Another potential data source could be a registry (or database) of research protocols or even raw datasets43, although the information in such a registry would be much more difficult to analyse than the registered records of clinical trials.

The need for clinical trial research on a given health problem – or the perceived need for such research – is only partly determined by the burden of disease posed by the problem. The severity of the corresponding product shortfall, the state of the relevant science and technology and disease trends can also affect the need for clinical trial research.6,44 In other words, the need for R&D will be relatively high for diseases for which effective product development has been scant and for emerging diseases, diseases posing increasing burdens and diseases on course for eradication, whereas clinical trials may be considered premature if basic science is lacking in new research areas. Caution is therefore warranted in interpreting the correlation – or lack of correlation – between the number of clinical trials conducted on a particular disease and the burden posed by that disease. The main strength of the findings of the present study lies in the general, global trends that the findings reveal. For more specific conclusions about individual diseases, registered trial data will have to be analysed alongside other sources of information.

To date, very little reliable information has been produced on how much clinical trial research is being conducted in lower-income countries.45 Although the present results help to fill this knowledge gap, it is important to note that the registration of trials has not been enforced equally around the world. Many countries still have no legislation to enforce registration12 and not all journals in which clinical-trial data could be published are covered by the journal associations that have committed to enforcing trial registration.9,13 Furthermore, not all clinical trials are conducted with the goal of publication. It is difficult to verify or even estimate how many clinical trials remain unregistered, although it seems likely that at least some trials are never registered, especially in countries where there is no legal requirement for registration.30,46,47 Given that all major medical journals now require evidence of trial registration, as a condition for publication of any data from a trial, and that all studies that assess the effects of new medicines – for which regulatory approval is to be sought internationally – need to be registered, the quality and potential impact of any unregistered trials are questionable. Nonetheless, it is crucial that clinical-trial registration is enforced in every country, by means of national legislation and/or by ethical review boards, to ensure that a complete picture of the global distribution of clinical-trial research can be obtained.11,12,48

Before full use can be made of the ICTRP for exploring the health R&D “landscape”, several other limitations need to be addressed. First, even in those countries that have legislation on the registration of clinical trials, enforced registration is often limited to trials of drugs and – sometimes – devices, phase II–IV trials, and trials that recruit subjects in the country where the legislation is implemented.49 This problem has been recognized in the United States of America, where new legislation to ensure that all clinical trials of interventions are registered has been proposed.50 There also remain concerns about the quality of the data entered into the registered records of clinical trials10,51,52 and about problems with the unique identification of trials, which can lead to duplicate registration.19

Finally, the extraction, aggregation and analysis of the data in the ICTRP database currently require substantial manual labour. The formats of some of the data items differ across the registries covered by the ICTRP, which makes the automated aggregate analysis of data impossible. To remedy this limitation, the staff of the ICTRP are working with individual registries to harmonize the data recording formats across all of the registries that are covered by the platform. An alternative solution would be the development of algorithms to translate the variable information from individual registries into a common format and then classify the information into meaningful categories. ClinicalTrials.gov, one of the registries that provide data to the ICTRP, has already shown that the development of such data classification algorithms is feasible.29,53 Developing similar aggregation algorithms for the ICTRP – and making both the aggregated data and the results of the analysis of those data publicly available – would be an important step forward not only for the ICTRP but also for clinical trial transparency on a global scale.29

In conclusion, this study shows that WHO’s ICTRP constitutes a valuable resource for assessing the global distribution of clinical trials and for informing policy development and priority setting for health R&D. The findings of this study demonstrate that there is little correlation between burden of disease and the global distribution of clinical trial research and that populations in lower-income countries receive much less attention, in terms of clinical trial research, than populations in high-income countries. A more detailed understanding of the global health R&D “landscape” is needed to inform future R&D priorities. The ICTRP is one of several resources of information that will need to be “triangulated” to acquire a complete picture of what health R&D is being conducted, where and how it is being conducted, and who is conducting it. The ICTRP would constitute an essential part of any global observatory on health R&D.39 To increase the usefulness of the ICTRP further, it is important that the enforcement of clinical trial registration be increased, that the quality of the data in registered records be improved and that more possibilities for automated aggregate data analysis on the ICTRP be created.


Acknowledgements

We thank Colin Mathers from the World Health Organization for his help in collecting the burden-of-disease data used for this study. RFV has a dual appointment with the Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine, London, England.

Competing interests:

None declared.

References

Share