Bulletin of the World Health Organization

Challenges in developing national HIV guidelines: experience from the eastern Mediterranean

Anja De Weggheleire a, Veronique Bortolotti b, Maria Zolfo a, Siobhan Crowley c, Robert Colebunders a, Gabriele Riedner b & Lutgarde Lynen a

a. Department of Clinical Sciences, Institute of Tropical Medicine, Nationalestraat 155, 2000 Antwerp, Belgium.
b. World Health Organization, Eastern Mediterranean Regional Office, Cairo, Egypt.
c. United Nations Children’s Fund, Pretoria, South Africa.

Correspondence to Lutgarde Lynen (e-mail: llynen@itg.be).

(Submitted: 19 October 2010 – Revised version received: 25 March 2011 – Accepted: 28 March 2011 – Published online: 29 April 2011.)

Bulletin of the World Health Organization 2011;89:442-450. doi: 10.2471/BLT.10.083790

Introduction

The clinical management of human immunodeficiency virus (HIV) infection is evolving rapidly and it is a challenge to ensure that clinical practice guidelines remain up to date, valid and evidence-based. Yet, this is a prerequisite for high-quality care.1,2

Since 2003, the World Health Organization (WHO) has developed and regularly revised global recommendations for postexposure HIV prophylaxis, the prevention of mother-to-child HIV transmission and antiretroviral therapy (ART) for adults, adolescents and children.36 In individual countries, the effective implementation of HIV clinical services relies on national acquired immunodeficiency syndrome (AIDS) programmes developing and updating national guidelines, usually by either adopting or adapting international reference guidelines or by de novo guideline development. These strategies must employ rigorous methodologies to ensure the validity of the resulting guidelines.79

While assisting countries in the WHO Eastern Mediterranean Region with national HIV guideline development, WHO regional officers observed that national guideline development groups found it difficult to adopt a systematic approach. Consequently, they felt that the resulting national guidelines were suboptimal and, moreover, that the lack of a systematic approach hampered the process of external review by WHO regional officers. Similar observations have been made previously about local guideline adaptations in other medical fields.10,11

In response, the WHO Regional Office for the Eastern Mediterranean, in collaboration with the United Nations Children’s Fund (UNICEF) Middle East and North Africa Regional Office, proposed that national AIDS programmes in the region should organize a review of all HIV clinical practice guidelines developed locally. The purpose was threefold: to gain insight into the challenges of national HIV clinical guideline development and adaptation; to appraise the guideline development methodologies used and the concordance of the guidelines’ content with WHO 2006 and 2007 global recommendations;36 and to enhance regional capacity for sound guideline development and appraisal. The review was not intended to evaluate guideline implementation at the service delivery level.

This paper reports the main findings of the review of national HIV clinical guidelines from countries in the eastern Mediterranean.

Methods

Guideline review process

The WHO Regional Office for the Eastern Mediterranean proposed a review of HIV clinical practice guidelines to members of national AIDS programmes in the region and asked an academic partner, the Institute of Tropical Medicine in Antwerp, Belgium, to guide the process. In October 2008, the review’s objectives and modalities were discussed with a group of regional and international HIV experts, including representatives of the WHO Regional Office for the Eastern Mediterranean, the UNICEF Middle East and North Africa Regional Office, WHO headquarters and the Institute of Tropical Medicine in Antwerp. By May 2009, the national AIDS programmes of all 23 countries in either the WHO Eastern Mediterranean Region or the UNICEF Middle East and North Africa Region (Box 1) had been invited to submit their HIV clinical practice guidelines on the prevention of mother-to-child transmission, postexposure prophylaxis, clinical management of HIV infections including the treatment and prevention of opportunistic infections, and ART for adults and children.

Box 1. Countries in the Eastern Mediterranean Region of the World Health Organization or the Middle East and North Africa Region of the United Nations Children’s Fund, 2009

Afghanistan,a Algeria,b Bahrain, Djibouti, Egypt, Islamic Republic of Iran, Iraq, Jordan, Kuwait, Lebanon, Libyan Arab Jamahiriya, Morocco, Oman, Pakistan,a Qatar, Saudi Arabia, Somalia,a Sudan,c Syrian Arab Republic, Tunisia, United Arab Emirates, West Bank and Gaza Strip, Yemena

a Afghanistan, Pakistan and Somalia are part of the WHO Eastern Mediterranean Region but not the UNICEF Middle East and North Africa Region.
b Algeria is part of the UNICEF Middle East and North Africa Region but not the WHO Eastern Mediterranean Region.
c Both northern and southern Sudan AIDS programmes were invited to participate in the review.

After submission of the guidelines, the key informants of the national AIDS programmes were asked to complete a 49-item country questionnaire on the process involved in developing individual clinical practice guidelines. The questionnaire responses were sent to the reviewers together with the guidelines. Five teams of regional and international reviewers, hereafter referred to as the guideline review group, independently evaluated the development methodology and clinical content of the guidelines.

Two appraisal tools were developed for the guideline review: the process of guideline development was assessed using an adapted version of the Appraisal of Guidelines Research and Evaluation (AGREE) instrument from the AGREE Collaboration12 and guideline content was assessed by comparison with WHO 2006 and 2007 global recommendations on HIV treatment.36 The methodology and content of the guidelines were appraised by at least two reviewers. To ensure that any inconsistencies between different guidelines from a single country were identified, the same reviewers appraised the methodology and content of all guidelines submitted by that country. To ensure impartiality, none of the reviewers appraised guidelines that he or she had helped to develop or adapt.

All data from the country questionnaires and on guideline methodology and content were entered into an Access database (Microsoft Corporation, Redmond, USA).

During a consensus workshop in June 2009, the guideline review group discussed the methodology of the review process and the results of the review. In addition, recommendations for future local guideline development and adaptation were formulated following a discussion of the review’s findings with representatives of national AIDS programmes, WHO country, regional and headquarters staff and HIV experts from the region.

Guideline review and appraisal tools

The guideline development methodology was appraised using a tool based on the framework of the AGREE Collaboration instrument and incorporating elements of a Joint United Nations Programme on HIV/AIDS (UNAIDS) document entitled Developing HIV/AIDS treatment guidelines.13 The appraisal tool included 33 items (Table 1) that were scored using a 4-point Likert scale (i.e. 4 = strongly agree, 3 = agree, 2 = disagree and 1 = strongly disagree) and grouped into five quality domains: scope and purpose, stakeholder involvement, rigour of development, clarity and presentation, and clinical applicability and flexibility. To help users understand the items, documentation provided with the tool contained explanatory examples. Reviewers scored clinical practice guidelines using information about the guideline development process contained in the guideline documents themselves or obtained from country questionnaires.

Mean scores on the five quality domains were calculated in accordance with AGREE procedures,12 with standardized scores ranging from 0 (lowest) to 1 (highest). There is no accepted threshold for differentiating between high- and low-quality guidelines with AGREE instrument scores.14,15 However, for our analysis we set a domain score threshold of 0.60 as indicating “reasonable” quality.

Guideline content was appraised using a concordance checklist (available from: http://telemedicine.itg.be/telemedicine/uploads/contentreview_checklist.pdf) divided into five sections dealing with specific topics: postexposure prophylaxis, the prevention of mother-to-child HIV transmission, clinical management, ART in adults and ART in children. Each section contained a comprehensive list of programmatic and clinical recommendations based on WHO guidelines from 2006 and 2007 and subsequent publications.36,1622 The reviewer had to verify whether each recommendation was included in the national guidelines and was accurate and up to date. In addition, any gaps in clinical content were identified.

For this paper, the analysis of the concordance between national guidelines and WHO recommendations was limited to selected core recommendations on the prevention of mother-to-child HIV transmission, ART in adults and paediatric ART. A more extensive analysis of concordance was judged inappropriate because of the imminent revision of WHO global guidelines, which will require countries to adapt their clinical practice guidelines in the near future.

The final assessment of national HIV clinical practice guidelines was based on an analysis of the review results and ensuing discussions at the consensus meeting.

Results

Guidelines were submitted by 14 national AIDS programmes from the 23 countries invited to participate in the review. Two submissions were not retained: the first because an electronic copy was not available and the second because it was an exact copy of non-WHO reference guidelines, which made the concordance and methodological review less informative. The 12 participating national AIDS programmes, which are referred to as programmes A to L in Table 2, submitted a total of 20 sets of guidelines: 13 were topic-specific (e.g. on the prevention of mother-to-child HIV transmission), while 7 covered more than one topic. Most participating countries had started to develop HIV clinical guidelines only in 2006 or later. Eighteen reviewers, who were divided into five teams, carried out a minimum of two methodological and two content evaluations for each set of guidelines, except for guidelines on the prevention of mother-to-child HIV transmission from national AIDS programme I and for the comprehensive guidelines from national AIDS programme J (Table 2).

Quality of the guideline development methodology

A total of 51 appraisals of the guideline development methodology were available for analysis. Standardized mean scores on the five quality domains for each of the guidelines are shown in Table 2.

Overall, only one of the 20 clinical practice guidelines had a satisfactory score in all five quality domains. Nevertheless, the quality of the clinical practice guidelines was generally satisfactory in terms of their scope and purpose (median domain score: 0.82; interquartile range, IQR: 0.58–0.89) and clarity and presentation (median domain score: 0.67; IQR: 0.50–0.78). However, about one-third of guidelines did not reach the domain score threshold of 0.60 in these two domains. The median score for the quality domain of rigour of development, which was 0.45 (IQR: 0.34–0.55), indicates that there was a general failure to adhere to the appropriate criteria; only one set of guidelines scored higher than 0.60 on this domain. Scores on the domains of stakeholder involvement and applicability and flexibility were slightly better but remained poor overall: the median score was 0.54 (IQR: 0.42–0.63) and 0.57 (IQR: 0.45–0.69) for the two domains, respectively.

Median scores for the 20 sets of guidelines on all 33 items of the methodology appraisal tool are listed in Table 1. The median score was 2.0 or less for the following five items: seeking patients’ views and preferences; declaring potential conflicts of interest among guideline committee members; providing procedures for updating guidelines; providing a mechanism for guideline users to make comments; and describing ethical issues that may arise in applying the guidelines.

Content review and concordance with WHO recommendations

A total of 55 content appraisals were available for analysing concordance between the clinical content of the 20 submitted guidelines and WHO recommendations.

Fifteen of the 20 clinical practice guidelines cited WHO recommendations as one of their main references: 10 cited WHO 2006 and 2007 recommendations,36 while 5 cited WHO 2003 recommendations.23 Two guidelines cited non-WHO guidelines as a principal reference, namely United States Department of Health and Human Services 2006 guidelines,24 while no clear references were available for the remaining three guidelines. On the basis of the guideline content review, the guideline review group concluded that five sets of guidelines (25%) were clearly outdated and referred to WHO 2003 recommendations. None of the 20 guidelines had been amended with updates since their initial publication.

Application of the content verification checklist to all 20 guidelines enabled the reviewers to assess the strengths and weaknesses of around half the national AIDS programmes in the WHO Eastern Mediterranean Region and the UNICEF Middle East and North Africa Region. There were striking recurrent weaknesses in the following areas: antiretroviral prophylaxis for the prevention of mother-to-child HIV transmission; HIV diagnosis in exposed infants; care and prevention interventions for people living with an HIV infection; management of the side-effects of antiretrovirals; management of immune reconstitution syndrome associated with ART; monitoring patients on ART; treatment failure; and switching to second-line therapy. In addition, recommendations on counselling, psychosocial support, non-occupational postexposure prophylaxis and treatment of special groups, such as individuals with HIV and tuberculosis coinfections, individuals with HIV and hepatitis B or C virus coinfections and injecting drug users, were covered either superficially or not at all by most guidelines.

For the concordance analysis of core recommendations on ART for adults, paediatric ART and the prevention of mother-to-child HIV transmission, only guidelines or sections of guidelines that were relevant to the topic and not outdated were considered: seven guidelines dealt with adult ART, five with paediatric ART and seven with the prevention of mother-to-child HIV transmission.

The decision to initiate ART in adults, whether pregnant or not, adolescents and children was based on WHO clinical staging and the CD4 T-cell count in all but two countries. These two countries use a viral load cut-off of > 100 000 copies/mL. Five adult ART guidelines recommended that therapy should be started in asymptomatic adults as soon as the CD4 cell count drops below 200 cells/mm3; the remaining two guidelines used a less stringent cut-off of 350 cells/mm3.

Three of the five paediatric guidelines recommended that 2008 WHO revised criteria should be used to decide when ART should be started in infants and children: i.e. when the proportion of CD4 cells is < 15% in children aged 5 years or more or < 20% in those aged between 12 and 59 months, and in all infants aged under 1 year with a confirmed HIV infection or presumptive severe HIV disease.16

The eligibility criteria for ART in pregnant women cited in the seven relevant guidelines were less concordant with WHO 2006 recommendations.4 Three guidelines adopted WHO 2006 recommendations or less stringent criteria, one differed slightly by recommending “prophylaxis only” for all patients with clinical stage-I disease regardless of CD4 cell count and the remaining three did not specify any clear criteria.

Only one set of guidelines covering adult ART and one on the prevention of mother-to-child HIV transmission recommended first-line treatment with two nucleoside reverse transcriptase inhibitors and a boosted protease inhibitor. The other 12 guidelines for these two patient groups recommended the first-line combination preferred by WHO: two nucleoside reverse transcriptase inhibitors and a non-nucleoside reverse transcriptase inhibitor. Nine guidelines from seven countries recommended that first-line ART for adults, whether pregnant or not, should be based on the combination of zidovudine and lamivudine, which is one of the backbones of nucleoside reverse transcriptase inhibitor treatment preferred by WHO, rather than on other possible combinations of nucleoside reverse transcriptase inhibitors. Stavudine was recommended in cases where there was zidovudine toxicity by seven guidelines from five countries but only one set of adult ART guidelines adopted the modified stavudine dosage recommendation.22 Three of the seven adult ART guidelines singled out efavirenz as the non-nucleoside reverse transcriptase inhibitor of choice for adults who are not pregnant. In three other guidelines, no preference was specified and one opted for a boosted protease inhibitor regimen.

Three of the five sets of paediatric guidelines recommended that the first-line regimen in infants should take into account previous exposure to non-nucleoside reverse transcriptase inhibitors. The other two sets of guidelines still recommended treatment with two nucleoside reverse transcriptase inhibitors and a non-nucleoside reverse transcriptase inhibitor for all infants regardless of their previous exposure to antiretroviral drugs. All guidelines recommended treatment with two nucleoside reverse transcriptase inhibitors and a non-nucleoside reverse transcriptase inhibitor for children aged over 1 year who are starting ART.

Guidance on HIV diagnosis in children aged under 18 months was adapted correctly from WHO generic guidelines in two of the five sets of paediatric guidelines. In the other three, misinterpretations led to ambiguous and incorrect recommendations.

In three of the seven sets of guidelines on the prevention of mother-to-child HIV transmission, recommendations on antiretroviral prophylaxis for pregnant women who are not eligible for ART for their own health failed to comply with WHO 2006 recommendations:4 one did not provide any guidance on prophylaxis, one had an inconsistency between the text and tables and one omitted the administration of zidovudine and lamivudine for 7 days following peripartum single-dose nevirapine.

Review process evaluation

Generally, the reviewers liked the methodology appraisal tool based on the AGREE instrument and found it easy to use. Moreover, the adaptations made to the original AGREE instrument by integrating items from the UNAIDS document on developing HIV guidelines and by attempting to fit other AGREE items to the particularities of national guideline development were felt to be appropriate for the purpose of this review.13 The lack of information on the guideline development process in most guideline documents would have made it difficult to award a score using the appraisal tool if information from country questionnaires had not been available.

The methodology appraisal tool was also useful for increasing the reviewers’ awareness of potential common biases that may have affected the validity of the guidelines, e.g. the exclusion of some professional groups and conflicts of interest. The guideline review group recognized that the methodology appraisal tool could be used as a guide for comprehensive methodological appraisals or as a checklist for development committees during guideline development and adaptation.

The content concordance checklist was more cumbersome and time-consuming to use, particularly with outdated guidelines, guidelines that consisted only of flowcharts and guidelines whose structure was completely different from the WHO global guidelines. Only two guidelines had a similar structure and format, including the order of chapters, as the WHO 2006 generic guidelines, which made content verification for this review much more straightforward. Nevertheless, reviewers generally appreciated the comprehensiveness of the checklist as it enabled them to become familiar with WHO generic guidelines while identifying gaps in the country guidelines.

The guideline review group also felt that the whole process, from the initial meetings to discuss and plan for the review to the consensus meeting at which results were shared, has enhanced the region’s capacity to develop methodologically and technically valid HIV clinical care guidelines in the future.

During the consensus meeting, the guideline review group formulated a set of pragmatic recommendations which we believe will enable countries to produce more current, coherent, correct, comprehensive and customized (i.e. 5 Cs) national clinical practice guidelines (available from: http://telemedicine.itg.be/telemedicine/uploads/recommendations.pdf).

Discussion

The appraisal tools used in this review enabled the development methodology and clinical content of HIV clinical practice guidelines to be evaluated in detail. Overall, the review’s findings confirm that translating WHO global HIV clinical care recommendations into valid, evidence-based, context-specific and up-to-date national guidelines remains a major challenge.

We identified important gaps in the guideline development methodology in 19 of the 20 guidelines reviewed. Weaknesses principally concerned the methods used to select and analyse reference guidelines and the formulation of tailored recommendations (both of which were assessed by the rigour-of-development domain of the methodology appraisal tool), the extent to which guidelines represented the views of their intended users (assessed by the stakeholder-involvement domain) and the extent to which the implications of applying the guidelines were considered and taken into account (assessed by the applicability-and-flexibility domain).

The clinical content of a quarter of the guidelines evaluated was mainly out of date, even though the guidelines were still officially in use. For most of the remainder, the coherence and comprehensiveness of the recommendations were limited because guideline development was not rigorous enough and guidelines were not sufficiently adapted to their context.

In some cases, the absence of one or more clearly stated principal reference guidelines led to incoherent and incomplete recommendations. Moreover, generic reference guidelines that gave complex recommendations or recommendations involving several options also tended to result in erroneous or unclear national guidelines. In particular, sections of WHO guidelines that presented several management options and lacked clear-cut recommendations were more likely to be poorly adapted.

Overall, the review group felt that WHO global recommendations may not result in better clinical care unless efforts are made at global, regional and national levels to help countries keep up with the pace of change and ensure that their national guidelines are of a sufficiently high quality.

As a result, we devised a set of recommendations for developing national clinical practice guidelines (available from: http://telemedicine.itg.be/telemedicine/uploads/recommendations.pdf). These recommendations were made in the knowledge that national AIDS programmes were planning to revise their HIV clinical practice guidelines to take account of the 2010 revised WHO recommendations on ART in adults, paediatric ART, the prevention of mother-to-child HIV transmission and infant feeding in the context of HIV infection.2528 Some of our recommendations had previously been suggested by other groups.29

Previous papers have reported only on the concordance between key recommendations in WHO reference publications and national guidelines. This is the first paper to describe the methodological problems that arise when countries translate generic recommendations into contextualized national clinical practice guidelines. It is also the first paper to report on the application of an adapted AGREE instrument to HIV clinical practice guidelines.

The guideline review group concluded that the methodology appraisal tool based on the AGREE instrument and used in this review is a user-friendly tool for guideline appraisal and can serve as a checklist of essential methodological issues that should be taken into account during guideline development.

Conclusion

This review highlights the prevailing lack of a systematic approach to adapting national HIV clinical practice guidelines from generic global recommendations. In particular, national guideline development committees find it difficult to achieve an acceptable degree of rigour during guideline development and to customize guidelines adequately to the local context while ensuring the accuracy of the clinical content.

The impact of generic international guidelines could be maximized by paying more attention to and putting more effort into the development of national adaptations. To this end, clearly identifying core recommendations, recommendations in revised guidelines that differ from those in previous guidelines and recommendations that depend most on the context would be helpful, as would the development of templates for regional or national guideline adaptation.

We believe that future guideline development in the WHO Eastern Mediterranean and UNICEF Middle East and North Africa regions will benefit from the review process and we encourage other regions to launch similar initiatives since many will face comparable challenges with guideline development.


Acknowledgements

We would like to thank: Dr Séverine Caluwaerts, Dr Francoise Lequarre, Dr Natalie Lorent and Dr Johan Van Griesven of the Institute of Tropical Medicine, Antwerp Belgium; the regional experts Dr Nour Elhouda Ata Alla (Sudan), Professor Soumia Benchekroun (Morocco), Professor Kamal Marhoum El Filali (Morocco), Dr Sanaa Filemban (Saudi Arabia), Dr Ayesha Kahn (Pakistan), Dr Emmanuel Lino (Sudan), Professor Minoo Mohraz (Islamic Republic of Iran), Dr Jacques Mokhbat (Lebanon), Dr Mayada Nabeih (Yemen) and Dr Omer Nemery (Sudan); WHO Country Office staff responsible for HIV/AIDS Dr Rogers Busulwa (Sudan), Dr Neway Fida (Sudan), Dr Mitra Motamedi (Islamic Republic of Iran), Dr Moses Mutebi (Yemen), Dr Deq Said (Somalia) and Dr Endalamaw Tegegne (Sudan); UNICEF Middle East and North Africa Regional Office staff Samir Anouti and Dr Mahendra Sheth; and WHO headquarters staff Dr Tin Tin Sint and Dr Charlie Gilks (now UNAIDS India).

Funding:

Funding for this review was provided by the WHO Regional Office for the Eastern Mediterranean and the UNICEF Middle East and North Africa Regional Office.

Competing interests:

None declared.

References

Share