Evaluation of regional workshops and follow up activities to strengthen the role of healthcare professional associations in achieving MDGs4 & 5
Many low- and middle-income countries are not on track to reach the public health targets set out in the Millennium Development Goals (MDGs). Research has indicated that health-care professional associations (HCPAs) can greatly contribute to the achievement of MDGs 4 and 5. However, these associations are weak and uncoordinated. Therefore, in line with the HCPA Joint Statement, the Partnership for Maternal, Newborn & Child Health (PMNCH) organized three regional workshops in response to: (1) the growing recognition of the potential role of HCPAs in addressing the human resource crisis; (2) a commitment by international HCPAs to collaborate on strengthening national HCPAs and increase their ability to engage in advocacy activities; and (3) a need to increase involvement of HCPAs in planning and implementing programmes and polices related to reproductive, maternal, newborn and child health (RMNCH).
The three workshops were held in November 2007 (Blantyre, Malawi), March 2008 (Ouagadougou, Burkina Faso) and November 2008 (Dhaka, Bangladesh). They brought together representatives from national HCPAs involved in RMNCH service delivery (pharmacy, nursing, midwifery, obstetrics and gynaecology and paediatric associations), public-sector representatives and other key stakeholders. Their aim was to enhance the role of HCPAs in achieving MDGs 4 and 5.
PMNCH structured the workshops around five key growth areas: (1) advocacy; (2) human resource; (3) organizational strengthening; (4) service quality improvement; and (5) RMNCH planning. The 17 participating countries were: Afghanistan, Bangladesh, Burkina Faso, the Democratic Republic of Congo (DRC), Ethiopia, Haiti, India, Malawi, Mali, Myanmar, Nepal, Niger, Nigeria, Pakistan, Senegal, Tanzania and Uganda.
This report assesses the regional workshops at three levels: (1) the achievement of the overall workshop objective, which was to increase the capacity of national HCPAs to contribute to the development of national RMNCH policies and programmes; (2) interactions between and among HCPAs, ministries of health (MOH) and development partners; and (3) the implementation of country specific action plans (CSAPs) defined during the workshops.
It is to be noted that points (2) and (3) move towards the achievement of the overall workshop objective. The report also focuses on the factors facilitating and inhibiting implementation of the original workshop objectives, and makes suitable recommendations.
The evaluation team used both quantitative (web survey to all participants) and qualitative research techniques (desk reviews of available documents and sample key informant interviews). 195 participants attended the three workshops – 58 in Blantyre, 59 in Ouagadougou and 66 in Dhaka. There were a further six participants from PMNCH and six representing the advisory board to the PMNCH. The latter were salient senior representatives from international HCPAs involved in direct coordination, organization and follow-up of regional workshops along with PMNCH. It should be noted that the evaluation was hampered by difficult communications (poor internet access and telephone connectivity) and lack of baseline information on national HCPAs.
Themes and recommendations
Recommendations are discussed under seven themes, which evolved by converging evidence from all three data sources (triangulation of data).
- PMNCH support. PMNCH support was mentioned in all three data sources as a main source of encouragement to move forward in the implementation of CSAPs.
Recommendation: PMNCH should consider where it can most usefully support future workshop participants in implementing their actions plans.
- HCPAs, MOH and United Nations agencies: individual roles and need for coordination.
Several respondents noted the need for a more integrated and coordinated approach to enhancing HCPAs’ capacity to implement CSAPs and contribute to RMNCH. This applied at all levels, from health ministries and international agencies down to individual HCPAs.
Recommendation: PMNCH and HCPAs should develop a joint strategy for more effective coordination in future. The focus should be on areas they can influence directly.
- Financial constraints. Survey participants from Bangladesh, Myanmar, Burkina Faso, Mali, Niger and DRC all pointed out lack of funding as a major impediment towards taking CSAPs forward. Interviewees from the advisory board, DRC and Haiti mentioned financial constraints as a major barrier to the accomplishment of targets laid out in the CSAPs.
Recommendation: PMNCH should consider earmarked funds for follow-up activities, a funding assessment checklist for each CSAP, and the potential of additional funding sources.
- Ownership of the process: delegation of a full-time focal person to provide leadership. The survey respondents from Uganda and Bangladesh mentioned lack of personal involvement and failure of focal persons as major factors impeding implementation of CSAPs. Other feedback supported the view that ownership of the process and dedicated human resource are essential for post-workshop follow-up.
Recommendation: PMNCH should consider the appointment of a focal person for each country during the pre-workshop planning phase, avoiding senior people who may be too busy to do the job properly.
- Communication strategy. Lack of clear lines of communication between PMNCH, development partners and HCPAs was referred to as a weakness of CSAP implementation by interviewees from the advisory board, Burkina Faso and Bangladesh.
Recommendation: PMNCH should ensure that appropriate communication methods are used for each country, and that post-workshop communications are well-coordinated and integrated across stakeholders.
- Regular supervision and monitoring. A survey respondent from Mali, interviewees from the advisory board and nine out of 13 interviewees from the Dhaka workshop all said that regular supervision and monitoring were needed.
Recommendation: PMNCH should consider adopting a results-based management (RBM) approach for future workshops.
- Realistic action plans. Nine interviewees (four from the advisory board, four from Pakistan and one from DRC) said that workshop objectives could not be implemented unless CSAPs were realistic.
Recommendation: PMNCH should look closely at how CSAPs are framed during workshops, and ensure that lessons learned are carried forward to future workshops.
- Proposed conceptual framework. Without strong and sustained follow-up by PMNCH, and a clear sense of ownership and commitment by HCPAs, it seems possible that the current workshop model will continue to under deliver.
Recommendation: A new conceptual framework (Figure 1) could serve as a useful reference when planning similar activities in future. It integrates most of the recommendations made in this report, linking the original idea of regional workshops with the final desirable impact.
Limitations of this Evaluation
Our evaluation was hampered by poor internet access and telephone connectivity and a lack of baseline information on national HCPAs. Also, a proper analysis of impact requires a comparison group. The absence of control HCPAs and an adequate sample size impaired assessment. The only source of the “before status” of RMNCH initiatives was anecdotes from interviews (with the element of recall bias) and desk reviews. These offered little for baseline comparison.
Although the desk reviews provide mixed evidence, most interview participants did not link their country’s post-workshop activities with the PMNCH workshop. Almost all interview respondents mentioned that an evaluation conducted after a lapse of time (about 24 months) was not feasible and evoked a recall bias.
Moreover, legal and contract-issuing procedures took time, which delayed the start of the evaluation. Direct face-to-face communication with the workshop participants was not possible and it was realized rather late that there would be a poor response to web surveys and phone connections. Aga Khan University does not have readily available direct international telephone connection points, so initially the evaluation team had to use mobile phones to schedule interviews with the workshop participants. Further challenges were posed by time differences across zones and the need to hire a French-speaking consultant to conduct some interviews.
This evaluation suggests there may be limitations to the current workshop format as a tool for enhancing the capacity of HCPAs to contribute to RMNCH plans. On the positive side, the workshops generated goodwill, raised awareness of key issues and led to increased interaction between HCPAs. On the negative side, most countries (88%) were unsuccessful in fully implementing their CSAPs and achieving the three key workshop objectives, while post-workshop activities were bedevilled by a range of difficulties.
In future, the challenge for PMNCH is to ensure that workshops are more successful in meeting all their objectives – and especially that of increasing the capacity of HCPAs to contribute to the development of national RMNCH policies and programmes. The proposed conceptual framework in this report outlines an approach that might achieve greater impact, and ultimately lead to better health outcomes for women and children.
Based on this report, we feel secure in saying that these sorts of workshops can be used in the future for increasing interactions among the HCPAs. In fact the regional approach is better in the light of given country level constraints provided that the elements of our proposed conceptual framework are taken into account. Finally, we found that country level advocacy was good in Burkina Faso and Niger (+++ on our scale) and helped in achieving the CSAPs. Therefore in our opinion workshops can encourage country level advocacy if MOH is involved as a major stakeholder during the implementation phase.