|
|
11 YT-RAR: Evaluation
Last Visited Page
| Previous YT-RAR Chapter |
Next YT-RAR Chapter
Printer Friendly Version
|
Summary
This chapter offers guidelines for evaluating tobacco control interventions.
|
Chapter Contents
- Evaluation of tobacco interventions
- Evaluation terminology
- Implementation evaluation
- Impact evaluation
1 Evaluation of tobacco interventions
Tobacco interventions need to be evaluated to show whether they are properly targeted, whether they work the way as intended, whether they are effective, what they cost, and whether there are unexpected problems or negative effects. To answer these questions, evaluation focuses on particular aspects of a project or intervention - such as its coverage, cost or health outcomes - using a combination of qualitative and quantitative methods. Evaluation concepts and methods will be found in TG-RAR Ch 11 Evaluation.
Evaluation can be conducted on almost any intervention:
- a project that delivers tobacco educational information to students
- a mass media campaign aimed at young smokers
- a government policy of no-sales of tobacco products on government premises
- a new law banning provision or sale of matches to tobacco purchasers.
It is possible, for example, to evaluate whether a law reaches its target population, is appropriately enforced, and has the desired impact. Evaluation can be undertaken of whether a policy is understood by its target population (e.g. customers in a non-smoking establishment such as a restaurant), how it is implemented (whether they and others comply with it) and what impact it has (rates of tobacco use at the establishment).
|
Evaluation can take place at a number of different levels:
Individual level indicators – for example:
- Change in knowledge related to the health effects of tobacco
- Change in attitudes related to tobacco use
- Change in actual behaviours (starting, quitting, reducing, etc.)
Community level indicators – for example:
- Changes in level of tobacco use among peer groups
- Changes in expectation of tobacco use among peers
- Changes in social norms (smoking at work increasingly socially stigmatised)
Structural level indicators – for example:
- Passage of a law protecting youth non-smokers from second-hand smoke in public
- Implementation of legislation increasing tobacco tax by 40%
|
Top
2 Evaluation terminology
Evaluation terminology helps in producing accurate and clear summaries of what an intervention is actually achieving compared to what it aims to achieve.
|
Terminology
|
Example
|
|
Goal (sometimes called aim)
|
To prevent tobacco use among young people.
|
|
Objective
|
To change youth attitudes about the acceptability of tobacco use among their peers.
|
|
Target
|
Increase by 50% the youth perception that tobacco companies are manipulating and exploiting young people.
|
|
Strategy
|
School based education.
|
|
Activity
|
To deliver a tobacco education programme using the theme of media literacy that is delivered to school children aged 14-15 years (target population) at all at schools in the community twice per year for half a day each (contact strategy).
|
|
Output
|
A tobacco education program based on the skills of media literacy that is delivered in all schools in the community.
|
|
Milestones
|
Establish a project committee by month two, open the project by month four, deliver a workshop production in six schools by month six.
|
|
Outcomes
|
Schoolchildren are more knowledgeable about the aims of the tobacco industry, their sales and promotion, and advertising techniques. Change in attitude toward tobacco industry and tobacco use.
|
Top
Evaluation can examine the implementation of the project, its impact, its cost, or all three. It is usually necessary to determine that a project has been properly implemented before there can be an evaluation of its impact.
|
Case study: exposure to 'Don't be Duped' campaign
We wanted to distinguish between: (a) failure of project implementation – the intervention did not happen as planned. For example people did not receive the information package or few people went to see the music concert, and (b) failure of intervention strategy – the intervention happened, but it did not have an effect. For example, people saw the posters and/or attended the rock concert but tobacco use (or attitude toward tobacco) did not change. We designed some exposure and participation questions. We wanted to know:
- Did they see the "Don't be Duped" posters?
- Did they receive the information package that was sent to them?
- Did they understand the campaign as it was intended?
- Did they participate in the community "Don't be Duped" protest?
- Did they participate in the "Don't be Duped" community event rock music concert?
To assess exposure, we showed them a photograph of the poster and asked them:
- Have you seen this? Where did you see it? What did it say? Did you receive a copy? Can you show it to me? What did you do after reading it?
|
Top
Interventions have a variety of outcomes by which they might be assessed. Outcomes are generally changes that occur in the target population, such as rate of tobacco use initiation or sales to youth, or level of awareness. To measure impact there must be clearly measurable project objectives that state the outcomes that the programme seeks to achieve. Considerations to keep in mind when planning a tobacco related evaluation:
- The prevalence of tobacco use does not normally change quickly over time
- Tobacco consumption (units per day) is more sensitive than prevalence to tobacco control interventions
- Initiation rates are more sensitive than prevalence to tobacco control interventions
- The poor and young are more sensitive to price changes than adults
- The health effects from tobacco typically reflect many years of use – so young smokers are rarely reflected in health service provision
4.1 Did the intervention cause the outcome?
Assessing outcomes is extremely important. Many project evaluations report relationships between interventions and outcomes. If possible, it is important to assess whether the outcome occurred as a result of the intervention.
|
Example: causes of changes in awareness of tobacco advertising tactics
A survey one month after a round of media literacy workshops at district schools finds that students have an exceptionally high awareness of specific tobacco industry advertising tactics. Initially, this seems to suggest a stunning success on the part of the half day workshop. Further analysis of students' feedback reveals, however, detailed awareness of tactics never discussed in the workshops. It is later discovered that a popular television programme recently aired an episode in which the teenage main character learns about certain underhanded advertising tactics of a tobacco company – the very tactics, it turns out, that the school children all recall so vividly.
|
An assessment of the impact of a project can only be made with varying degrees of plausibility. To ensure more sensitivity to the impact of the project, select indicators sensitive to and reflecting expected changes.
Impact assessments are generally made by comparing information on participants and non-participants, or by comparing the same participants over time.
Last Visited Page
| Previous YT-RAR Chapter |
Next YT-RAR Chapter
Printer Friendly Version
Top
|
