This entry is our analysis of a study considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.
Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text
Copy title and link
| Comment/query | Tweet
Effects of the National Youth Anti-Drug Media Campaign on youths.
Hornik R., Jacobsohn L., Orwin R. et al.
American Journal of Public Health: 2008, 98(12), p. 2229-2236.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Hornik at rhornik@asc.upenn.edu. You could also try this alternative source.
Could the US government's biggest ever attempt to use the media to turn US youth away from cannabis actually have done the reverse? At best it was a disappointment; at worst, it counterproductively fostered the impression that 'Everyone's doing it'.
Summary Several previous documents (see evaluation web site and this Findings analysis) from evaluators appointed for the US Congress have assessed different stages of the US National Youth Anti-Drug Media Campaign as it ran from its inception late in 1999 to 2004. The featured report draws the threads together and reaches conclusions about its impact on the uptake of cannabis use and on related attitudes and intentions among the subset its 9–18-year-old targets The other major target were parents. Cannabis became a specific focus when in 2002 the youth component of the campaign shifted to what was called the Marijuana Initiative. aged at least 12 and a half.
Funded by nearly $1 billion appropriated by the US Congress, the campaign aimed to educate and enable America's young to reject illegal drugs, to prevent initiation of drug use (especially cannabis and solvents), and to persuade occasional drug users to stop. Youth-focused communications mainly broadcast through television and radio ads (but also through posters and cards illustrations) aimed to achieve these objectives by bolstering young people's resistance skills and confidence in their abilities to reject drug use, correcting mistaken assumptions about how 'normal' or accepted drug use was among their peers, promoting positive drug-free alternatives, addressing the benefits of not using drugs, and highlighting the negative consequences of drug use, including effects on academic and athletic performance. Main features of the study's methodology are described below.
Data for evaluating the campaign derived primarily from annual reinterviews of national samples of 9–18-year-old youngsters and their parents. Initial interviews were conducted between November 1999 and June 2001 and the final round between mid-2003 and mid-2004. The sample was selected to provide a nearly unbiased national cross-section. About two thirds of young people agreed to join the study and supplied data at the initial interviews. Of these, 86% to 93% still eligible for the study were re-interviewed in subsequent rounds, totalling from about 8100 to 5100 young people in each round. The featured report focuses on children aged from 12 and a half to 18 This was because by the final round of interviews few children were under 12.5 years of age. To prevent analyses being complicated by children of different ages being sampled at each round, this was set as the lower limit. 18 years of age was the upper limit of the ages targeted by the campaign. However, the authors say that all the reported conclusions presented were supported by prior analyses of a broader age range of children. who either in the current interview or in the previous one had not Before the last stages of the study there were too few existing users of the drug to test impacts on persuading occasional users to stop. used cannabis.
Since the campaign was national, the researchers could not recruit comparison samples from areas not exposed to its messages. Instead they tested its impact primarily Also assessed was whether cannabis use and related attitudes and beliefs among young people changed over the course of the campaign. There was no change in the prevalence of cannabis use among people aged 12.5 to 18 between 2000 and 2004. A small but significant increase in anti-cannabis beliefs and attitudes was not accompanied by significant parallel gains in intentions not to use, social norms, or self efficacy. There were some significant year-to-year changes (including an anti-cannabis shift in intentions from 2002 to 2004) and a few significant changes for subgroups of the samples. by relating Relationships between exposure to the campaign and these cannabis-related measures were adjusted to take account of individual and household characteristics (some derived from the responses of their parents) which the study found to be related to exposure to the campaign and/or to the young person's cannabis use. The intention was as far as possible to statistically 'eliminate' influences which might create a spurious relationship between exposure to the ads and cannabis use, clearing the way for a more valid estimate of the possible impact of the ads themselves. how many times each young person recalled seeing or hearing anti-drug ads in general, or those from the campaign in particular, to their answers to various questions about cannabis. If the ads had been effective, the more intensely children had been exposed to them, the less likely non-users should have been to later try using cannabis. Associated with this should have been (then or later) a corresponding impact on attitudes and beliefs protective against cannabis use. These outcome measures included young people's lifetime or recent use of the drug, and whether they definitely intended Itself strongly predictive of whether they actually did not later use the drug. not to use it in the next year. The researchers also created an index Weighting the different components so they were closely related to intention to use. of the youngster's balance of positive versus negative attitudes/beliefs relating to the drug. Similarly constructed was a 'social norms' index based on their beliefs about how often their peers used cannabis, and about how negatively their friends, parents and other people in their lives would react if they knew the young person themselves were using the drug.
Nearly all the young respondents recalled seeing or hearing at least some the campaign's ads, but there was marked variation in the extent of this exposure. Yet this variation was essentially unrelated to their current attitudes and beliefs about cannabis use, including intentions to use. More significantly, greater exposure in any given year of the campaign was not associated in the following year with fewer youngsters starting to use the drug, nor with attitudes and beliefs protective against cannabis use. If anything, the reverse was the case; in respect of exposure to the campaign itself, one of the associations was neutral and the remaining four in the 'wrong' direction. These included a
possibly chance
Especially if multiple outcomes tend not to covary, the more are measured, the more likely it is that some will reach the threshold for a statistically significant difference purely due to chance variations in the samples rather than any real impact of the interventions being tested. For example, by convention, if a difference would happen only 1 in 20 times by chance, it is considered a non-chance occurrence possibly due to the intervention. But if, say, 20 independent outcomes are measured, more often than not one would cross this threshold purely by chance. To cater for this, it is recommended1 that researchers consider raising the threshold (in the example, according to some adjustment methods to as high as 1 in 400) before each of the outcomes is considered to have reflected a statistically significant difference.
1 International Conference on Harmonisation Of Technical Requirements for Registration of Pharmaceuticals for Human Use. "ICH harmonised tripartite guideline statistical principles for clinical trials." Statistics in Medicine: 1999, 18, p. 1905–1942.
statistically significant trend for social norms to become more favourable to cannabis the more ads the child recalled experiencing, and a non-significant tendency for more exposure to be followed by a greater chance that the child would try cannabis during the following year. The picture was the same when the sample was broken down in to different sub-groups to test if certain children responded well to the campaign. Among these 80 tests of ads in general and the campaign ads in particular, 20 reached statistical significance, all but one in the 'wrong' direction. Results from the second half of the study period, when cannabis had become a specific target, were similar.
The researchers concluded that while the campaign had successfully reached the children it targeted, there was no evidence that this exposure had the intended impacts on their cannabis uptake, and it may have promoted more pro-cannabis attitudes and beliefs.
commentary Despite unprecedented funding and government backing, this most high-profile of campaigns seems to have left its young targets unmoved or possibly nudged in the opposite direction to that intended, an object lesson in the difficulty of constructing persuasive messages in respect of one of the least dangerous drugs, and the risk that the attempt could backfire. It should however be remembered that parents did in some respects react as intended to the parental strands of the campaign. Also, other health promotion mass media campaigns (including some targeted at substance use) have been unable to demonstrate the intended effects on behaviour, though the evidence tends to be weak, especially in respect of young people. For example, smoking has been a major target, but just four Other studies involved advertising campaigns combined with health promotion activities of a different kind. methodologically strong studies of campaigns aimed at young people were found by reviewers up to 1998. Of these, only one – which employed a dubious strategy No adjustment was made for the fact that communities were allocated to be intervention and non-intervention sites yet the results were analysed as if individuals had been allocated. to analyse its results – found any impact on smoking. A later review of US studies still found few in which media campaigns were 'uncontaminated' by other initiatives, though these few studies often found reduced smoking in youth and adult target populations. Among the most convincing of recent studies was one which linked local variation in the intensity of a national US anti-smoking campaign to the initiation of smoking by teenagers and young adults.
After the period reported on in the featured study, the campaign again changed tack, adopting the label Above the Influence to signify the incorporation of messages encouraging children to avoid or resist peer influences which promote substance use. The campaign's own monitoring suggests this has avoided its predecessor's counterproductive impact on social norms related to cannabis use. But at the time of writing this latest revision of the campaign had yet to be subjected to the kind of independent evaluation which was unable to support the earlier versions.
However, it remains possible that the earlier campaigns really did have the intended impacts, but these were not picked up by the featured study. Details in background notes. In summary, there seems little to support arguments that the ads stuck in the mind of children most likely to use cannabis, creating the illusion that seeing the ads caused pro-cannabis effects. From the study's broader findings, it also seems unlikely that there were counterbalancing positive impacts on the more high-risk children or (these were excluded from the featured report) children who had already tried the drug. Two studies (details in background notes) in the same two medium-sized cities, either of the campaign itself, or of anti-cannabis ads specially developed to target high sensation-seeking teenagers, suggested that such teenagers may respond as intended, even if their less sensation-hungry peers are unmoved. Due partly to methodological problems and to the limited nature of the samples, these are not a persuasive counter to findings from the national study. What cannot however be excluded is the possibility that the outcomes in that study might have been partly due to differences between children who recalled lots and those who recalled few ads, rather than purely due to their exposure to the ads.
Another set of arguments accepts that the campaign was ineffective, and tries to explain why. Again, for details see background notes. Meta analyses and reviews (1 2) combining the results of relevant studies across health promotion offer clues to success factors lacking from the youth-oriented strands of the National Youth Anti-Drug Media Campaign: media campaigns are most effective when they publicise and reinforce an associated regulatory or law enforcement initiative; promoting new behaviours is easier than trying to prevent or stop problem behaviours; adults have responded better than children. It is also important (as some studies directly relevant to the featured campaign showed) to pre-test and adapt the campaign to the reactions of samples of the intended targets – a process which the US government admitted
was inadequate
Announcing the Marijuana Initiative on 23 May 2002, the Executive Office of the President Office of National Drug Control Policy said: "... we are implementing a number of changes, effective immediately:
1. Prior to being aired, all TV ads will undergo rigorous testing.
2. ..."
at least until the cannabis-specific phase.
The most far-reaching argument, advanced by the study authors themselves, and for which there was some evidence, is that no matter what deterrent impact the campaign's explicit messages may have had, these were (possibly more than) counterbalanced by an implicit message that drug use was hard to resist and common among children of the same age as the viewer or listener, a sometimes powerful influence promoting substance use. Why else, the young viewers might subliminally have reasoned, would the government be so keen to warn us about drugs and think we need help to resist?
The featured study had to rely on 'messy' real-world data. Others have been able to exercise greater control in examining how children react to the same or similar ads. They too support the simple explanation that the campaign seemed ineffective because it was, and also show how it might have been counterproductive. In one study nearly half the tested ads were seen as less effective in deterring youth substance (in particular, cannabis) use than simply watching a neutral TV programme. An offshoot of this study found that watching ads which graphically portrayed the 'gateway' message ('soft' drug leads to 'hard' drug use and addiction) left children feeling more positive about cannabis and more likely to use the drug, it seems because those most likely to use tended to "move towards disbelieving that regular marijuana use has negative consequences".
These studies suggest that among young people most likely to use cannabis, focusing on harmful consequences was a difficult strategy to carry off with any credibility in respect of a drug where clear-cut examples are hard to find. Apart from the unintended 'Everyone's doing it' message, some other features may also have undermined the campaign's effectiveness. A major theme implied that the choice young people faced was between cannabis use and other valued activities and identities, yet the experience of many will have been that usually no such dilemma presents itself. In turn this theme rested on the theme that cannabis use detrimentally dominates young lives, a depiction which all but a few could deflect as 'nothing like them' or their friends. Finally, there were explicit urges to independence of mind ("We need to stand up for ourselves and become independent thinkers"); if taken to heart, these might as easily have led to rejection of the government-sponsored messages as their acceptance.
Thanks for their comments on this entry in draft to Robert Hornik of the University of Pennsylvania. Commentators bear no responsibility for the text including the interpretations and any remaining errors.
Last revised 05 February 2010. First uploaded 14 January 2010
Background notes to this entry
Comment/query
Open Effectiveness Bank home page
Top 10 most closely related documents on this site. For more try a subject or free text search
REVIEW 2011 The effectiveness of anti-illicit-drug public-service announcements: a systematic review and meta-analysis
NASTY SURPRISES 2006 Boomerang ads
REVIEW 2015 Prevention of addictive behaviours
STUDY 2006 Effective delivery and positive message works for school-based media campaigns
STUDY 2011 Achieving positive change in the drinking culture of Wales
STUDY 2010 Preventing drug abuse among adolescent girls: outcome data from an internet-based intervention