A preliminary study of the population-adjusted effectiveness of substance abuse prevention programming: towards making IOM program types comparable
Effectiveness bank home page. Opens new windowReview analysis

This entry is our analysis of a review or synthesis of research findings considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original review was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the review. Below is a commentary from Drug and Alcohol Findings.

Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text
Copy title and link | Comment/query |

A preliminary study of the population-adjusted effectiveness of substance abuse prevention programming: towards making IOM program types comparable.

Shamblen S.R., Derzon J.H.
Journal of Primary Prevention: 2009, 30, p. 89–107.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Shamblen at sshamblen@pire.org. You could also try this alternative source.

One of the biggest strategic decisions facing prevention planners is whether to target high-risk groups or to prioritise universal programmes. This analysis won't decide the issue, but it does create an important new tool for comparing these strategies.

Summary The US Institute of Medicine distinguishes between prevention programmes based on who is targeted: the entire population (universal); those at risk (selective); or people exhibiting the early stages of use or related problem behaviour (indicated). Evaluations suggest that while universal programmes can reduce and prevent substance use, selective and indicated programmes are both more effective and have better cost-benefit ratios. Nevertheless, universal programmes may have a greater impact across the population because they affect more people. This paper attempts to 'level the playing field' by comparing the impact of these three different types of programmes in reducing and preventing tobacco, alcohol or cannabis There were too few studies reporting on other substances to include these substances in the analysis. use not only among individuals exposed to the intervention, but also across the same-age population as a whole (ie, those exposed and not exposed to the intervention), often termed a 'public health' perspective. Effectively the analysis statistically 'transformed' selective/indicated programmes in to their universal-impact equivalents.

To do this the 102 programmes listed as effective or model by the US National Registry of Effective Prevention were examined to identify those specified in a manual and evaluated in such a way that the results could be included in a meta-analysis A study which uses recognised procedures to summarise quantitative results from several studies of the same or similar interventions to arrive at composite outcome scores. Usually undertaken to allow the intervention's effectiveness to be assessed with greater confidence than on the basis of the studies taken individually. combining their outcomes. The analysis was able to include the results of 43 studies of 25 programmes classified as either universal, selective or targeted depending on the way the samples were selected for the studies. Impacts were assessed in terms of effect sizes, A standard way of expressing the magnitude of a difference (eg, between outcomes in control and experimental groups) applicable to most quantitative data. Enables different measures taken in different studies to be compared or (in meta-analyses) combined. Based on expressing the difference in the average outcomes between control and experimental groups as a proportion of the variability in the outcome across both groups. a metric which enables different outcomes to be compared and combined. To estimate the population-level impact of non-universal programmes, these calculations were adjusted for the proportions of the population targeted For selective programmes, this was the at-risk population, operationalised as the proportion exhibiting risk factors (or lacking protective factors) which would mean that half would go on to use each of the three substances in the study. These calculations were based on a national US survey. On this basis, 31% of 12–17-year-olds were at risk for using tobacco or alcohol and 12% for cannabis. For indicated programmes the proportions were based on (in the same survey) the proportions of the population in different age ranges who reported using each of the substances in the past month. For tobacco the proportions for 12–17-year-olds, 18–25-year-olds, and people aged 26 and over were 17%, 45%, and 30%; for alcohol the corresponding proportions were 17%, 57%, and 49%; and for cannabis, 7%, 14%, and 3%. by the programme.

Effect sizes for prevention programmes

Because there were relatively few selective or indicated programmes, and because many fell in both camps, estimates for these were combined. As expected, for the populations they targeted selective/indicated programmes for alcohol and cannabis use had greater effects than universal programmes. Unexpectedly, this was not the case for tobacco, where universal programmes had greater impacts chart. But as discussed above, selective/indicated programmes affect fewer people. When their impacts were statistically 'spread' across the population as a whole, they remained slightly more successful in reducing drinking and less successful in curbing smoking. In respect of cannabis use, the verdict changed; though relatively successful with the populations they targeted, across the population as a whole selective/indicated programmes fell short of the impact of universal programmes.

The upshot was a consistent advantage for universal programmes in preventing smoking, the same for selective/indicated programmes in curbing drinking, but for cannabis a mixed picture; greater impacts for selective/indicated than universal programmes on the their target populations, but a lesser impact across the population as whole. On the basis of these figures, from a public health perspective universal programmes achieve the greatest tobacco and cannabis use reductions, selective/indicated programmes the greatest drinking reductions.

Findings logo commentary These findings correct the false impression that because programmes focused on at-risk groups or individuals have greater and more statistically significant effects, this means they are preferable to universal programmes if the aim is to affect the population as a whole. Only in respect of drinking Presumably because (even in the US context where drinking is relatively uncommon compared to Europe, including the UK) drinking is so common that selective/indicated programmes come close to targeting the population as a whole, and it is easier to identify who to target with selective or indicated programmes. was this impression correct. Conclusions are very tentative because differences were sometimes small and because programmes within the same broad universal/selective/indicated categories significantly differed in their effectiveness, meaning the results of the analysis are likely to change if other programmes are included in the mix. However, the methodology pioneered in the study offers a way for public health authorities to level the playing field between universal and more selective programmes, whatever the mix of programmes for which data is available at the time, aiding the decision over which to prioritise.

Several issues remain to complicate this decision. Counting against selective programmes is the difficulty (previously addressed by one of the featured study's authors) of discerning who really is at risk of later use or problem use, meaning such programmes will exclude many who later prove to have been in need of their services. Because youngsters usually have strong incentives to hide signs of a developing problem, a similar issue hinders the targeting of indicated programmes. Partly because of this, delaying intervention until problems indicators have surfaced might be to leave it too late. Indicated programmes also face the issues of avoiding stigmatisation, counterproductive labelling of youngsters as on track to become problem adults, and the possibility that grouping these youngsters together for intervention will give a peer reinforcement boost to anti-social development.

Universal programmes also face complicating issues. They may lead the population as a whole to reduce use of the targeted substances, but within this population may at the same time exacerbate inequalities in use and health. This happens because well resourced individuals and groups are best able to take on board and act on health promotion messages, while the most vulnerable are less able to do so. The net result is to widen the gap between the two. Selective and indicated programmes should help mitigate this effect by delivering more intensive services to the more vulnerable sections of the population – assuming, as noted above, that these can be identified and that the programmes are able to counter sometimes deep-seated problems. Aggravation of health inequalities is not however an inevitable side-effect of universal programming. Sometimes (example 1; example 2) it can work the other way, partly because universal programmes cannot reduce use/problems in children who are in any event never going to engage in these behaviours.

Finally, though selective and indicated programmes sometimes make things worse, so too can universal programmes, especially those based on fear-arousal. Portraying drug-related damage may inadvertently attract youngsters Cragg Ross & Dawson Ltd. 1988/89 misuse of drugs anti-injecting campaign. Qualitative research report. July 1989. This report on the UK anti-injecting/anti-sharing campaigns running through the winter of 1988/9 found that emphasising revulsion and danger could attract non-injecting drug users into injecting. "For many respondents", commented the researchers, injection "represented the ultimate drug experience and initiation into an elite." Heroin itself – one of the drugs most likely to be injected – "had a formidable reputation... as... the ultimate drug." What made heroin 'ultimate' for the young drug users interviewed for the study was their belief that it led "most remorselessly" to injection, addiction and ruin – the theme of many of the pre-1988 anti-heroin ads in England and Wales. "For those fascinated by drugs as a challenge, heroin was the most challenging. Using heroin had clear potential to be an act of defiance," said the report's authors. bent on self-destruction or who see extreme forms of drug use as the ultimate drug experience and initiation into an elite. Those already using in the ways portrayed in the messages can feel under attack Andrew Irving Associates Limited. Anti-misuse campaign: qualitative evaluation research report. January 1986. This report concerned Britain's 1985/86 Heroin Screws You Up campaign. and further alienated from society. Fear-arousing messages also make communication between children and parents so fraught with emotion Cragg A. "The two sides of fear". Druglink: 1994, 9(5), p 10–12. that both steer clear, making it more difficult for parents to intervene. Even non-fear based campaigns risk counterproductively making use the targeted substance seem ('Otherwise why would they bother?') widespread, 'normal' and difficult to avoid.

The most fundamental issue facing both sorts of programmes relates to where the bulk of preventable problems lie in the society for each substance – an amalgam of the distribution of the problems plus their susceptibility to intervention. If these arise only among highly atypical (for example, very heavy) users, universal programmes which curb substance use among normal users make little contribution to reducing related problems, while they may fail to tackle the few problem-generating users. In this scenario, compared to more targeted programmes, universal programmes may make a greater dent in use levels across the population, but a lesser dent in related problems. Because it focused solely on use levels, the featured study was unable to exclude this possibility.

Where even typical use levels are associated with significant problems, the reverse can be the case. Because there are (by definition) far more typical than atypical users, the bulk of related problems across society may be found among normal users – the so-called 'prevention paradox' much discussed in respect of drinking. By reaching typical as well as atypical users, universal programmes can reduce these problems to a greater degree than programmes targeting just the atypical end of the spectrum. Of course, this argument only applies if universal programmes actually work. Raising the price of alcohol seems one effective example.

In the end these decisions must be taken on the basis of how things stand in a particular population for a particular problem related to a particular substance, and the adequacy of the interventions which can be brought to bear on those problems. It can vary for example by age; in some societies, acute problems like accidents and fights are spread across the younger population, but among the over-30s, largely confined to the heaviest drinkers. It can also vary depending on the problem being targeted: long-term health damage such as liver disease is concentrated among very heavy drinkers, but one-off incidents such as accidents and violence are more widely spread. When the targeting issue was looked at in respect of secondary school children in one Australian state, the verdict was that most problem drinking and smoking was found among children with moderate or low levels of risk factors. Because these children would be missed by selective programmes, the implication was that universal programmes were a priority. The reverse was the case for illegal drug use. But as this review argued, the evidence does not support mounting either type of programme exclusively.

Last revised 10 May 2009. First uploaded 10 May 2009

Open Effectiveness Bank home page

Top 10 most closely related documents on this site. For more try a subject or free text search

REVIEW 2007 Using correlational evidence to select youth for prevention programming

REVIEW 2015 Prevention of addictive behaviours

REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking

DOCUMENT 2011 European drug prevention quality standards: a manual for prevention professionals

STUDY 2003 Substances, adolescence (meta-analysis)

STUDY 2010 A brief image-based prevention intervention for adolescents

STUDY 2008 Independent review of the effects of alcohol pricing and promotion

NASTY SURPRISES 2004 Confident kids ... like to party

STUDY 2011 Effects of a school-based prevention program on European adolescents' patterns of alcohol use

STUDY 2008 Substance use outcomes 5½ years past baseline for partnership-based, family-school preventive interventions