Ringwalt C.L., Clark H.K., Hanley S. et al.
Prevention Science: 2010, 11(2), p. 172–184.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Ringwalt at firstname.lastname@example.org. You could also try this alternative source.
This real-world test of a prevention programme conducted by an independent researcher rather than the developer failed to replicate earlier positive results – in this case, in respect of Project ALERT, one of the two most widely implemented and respected US middle school drug prevention curricula.
Summary School-based drug prevention curricula constitute the USA's most common strategy to prevent adolescent drug use. This report evaluates the effects on adolescent substance use of one such curriculum, Project ALERT. The curriculum was considered promising by a leading US-government supported rating system and is one of the USA's two most commonly implemented middle school drug prevention curricula (the other is Life Skills Training). Applied to children aged between about 10 and 13, the lessons (11 in the first year and three 'boosters' in the second) seeks to motivate students not to use substances, to provide the skills to resist inducements from peers to use substances, and to support attitudes and beliefs that mitigate substance use. Teaching methods include guided class discussions, small group activities, role-playing exercises, and videos.
Project ALERT has been evaluated by a research team led by its developer with partially positive results, but another evaluation by an independent researcher not involved with the programme's development or dissemination found no statistically significant benefits. The featured study was the second independent evaluation and unlike prior studies, initiated the lessons in the US sixth grade among pupils generally aged 11–12. An earlier report based on findings after the completion of the two-year curriculum found that of eight substance use outcomes, the only statistically significant impact was that the proportion of pupils who had drunk alcohol in the past 30 days increased less steeply in Project ALERT schools. More specifically, though both ended at around 14% of pupils admitting past 30-day drinking, this represented a larger rise in controls schools because they started from a lower base. The featured report aimed to see if this impact had been sustained a year later (when the children would generally have been 13–14 years of age) and if there had been any delayed impacts on smoking, cannabis use and solvent misuse, the substances targeted by Project ALERT.
Schools serving the relevant ages from across the USA were invited to join the study, the main provisos being that if later randomly allocated to Project ALERT, they had to commit to teaching the lessons to all their sixth-grade (age 11–12) pupils, and that, apart from these lessons, they would not implement any evidence-based drug use prevention curricula in the sixth grade or higher. 34 schools agreed and met these and other criteria and (within school districts) were randomly allocated to Project ALERT or to act as control A group of people, households, organisations, communities or other units who do not participate in the intervention(s) being evaluated. Instead, they receive no intervention or none relevant to the outcomes being assessed, carry on as usual, or receive an alternative intervention (for the latter the term comparison group may be preferable). Outcome measures taken from the controls form the benchmark against which changes in the intervention group(s) are compared to determine whether the intervention had an impact and whether this is statistically significant. Comparability between control and intervention groups is essential. Normally this is best achieved by randomly allocating research participants to the different groups. Alternatives include sequentially selecting participants for one then the other group(s), or deliberately selecting similar set of participants for each group. schools against which the curriculum's impacts could be benchmarked.
A year later past 30-day drinking rate increases had evened out and were no longer held back significantly by Project ALERT's lessons. Including alcohol, none of the eight Proportions ever used or used in past 30 days in relation to each of the four substances. substance use outcome measures afforded even a marginal indication that the lessons had curbed the growth in the numbers of pupils having ever or recently used the four substances they were questioned about. The study also asked pupils questions which tapped risk and protective factors Editor's note: The featured study says the measures were as used in two previous studies of which the most complete description was in the study at web address http://dx.doi.org/10.1007/s11121-005-0015-0. This says: "We also examined a series of outcomes shown in the research literature to be mediators of substance use. These included future expectations to use, friends' approval of substance use, normative beliefs about substance use, perceived positive and negative consequences of substance use, beliefs about substance use harm, substance use addiction beliefs, exposure to substance use, and resistance efficacy." thought to underlie substance use, and did indeed find these were related to intentions to use and that intentions were related to use. However, these factors were not significantly affected by the lessons, perhaps the reason why they also failed to affect substance use.
The authors concluded that their findings do not support the long-term effectiveness of Project ALERT when delivered to sixth grade pupils. Among the possible reasons are that these pupils (the programme was originally developed for seventh and eighth grade pupils) are too young to get to grips with some of the lessons and/or to find the content relevant. Also, all the teachers were new to the curriculum and on average they only moderately adhered to the intended content and methods and taught in a way likely to engage pupils. On the other hand, the study gave the lessons extra chances to shine by ensuring saturation application in the relevant grades and that no other evidence-based approach in the control schools could compete with the lessons. More broadly, the authors argued that if evidence-based drug prevention curricula continue to fail to demonstrate effectiveness in large scale trials conducted by
Editor's note: A reference is being made here to the 'researcher allegiance' effect. In several social research areas,1 programme developers and other researchers with an interest in the programme's success have been found to record more positive findings than fully independent researchers. Such overlaps between developers and researchers are endemic2 in drug prevention research.
1. See articles at the following web addresses:
2. See article at the following web address:
http://dx.doi.org/10.1016/j.evalprogplan.2007.06.004 researchers, it may be time to develop and test more holistic approaches which integrate both environmental and curricular approaches.
commentary It might also be noted that the measures used in the featured study reflected the severity of substance use only to the degree that a child had used one of the assessed substances once in the past 30 days. Among the findings from a previous study were that growth in numbers regularly smoking or engaging in risky or harmful drinking had been curbed by the lessons.
Last revised 08 June 2011
STUDY 2000 Education's uncertain saviour
REVIEW 2015 Prevention of addictive behaviours