This entry is our analysis of a study added to the Effectiveness Bank. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study.
Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text
Copy title and link
| Comment/query | Tweet
Project SUCCESS' effects on the substance use of alternative high school students.
Clark H.K., Ringwalt L., Hanley S. et al.
Addictive Behaviors: 2010, 35, p. 209–217.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Clark at hclark@pire.org. You could also try this alternative source.
In what is becoming a pattern, this rigorous, real-world test of a prevention programme conducted by an independent researcher rather than the developer failed to replicate earlier positive results – in this case, in respect of an education/counselling programme for US teenagers diverted from mainstream schooling.
Summary Project SUCCESS is a substance use prevention programme which targets high risk secondary school pupils or those already using substances. It was developed for use in the US alternative high school system which teaches pupils who have exhibited truancy, academic failure, substance use, delinquency, or other problem behaviours which required them to be moved out of mainstream education, sometimes by order of juvenile justice authorities. The programme is mainly delivered to pupils in small groups and one-to-one. Its four components are: the Prevention Education Series, a four-topic substance use prevention programme taught to small groups over six to eight weekly sessions; individual and group counselling; communications with parents; and referrals to community agencies.
These are implemented in schools by professional counsellors from community-based prevention agencies. To build relationships, the counsellors engage in outreach to parents, pupils, and the community by participating in task forces and attending related school and community events. To expose every pupil to universal drug prevention, to reinforce the counsellors' credibility and rapport with pupils, and so they learn more about individual pupils, counsellors teach the Prevention Education Series to all pupils. As a basis for recommending at-risk pupils for further intervention, counsellors screen pupils individually to assess their own and their family's use of alcohol and other drugs and their need for professional treatment or other services. Students who need further attention may then receive individual counselling or participate in any of ten different types of small groups. Those who require more intensive services may be referred to community-based treatment providers.
We evaluated the effects of Project SUCCESS on adolescents' substance use immediately following programme implementation, and again one year later. Two successive cohorts totalling 14 alternative high schools were randomly assigned to Project SUCCESS or to act as control A group of people, households, organisations, communities or other units who do not participate in the intervention(s) being evaluated. Instead, they receive no intervention or none relevant to the outcomes being assessed, carry on as usual, or receive an alternative intervention (for the latter the term comparison group may be preferable). Outcome measures taken from the controls form the benchmark against which changes in the intervention group(s) are compared to determine whether the intervention had an impact and whether this is statistically significant. Comparability between control and intervention groups is essential. Normally this is best achieved by randomly allocating research participants to the different groups. Alternatives include sequentially selecting participants for one then the other group(s), or deliberately selecting similar set of participants for each group. schools, yielding seven schools per condition. The schools had a fluid pupil population attending for different reasons and at different times for different periods. On average pupils in Project SUCCESS attended about half the intended education sessions, 17% attended at least one group counselling session, and 36% at least one individual counselling session. As might be expected in a 'real world' trial, not all programme components were implemented as intended. For example, none of the counsellors ran a Newcomers group intended to ease transition in to the school, destigmatise the programme, and develop positive peer relationships. Just a fifth of pupils were formally screened by the counsellors for substance use risk.
Main outcomes included use in the past 30 days of alcohol, cannabis, or illegal drugs excluding cannabis, and drinking to intoxication. Past 30-day cigarette use was also assessed. At the start of the study around half the pupils had used each of cigarettes, cannabis and alcohol, and half had got drunk in the past 30 days. Nearly 4 in 10 had used other illegal drugs. Of the ten possibilities tested (five substance use measures at two follow-up times), the only statistically significant difference between Project SUCCESS and control schools was a greater reduction in the proportion of pupils who had used illegal drugs other than cannabis in the 30 days before the end of the year in which the programme had been implemented. No other differences at this time or a year later (when the difference in illegal drug use had all but disappeared) approached being statistically significant. Neither did an extra test reflecting the range of illegal drugs other than cannabis. It seemed possible that the results were due to pupils receiving few Project SUCCESS sessions. To test this, the sample was subdivided by their degree of exposure to the programme. Still there were no statistically significant outcome differences. There was also little indication that the programme worked for boys or girls of different racial groups and who drank at different intensities at the start of the study.
The authors concluded that the study did not provide evidence of Project SUCCESS' effectiveness, but that it might perform better in schools with higher and more regular rates of attendance. They noted that Project SUCCESS was not the first programme for at-risk secondary school pupils which has failed to show positive effects on substance use when evaluated independently (ie, not by the programme developer) in large-scale trials. Given these kind of contradictory findings, it has been argued that highly controlled trials of well implemented programmes conducted in one location
by developers
Editor's note: A reference is being made here to the 'researcher allegiance' effect. In several social research areas,1 programme developers and other researchers with an interest in the programme's success have been found to record more positive findings than fully independent researchers. Such overlaps between developers and researchers are endemic2 in drug prevention research.
1. See articles at the following web addresses:
http://dx.doi.org/10.1007/s11292-009-9071-y
http://dx.doi.org/10.1177/0193841X06287188
http://dx.doi.org/10.1093/clipsy.6.1.95
2. See article at the following web address:
http://dx.doi.org/10.1016/j.evalprogplan.2007.06.004
do not constitute sufficiently compelling evidence to support their widespread dissemination, nor should they qualify these programmes for inclusion on lists of effective programmes. Given the lack of consistent positive findings for these types of programmes, it is not surprising that there is ongoing discussion about the standards of evidence to which they should be held accountable.
Last revised 06 June 2011
Comment/query
Open Effectiveness Bank home page
Top 10 most closely related documents on this site. For more try a subject or free text search
STUDY 2012 Brief intervention for drug-abusing adolescents in a school setting: outcomes and mediating factors
REVIEW 2014 Interventions to reduce substance misuse among vulnerable young people
NASTY SURPRISES 2004 Confident kids ... like to party
STUDY 2010 What works? A 15-year follow-up study of 85 young people with serious behavioral problems
REVIEW 2015 Prevention of addictive behaviours
STUDY 2011 Effects of a school-based prevention program on European adolescents' patterns of alcohol use
STUDY 2010 A brief image-based prevention intervention for adolescents