Effects of the Positive Action programme on problem behaviours in elementary school students: a matched-pair randomised control trial in Chicago
Effectiveness bank home page. Opens new windowResearch analysis

This entry is our analysis of a study considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.

Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text
Copy title and link | Comment/query |

Effects of the Positive Action programme on problem behaviours in elementary school students: a matched-pair randomised control trial in Chicago.

Li K-K., Washburn I., DuBois D.L. et al.
Psychology and Health: 2011, 26(2), p. 187–204.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Li at ben.li@cityu.edu.hk.

In Hawaii and then the less promising schools of Chicago, a primary school programme aiming to improving school climate and pupil character development had substantial and, in Chicago, lasting preventive impacts – another illustration that focusing on drugs is not always the best way to prevent drug problems.

Summary The Positive Action programme to improve the climate in schools and pupils' social-emotional and character development includes a classroom curriculum, a school climate programme, and a parent and community programme, each working in synergy. The aim is that students become happier and more motivated, teachers freer to focus on teaching instead of classroom control, and head teachers become better leaders. Parents and community members get more involved in education, and schools become more effective.

Positive Action's philosophy is that people feel good about themselves when they adopt positive attitudes and take positive actions like respect, responsibility, self-honesty, self-improvement, a healthy, drug-free lifestyle, and academic achievement; thoughts lead to actions, and those actions lead to feelings about ourselves, which lead to more thoughts, in what can be a negative or a positive spiral.

Throughout the programme, positive actions for the body, mind, and feelings are presented in a framework of six unit concepts, with a seventh for review:
• self-concept: what it is, how it's formed, and why it's important;
• positive actions for your body and mind;
• managing yourself responsibly;
• treating others the way you like to be treated;
• telling yourself the truth;
• improving yourself continually;
• review.

Studies have suggested that the programme enhances academic achievement and school involvement and reduces disciplinary referrals and violence. A four-year trial in Hawaii matched pairs of schools and randomly allocated one of each pair to the programme. By the time pupils were in their fifth grade (age about 10–11) it found significant reductions in self-reported substance use, violence and sexual behaviours in Positive Action schools versus their matched control A group of people, households, organisations, communities or other units who do not participate in the intervention(s) being evaluated. Instead, they receive no intervention or none relevant to the outcomes being assessed, carry on as usual, or receive an alternative intervention (for the latter the term comparison group may be preferable). Outcome measures taken from the controls form the benchmark against which changes in the intervention group(s) are compared to determine whether the intervention had an impact and whether this is statistically significant. Comparability between control and intervention groups is essential. Normally this is best achieved by randomly allocating research participants to the different groups. Alternatives include sequentially selecting participants for one then the other group(s), or deliberately selecting similar set of participants for each group. schools.

The featured study aimed to replicate the Hawaii trial using a similar methodology, but among pupils with greater developmental disadvantages – primary school pupils in the publicly funded schools of urban Chicago. Seven matched pairs of schools joined the study and participated during the 2004/05 through 2006/07 academic years. They had been drawn from 483 schools, of which 68 met the study's criteria, including that their pupils were at relatively high risk Pupil roll mostly from poor families and the school has a relatively poor academic achievement record. of problem behaviours. Of these 68, 18 schools agreed to participate in the study on the understanding that they would be matched with another from the 18 and randomly assigned to implement or not the Positive Action programme. Seven pairs of schools could be adequately matched on indicators of pupil achievement, attendance, ethnicity, and family income, pupil and parental involvement, teacher qualifications, and the local crime rate. All 14 schools participated in the study.

Pupils in their third grade aged roughly eight to nine were followed up as they moved through the schools to the fifth grade. Incoming pupils who joined their classes were also included in the study, though for these pupils no baseline data was available (instead it was estimated) and they will not have received the full Positive Action curriculum. Nearly 80% of pupils and parents agreed to the pupils being included in the study. The featured report is based on the roughly 510 who filled in questionnaire surveys on substance use, violence, bullying and disruptive behaviours at the final assessment, It was only that this point that pupils were asked whether they had ever used drugs and their experience of serious violence, it being judged inappropriate to put these questions to younger children. of whom 290 were among the 590 in the study from the start; the remainder were pupils who joined the schools later.

Positive Action schools were offered the standard support package available from the company marketing the programme, including training by the developer, workshops for key players, individual consultations for teachers, and site visits and consultations with school leaders to assess and address obstacles to full and faithful implementation. Additional to this standard package were workshops targeted at the teachers of the pupils in the study to further enhance fidelity of implementation. The version tested in the featured study was intended to comprise over 140 15-minute lessons per grade taught four days a week.

Main findings

Over the three years of the study the quality with which the programme was implemented improved, until by the end of all but one of the seven schools achieved at least a moderate level.

After three years now aged about 10–11, pupils sampled from Positive Action schools were significantly less likely to say they had ever used alcohol or drugs than pupils in control schools. Specifically, they were asked if they had ever smoked a cigarette, drank alcohol, got drunk, used cannabis, or used other more serious drugs. Pupils who admitted to none scored 0, up 5 for those who admitted all five. Across all schools about 35% of pupils had used substances in at least one of these ways, but after taking in to account baseline differences, Including those estimated for pupils who joined the schools after the study started. scores in Positive Action schools were on average 31% lower than in their matched partner schools. Similarly they were respectively 36% and 41% lower for behaviour related to serious violence Including carrying or using a knife and gang affiliation. and for recently having bullied another child. Though 27% lower too for problem behaviours at school such as truanting and theft, this difference was not statistically significant.

There was no evidence that having been exposed to the full programme enhanced its impacts. Of the roughly 500 pupils who contributed analysable final follow-up data, about 280 had been in the schools from the start of the study so were (in the seven relevant schools) exposed to the full Positive Action curriculum. However, in terms of substance use, serious violence and problem behaviour at school, they benefited no more (in fact, non-significantly less) from the programme than pupils not exposed to the full curriculum, and were significantly less likely to have reacted to the programme by reducing the extent to which they had recently bullied other children.

The authors' conclusions

The featured trial in Chicago was the second to find the programme effective in reducing substance use and other problem behaviours among primary school pupils. At 0.27 to 0.41, the magnitude of the impacts expressed as effect sizes A standard way of expressing the magnitude of a difference (eg, between outcomes in control and intervention groups) applicable to most quantitative data. Enables different measures taken in different studies to be compared or (in meta-analyses) combined. Based on expressing the difference in the average outcomes between control and experimental groups as a proportion of how much the outcome varies across both groups. The most common statistic used to quantify this difference is called Cohen's d. Conventionally this is considered to indicate a small effect when no greater than 0.2, a medium effect when around 0.5, and a large effect when at least 0.8. are in the upper range found among such programmes, perhaps partly because the programme is intensive and the pupils, due to their often unpromising situations, were in a relatively good position to benefit.

Among the limitations of the study were the fact that baseline data had to be estimated for about half the pupils. It is also unclear why pupils fully exposed to the programme did not benefit more than those only partially exposed.


Findings logo commentary This study joins a small set which have found sometimes substantial substance use prevention impacts from programmes implemented in the early years of schooling which are not focused on substance use at all, but on overall child development and the prevention of a spiral of deviancy and alienation. A later report from the study yet to be published will focus on substance use and extend the follow-up to US grade eight, roughly age 13–14. By this stage just a fifth of the original grade three pupils were still at the schools. Nevertheless, amalgamating results from new and retained pupils, those in Positive Action schools were 20%–39% less likely to have ever used tobacco, alcohol, or cannabis than those in control schools, and also less likely to have got drunk. As in the featured report, the effects were just as great among the new pupils. The differences extended to the most serious forms of substance use recorded by the study, including having got drunk more than once (9% v. 15%) and used cannabis at least twice (11% v. 16%). Grade eight pupils in Positive Action schools also scored higher on a measure Reflecting prosocial interactions, honesty, self-development, self-control, respect for teachers, and respect for parents. of social-emotional and character development. This measure declined over the years of the study but less steeply in Positive Action schools, an effect which seemed to account for the impacts of the programme on substance use.

Especially given the impacts on incoming pupils, programmes like Positive Action may have a place in the first years of secondary schooling in the UK as well as in primary schools. Such programmes attract because they promise wide-ranging benefits in areas other than substance use such as the prevention of crime and violence and of mental health problems. Beyond specific programmes is the finding that substance use and problems are lower in schools distinguished by the high degree to which they productively engage pupils in their education and/or create a sense of being part of a valued school community – a sense promoted by warm and supportive schools with a caring, inclusive ethos, which emphasise prosocial values, encourage cooperation, show concern for pupils as individuals, allow pupils to participate in decision-making, and offer extracurricular activities.

Methodologically the featured study improved on the previous trial in Hawaii. In particular, It also used two-tailed tests of whether its findings were statistically significant, a procedure which does not in advance effectively deny the possibility of meaningful negative findings. it recruited schools to the study before they knew whether they had been allocated to Positive Action. In the Hawaii study, some schools dropped out on hearing they would be expected to implement the programme, possibly whittling down Positive Action schools to only those particularly willing to devote time to this kind of effort. In Chicago that did not happen, preserving the initial presumption that randomisation led to schools which were equivalent on unmeasured variables like their focus on pupil character development. Unlike the Hawaii study however, it did not provide the raw data from which its findings were derived, leaving it unclear whether more or less serious forms of substance use were most affected and how many pupils (as opposed to the ratio of pupils in Positive Action v. control schools) were prevented from engaging in these behaviours. The later report referred to above remedied this gap. Further methodological considerations below.

In both studies the programme developer was involved in training staff, an advantage which would presumably be hard to sustain if the programme were more widely implemented. Both too are vulnerable to the 'allegiance effect' – the tendency In several social research areas,1 programme developers and other researchers with an interest in the programme's success have been found to record more positive findings than fully independent researchers. Such overlaps between developers and researchers are endemic2 in drug prevention research.

1. See articles at the following web addresses:
http://dx.doi.org/10.1007/s11292-009-9071-y
http://dx.doi.org/10.1177/0193841X06287188
http://dx.doi.org/10.1093/clipsy.6.1.95
2. See article at the following web address:
http://dx.doi.org/10.1016/j.evalprogplan.2007.06.004
for studies conducted by investigators with a stake in the intervention to produce more positive results, a vulnerability they share with much other prevention research. Both too recruited by definition only schools willing at least in principle to devote considerable time to their pupils' broader development rather than their academic studies. As long as schools self-select for such programmes, this merely reflects the reality that only willing schools would take them on. However, these studies may not be a good indicator of how well the programmes would work if education authorities tried to maximise their coverage by mandating universal implementation.

Unlike in Hawaii, where exposing children to the programme for an additional one or two years appeared to reduce negative behaviours by half, in Chicago being exposed to more of the programme did not enhance its effectiveness. Rather than only limited and short-term implementation being adequate, the authors' favoured explanation is that new pupils rapidly adjusted to changes in the school-wide climate generated over the years of the programme, including the behaviour of pupils and reinforcement of positive behaviours by teachers. Generally in intervention research, a finding that more of an intervention leads to greater impacts is interpreted as further evidence that the intervention is indeed responsible for the changes. That this has not been a consistent finding in respect of Positive Action is puzzling but not altogether surprising given the differences in the Chicago and Hawaii settings. Also, though incoming pupils will have missed out on some of the Positive Action curriculum, they will have become rapidly immersed in the overall climate and behaviour norms and expectations which that curriculum and the other elements of the programme helped to establish.

Thanks for their comments on this entry in draft to Brian Flay of Oregon State University and Adam Fletcher of the London School of Hygiene & Tropical Medicine. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Last revised 13 December 2011

Comment/query
Open Effectiveness Bank home page


Top 10 most closely related documents on this site. For more try a subject or free text search

REVIEW 2011 The Good Behavior Game and the future of prevention and treatment

STUDY 2009 Evaluating mediators of the impact of the Linking the Interests of Families and Teachers (LIFT) multimodal preventive intervention on substance use initiation

STUDY 2008 Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes

HOT TOPIC 2017 It’s magic: prevent substance use problems without mentioning drugs

REVIEW 2015 Prevention of addictive behaviours

REVIEW 2012 An overview of prevention of multiple risk behaviour in adolescence and young adulthood

HOT TOPIC 2016 Should we start prevention in the cradle?

STUDY 2010 Does successful school-based prevention of bullying influence substance use among 13- to 16-year-olds?

STUDY 2008 Substance-focused initiatives not only way schools help prevent risky substance use

REVIEW 2011 Early intervention: the next steps. An independent report to Her Majesty's Government