Conrod P.J., O’Leary-Barrett M., Newton N. et al.
JAMA Psychiatry: 2013, 70(3), p. 334–342.
Positive and significant effects observed on a range of drinking outcomes suggest that high-risk pupils may have benefited from the psychosocial intervention over standard drug education. But unclear whether the personality matching strategy was the effective component.
Summary The Adventure Trial tested the impact of a personality-targeted intervention delivered by teachers on substance use and misuse among pupils in London. This followed the reported success of the Preventure trial, which found that personality-targeted interventions were effective in reducing motivations for drinking that involve coping with negative feelings, and reducing symptoms of problem drinking during a two-year period.
The Adventure Trial examined whether personality-relevant coping skills could prevent drinking problems among London high school pupils.
High-risk young people were allocated to receive an intervention matched to their specific vulnerability (‘hopelessness’, ‘anxiety-sensitivity’, ‘impulsivity’, or ‘sensation-seeking’).
Long-term benefits were observed, with reduced odds of high-risk pupils in the intervention group reporting drinking, ‘binge’ drinking , and problem drinking, compared to peers allocated to receive standard drug education.
The focus of the Adventure Trial was on high-risk young people who had one of four dominant personality traits, each associated with vulnerabilities that make early-onset substance use and other risky behaviours particularly rewarding or hard to resist:
• Hopelessness: A tendency to unhappiness, depression and feeling a failure, feelings relieved by intoxication.
• Anxiety-sensitivity: Fear of anxiety-related bodily sensations due to beliefs that such sensations will lead to catastrophic outcomes, for which substance use can represent a form of self-medication.
• Impulsivity: An inability to restrain seeking gratification in the presence of immediate rewards (such as the feelings available through substance use) despite longer term negative consequences.
• Sensation-seeking: Desire for intense and novel experiences, which can be expressed as a desire to ‘get high’ through drugtaking or heavy drinking.
A total of 21 secondary schools signed up for the study, representing 14% of the 148 schools approached. They were spread across 18 London boroughs and located in both densely populated, low-income areas and suburban areas. All the schools but one were state-funded.
The recruited schools were randomly assigned to deliver either the Adventure intervention (plus the usual drug education mandated in the national curriculum) or usual drug education alone. The latter acted as the control condition. One control school withdrew from the study and one intervention school was excluded because it could not commit to the full trial protocol.
Year-nine pupils in participating schools, with an average age of pupils was 13.7 years, were surveyed. If they scored above the school average for one or more of the personality traits listed they were considered high-risk. In Adventure schools they were allocated to the intervention matching their dominant personality trait. Low-risk pupils were the remaining 55% of the year-nine population who had consented to the study but who did not meet personality risk criteria and, therefore, did not receive interventions. All high-risk pupils who consented to take part were included in the analysis regardless of whether they attended the sessions or not. All high-risk and low-risk pupils (2643) were invited to participate in follow-up assessments at six-monthly intervals for two years.
The researchers and participants knew which group they were assigned to (this is known as an ‘open-label trial’), but assignment was masked from young people and teachers who did not participate in the programme, and young people participating in interventions were not informed of other interventions being offered and which of their peers participated in these other interventions. Those conducting follow-up sessions and quality control of data were also blind to intervention status.
Each intervention consisted of two 90-minute group sessions, and incorporated components from motivational enhancement therapy and cognitive behavioural therapy for depression, panic anxiety, and impulsivity. Teachers and other staff were supported to deliver the intervention with training and an accompanying manual. What was unique about the intervention was that it targeted personality traits rather than problems. Alcohol and drug use were only a minor focus.
The featured paper reports on the 24-month drinking outcomes of the Adventure Trial – drinking frequency, drinking quantity, ‘binge’ frequency, and problem drinking symptoms – which were calculated using statistical models that examined the average effect post-intervention (6–24 months), and the degree to which the effects grew or diminished during this time. In each case pupils were asked about their behaviour and experiences over the past six months and the results were divided into whether these events had occurred at all, and if they had, their severity or frequency.
Over the entire post-intervention follow-up period, the chances that a high-risk pupil would engage in drinking or ‘binge’ drinking, or report symptoms of problem drinking, were significantly greater than among low-risk pupils, seeming to validate the personality measures as an indicator of risk. However, this result was inconsistent, especially in schools allocated to the Adventure intervention, where supposedly high-risk pupils were not significantly more likely to ‘binge’ or to report symptoms of problem drinking.
Results indicative of the Adventure programme having restrained the development of drinking were broadly apparent among the high-risk pupils, but still of the 16 possible findings, it seems that just half were statistically significant.
Across the follow-up period, compared to high-risk peers in control schools, high-risk pupils in intervention schools had 29% lower odds of reporting drinking versus non-drinking. Corresponding figures for ‘binge’ drinking were 43%, and for problem drinking, 29%. No such finding was reported for the chances that a pupil will have consumed a “full drink” in the past six months – though if they had, Adventure pupils on average had drunk less. The intervention was also found to delay the natural progression to more risky drinking behaviour, including engaging in ‘binge’ or problem drinking and (among pupils who had experienced these) the increase in the amount of “full drinks” drunk on a single occasion and in the frequency of ‘binge’ drinking.
The analysts also explored whether the Adventure programme led to effects across all the pupils in the study, and among the low-risk pupils not offered the intervention but potentially affected by their higher-risk schoolmates who had been through the programme. There were some statistically significant findings in favour of the intervention, but the great majority of the findings did not evidence any significant advantage for the Adventure schools.
Across both high- and low-risk pupils in the study, over the follow-up period pupils in Adventure schools were significantly less likely to have drunk at all in the past six months, and nearly significantly less likely to have consumed a “full drink”, but no less likely to have ‘binged’ or to report symptoms of problem drinking. Among pupils who had experienced them at all during the past six months, generally the severity/frequency of these events was no less in Adventure schools, and nor did it progress less steeply over time.
Significant effects were much less likely among the low-risk pupils not exposed directly to the Adventure programme, but there were some. Across the entire follow-up, in Adventure schools they were less likely to have drunk at all in the past six months and progressed less steeply in their uptake of ‘binge’ drinking, until by the end of the follow-up period the odds of their having ‘binged’ over the past six months versus not were 35% less than among control low-risk pupils. But among pupils who reported these events at all in the past six months, their severity/frequency was no less in Adventure schools.
The overall picture was of long-term effects on drinking outcomes among high-risk pupils allocated to receive the personality-targeted intervention. These findings, the authors argue, “further support the personality-targeted approach to alcohol prevention” (1 2 3), and in particular its effectiveness when administered by appropriately trained school staff.
There were also novel findings of mild indirect effects beyond the population selected to receive interventions. The results suggested a delayed intervention effect in low-risk young people that gradually increased from six to 24 months, so that by the end of the trial, the low-risk intervention group had reduced odds of ‘binge’ drinking relative to the low-risk control group.
commentary Positively appraising the Adventure Trial, lead researcher Dr Patricia Conrod emphasised both the significant outcomes for high-risk pupils, and the trickle-down effect on their low-risk peers (the phenomenon referred to as the ‘herd effect’). Nick Barton, who at the time was Chief Executive of Action on Addiction which commissioned the trial, said the Adventure Trial was an “exciting development for prevention work in the UK … generally recognised as inadequate.”
The Adventure Trial was the third in a series of studies from Conrod and colleagues testing the effectiveness of school-based substance use interventions matched broadly to the high-risk personality traits of pupils, “in recognition of what appears to be personality-specific motivational
pathways to risky drinking in adolescence”. The published findings have focused on a number of positive and significant outcomes, which have given credence to the developers’ hypothesis that a personality-targeted approach would be an effective way of preventing and reducing substance use problems. However, the studies weren’t designed to detect whether the personality matching strategy was the effective component, because they did not determine whether an intervention matched to pupil characteristics would have outperformed the same or a similar intervention not matched to pupil characteristics. An earlier paper by the lead author acknowledged this point:
“ … To provide a stringent test of the efficacy of a client-treatment matching strategy, the same intervention must be administered to a control group in a manner that would theoretically be considered mismatched. Without such a control condition, the matching strategy cannot be proved effective over and above the effects of the treatment techniques per se.”
Comparing the intervention with no intervention (just the standard drug education every child received), meant that any results could have been due to the characteristics of the intervention itself (eg, being among a selected group of pupils, being listened to by an attentive adult), rather than the matching strategy. This doesn’t undermine the findings of the trial, but does limit the conclusions that can be drawn from the findings.
To our knowledge, the only study from the researchers that compared a matching with a non-matching strategy found no significant differences between them. It involved largely alcohol-dependent women in Canada aged 30 to 50, recruited via ads asking them to get in contact if they were concerned about their drinking or prescription drug use. A variant of the featured intervention was compared to a control intervention involving a motivational film on substance use problems and a supportive discussion with a therapist, a combination which it fairly consistently outperformed in reducing substance use. However, there were no statistically significant findings (though there were tendencies in this direction) indicating that the intervention bettered another intervention similar in every other way except that the content was not matched to the individual’s personality profile. These findings call in to question the matching strategy which above all distinguishes the featured intervention from other approaches.
Four key outcome measures were examined in the featured paper – drinking frequency, drinking quantity, ‘binge’ frequency, and problem drinking symptoms – and (to our calculations) there were 16 chances to find a statistically significant difference among the high-risk pupils. Of these, half (8 of 16) were positive and significant. The authors didn’t indicate which, if any, of the measures was the most important, though in the UK context where some under-age drinking isn’t unusual, it could be presumed that the biggest concern would be ‘binge’ drinking. Whether the intervention could be deemed a success on that front is unclear. While three out of four of the figures for ‘binge’ drinking were significant, the remaining non-significant figure suggested that among high-risk young people already ‘binge’ drinking, the intervention did not have a significant effect on average severity across the period 6–24 months post-intervention.
A small number of schools were recruited to the Adventure Trial, and spread across a similar number of London boroughs. This would have been a challenging context for the researchers to establish equivalent intervention and control group samples. And indeed when they analysed the characteristics of the groups, the make-up of students by ethnic background differed considerably, with 41% of adolescents in the intervention schools reporting white or European ethnic background versus 25% of adolescents in control schools. Differences in factors such as ethnicity and socioeconomic status could have influenced the development of substance use, the results of the study, and at the very least meant the researchers weren’t comparing like-for-like. In other situations, this could have been mitigated by researchers creating ‘matched pairs’ of schools based on core characteristics – randomising one school within each pair to the intervention condition and the other to the control condition. But in this case the strategy would have been difficult to apply as most schools would still have been in different boroughs, with different social, cultural and economic characteristics.
The Preventure and Adventure trials tested a selective intervention, targeted at a high-risk pool of young people, as opposed to the whole population. This focused attention and resources on those perceived to be most in need. But, as one well-informed and clear analysis concluded, we may not yet be able to predict future substance use well enough to risk leaving some people out.
Data from 940 studies tracking the development of cohorts of young people was used to test the extent to which standard risk and protective factors were related to alcohol, tobacco, and cannabis use. Most of the factors were indeed related to substance use, some fairly strongly, but on average relationships were weak. Some factors were not related to use, and a few were related in the ‘wrong’ direction. This predictive weakness was fundamental to why the paper advocated for persisting with universal prevention efforts.
In the featured study itself the supposedly high-risk pupils were not always found to be drinking more or more severely than their low-risk counterparts. And while the observed indirect ‘herd’ effect was promising, at this stage the findings are not convincing enough to warrant failing to intervene with pupils whose personalities supposedly place them at lower risk of problem drinking.
The original study protocol did not specify the measurement of an indirect effect. However, the amended protocol (dated after the data had been collected) did. It said “the remaining 55% of low risk students [would be] followed to examine population-level effects of the intervention”, though not as a primary or secondary outcome. This was one of several additional analyses beyond the scope of the primary outcomes. Specifying multiple supplementary analyses like this increased the researchers’ chances of finding a statistically significant result purely due to chance variations in the samples.
The shorter-term drinking outcomes and longer-term cannabis outcomes of the Adventure Trial have previously been examined in the Effectiveness Bank. Looking at these papers as well as the present one, it seems that not all the outcomes of the trial were reported, or reported in a way that corresponded with the original study protocol. For example, instead of reporting against general “drug use and misuse” (as was specified in addition to alcohol use and misuse), only the findings for one type of drug (cannabis) have been published so far. Between the original study plan, and the time the data had been collected, it seems for unexplained reasons the focus shifted from drug use as a whole to cannabis use, and instead of the planned focus on high-risk pupils as a whole, the samples were broken down further to examine the effect among each of the four personality groups, revealing a significant effect among one sub-group – sensation-seeking high-risk young people. The multiple outcomes here increased the chances of finding a statistically significant result purely due to chance variations in the samples. The possibility cannot be excluded that unplanned further analyses were conducted after the data had been collected to produce findings in favour of the intervention, when if only the planned analyses had been conducted, it would have to have been declared a failure.
The three papers to date that have reported on the Adventure Trial used different methods to analyse the data. The featured paper used something called “two-part latent growth modelling”. Had this been the only method used in the cannabis outcomes study, there would have been no statistically significant findings for high-risk pupils as a whole. The short-term drinking outcomes paper did not use two-part latent growth modelling, but instead the other methods which produced significant results in the cannabis paper – “logistic and linear regression analyses”. No explanation was given for the different analytical strategies – for example whether there was a reason for changing methods between the publication of papers. It means that the possibility cannot be excluded that methods were chosen on the basis of what ‘worked’ (ie, was both an appropriate method and one more likely to reveal significant effects).
Some of the researchers from the Adventure Trial were also involved in the intervention’s initial development and testing in the UK Preventure Trial, and the Canadian trial before it. This makes it difficult to rule out the possibility of some ‘researcher allegiance’ effects. This refers to the possibility that researchers with an investment in the outcomes of the trial may (perhaps inadvertently) influence those outcomes and/or the way they are presented in ways which match their expectations or interests. Researcher allegiance can manifest itself in a number of ways, making it more likely that positive findings will be reported than research which is fully independent, for example through: less rigorous testing/analysis (only a selection of the results being published, methods of analysis that are more liberal with estimates of statistical significance); and looking for ways to show the intervention works rather than testing effectiveness (multiple outcomes or subgroups, and post-hoc analyses). It could be argued that some examples of this were evident in the featured study.
The previously mentioned Preventure Trial was also set in London, among a highly selected set of 347 schoolchildren counselled by a professional psychologist rather than school staff. The intervention was associated with drinking reductions six months later, but these effects dissipated to insignificance over the next six months and remained so over the remainder of the two-year follow-up. This was in contrast to drink-related problems, experience of which increased over the first six months in the control group and remained higher than in the intervention group over the follow-up period. A similar study found that over the following six months the intervention delayed the expected increase in drinking among high risk pupils over the first six months of the follow-up, though again, by a year there was no significant difference in the drinking behaviour of pupils who had or had not been allocated to the intervention. The same trial found reduced uptake of cocaine and other drug use and a reduced frequency of drug use overall (but not cannabis in particular) over the two-year follow-up. In Canada too, the intervention was found to result in at least short-term (four months) drinking reductions in secondary school pupils.
Last revised 25 May 2017. First uploaded 02 May 2017