Send email for updates
Tanner-Smith E.E., Wilson S.J., Lipsey M.W.
Journal of Substance Abuse Treatment: 2013, 44, p. 145–158.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Tanner-Smith at firstname.lastname@example.org. You could also try this alternative source.
Multi-prong therapies centred on the family emerge as probably the most effective in this comprehensive and careful synthesis of the results of trials of non-residential programmes for substance using teenagers – but do the outcomes warrant the extra costs?
Summary Causes and consequences of substance use disorders may differ for adolescents and adults. One implication is that evidence on the effectiveness of treatment for adolescents should be based on research conducted with adolescents, not inferred from research with adults. To this end, the featured review amalgamated findings from research on young people aged 12–20 to assess the relative effectiveness of different non-residential substance use treatments for young people.
Studies included in the analysis were required to involve an identifiable non-residential treatment for substance use problems for patients aged between 12 and 20.
Generally the substance use outcomes of the various distinct treatment types represented in the studies did not significantly differ.
The exception was family therapies, which returned the most convincing and consistent evidence of comparative effectiveness, but even here the evidence was too limited to support definitive conclusions.
The only relatively comprehensive previous review with a similar remit was limited to trials which randomly allocated patients to different treatments or no treatment, and analysis of treatment differences was limited to cognitive-behavioural therapy and two forms of family therapy.
Instead, the featured analysis aimed to analyse all treatment modalities represented in all the available studies capable of providing useful information about comparative treatment effects. It included not just randomised trials, but also non-randomised studies (though in the event there were few) which took steps to eliminate or adjust for pre-existing difference between participants allocated to different treatments or to treatment versus no treatment.
Studies included in the analysis were required to involve an identifiable non-residential treatment for substance use problems for patients aged between 12 and 20 who met diagnostic criteria for substance abuse or dependence. Restriction to non-residential was to avoid the differences in participants, treatment modalities, and treatment intensity between these and residential programmes. Studies had to have been reported in English in 1980 or after. Comparisons of very similar treatments were not included in the analyses.
An extensive attempt was made to extract data about research methods, outcome measures, and sample and treatment characteristics which might have affected outcomes and to adjust for these in the analysis. New techniques were used to amalgamate data on the multiple outcomes reported in many of the studies.
An extensive literature search found 45 published and unpublished studies reported from 1981 to 2008, nearly all of which had randomly allocated participants to (usually) alternative approaches or (less commonly) to treatment versus no treatment. Because some studies evaluated several treatments, the 45 studies yielded 73 different comparisons between treatments, or between treatment versus no treatment. Outcomes related to alcohol were most commonly reported followed by mixed substance use and cannabis. Nearly half the samples included youngsters with clinical levels of psychiatric comorbidity and most participants had some arrest or police-contact history. On average their substance use was of moderate severity. Treatment duration averaged about 10 weeks, during which patients were seen on average about twice a week. The follow-up assessment was on average just over five months after the pre-treatment assessment, ranging up to four years.
In addition to post-treatment assessments, reports on all but one of the studies enabled the analysts to assess changes in substance from before treatment to the post-treatment follow-ups. Both types of substance use outcomes – post-treatment use levels and differences from before to after treatment – were analysed, but in different ways. The analysis of post-treatment assessments was based on comparisons within studies, offering the reassurance of the steps taken by researchers in each study to level the playing field. Before-to-after treatment changes in substance use included comparisons across studies, making the analysis more vulnerable to bias.
Additional to no treatment at all, the types of treatment compared in the studies were categorised as either: behavioural therapies such as contingency management based on the principles of rewards, punishment, and reinforcement; cognitive-behavioural therapies; family therapies; generic counselling in a group, individual or family format which does not fall in other clearly defined categories; approaches based on motivational interviewing including motivational enhancement therapy, which develops motivational principles into a self-contained, manualised treatment programme; combined motivational and cognitive-behavioural strategies; psychoeducation involving teaching clients about substance use and substance-related issues; pharmacological therapies; skills training (eg, in relaxation) which does not fall in another clearly defined category; and practice as usual.
Some treatment types tended on average to show somewhat larger, smaller, or about the same effects as the aggregate of all the treatments with which they had been compared. However, most of these effect sizes were not statistically significant. Among the few exceptions was family therapy, which had a greater positive (and often significant) effect across all the comparisons in which it was involved. This was also the case for motivational enhancement therapies, though over half the relevant studies compared these programmes with no treatment at all rather than an active alternative treatment. Only if patients had been allocated to no-treatment control groups were their substance use outcomes consistently and significantly worse than other patients.
Examining the direction and magnitude of the average effect sizes for the different comparisons revealed a general pattern:
No treatment Patients allocated to no bona fide treatment at all did significantly less well overall than those offered treatment of some kind, and also less well than when allocated to each different treatment modality except generic counselling. With this exception, it seems treatment is more effective at reducing substance use than no treatment.
Psychoeducation, generic counselling, and practice as usual Outcomes from these treatments compared unfavourably with almost every treatment with which they had been compared. They may be more effective than no treatment, but the evidence for that is rather limited.
Cognitive-behavioural and motivational enhancement therapies separately or combined, behavioural therapies, and pharmacotherapies This set of interventions have generally proved better than their comparators. Cognitive-behavioural therapies have led to better outcomes than both the treatment types above. Motivational enhancement therapies have also bettered psychoeducation, generic counselling, practice as usual and no treatment to roughly the same degree as cognitive-behavioural therapies. Overall too, motivational enhancement therapies have proved significantly better than the aggregate of their comparators, but that is due to comparisons with no treatment; compared with other active treatments, there was no significant difference. Pharmacological treatments have only been compared with placebos, when the magnitude of their average benefits suggested effects may be similar to those of the other treatments in this group.
Family therapies Family therapies have the strongest evidence of effectiveness. They have produced better outcomes than every treatment with which they have been compared, including those in the preceding group, and overall have proved significantly better than the aggregate of their comparators. The effect size of 0.26 against all other treatments equates, for example, to a reduction from on average using a drug 10 days in the past month to six days.
A second analysis looked at each type of treatment separately and computed the degree to which patients allocated to that treatment had reduced their substance use from before to after treatment. The set of studies and treatments included in this analysis differed somewhat from that for the preceding analysis based just on post-treatment results, so the findings too might differ simply for this reason. Also this analysis entails comparing a treatment from one study to a treatment from another, depriving the results of the reassurance of the level playing fields intended to be created within each study.
Across all treatments patients reduced their substance use substantially, aggregating to an effect size of 0.52. Reductions were greatest in respect of cannabis (0.58) and mixed substance use (0.65), and smallest for alcohol (0.31) and other specific substances such as cocaine or heroin (0.13). Even after taking into account differences in study methods, participants, and treatment characteristics, the treatments in these studies had significantly larger effects on cannabis and mixed substance use than on alcohol and use of other specific substances.
Once other factors had been taken into account, neither frequency of contact nor the quality of the implementation of the treatment were significantly related to improvements in substance use. However, on average adolescents in longer treatment programmes improved less. For example, when all other factors were set to their average levels, the results meant that pre- to post-treatment substance use reductions in one-day programmes (brief interventions) equated to an effect size of 0.68, versus 0.61 for 30-day programmes, 0.45 for 90 days, 0.37 for 120 days, and just 0.14 for programmes lasting 210 days. Since comorbidity, delinquency, and pre-treatment substance use severity were controlled in these comparisons, poorer outcomes for longer programmes cannot readily be explained by their caseloads having more severe problems.
Finally the analysts investigated in which types of treatments patients made the greatest reductions in substance use. The analysis also took other factors into account, but this time not the intensity, duration or quality of implementation of the treatments, since these were considered features of the types of treatment delivered in the studies. Except for practice as usual (whose effect size of 0.25 narrowly missed being statistically significant), in all the treatment types on average patients made statistically significant improvements, which were at their greatest after generic counselling (0.65) and family therapies (0.64). Despite these variations, the treatment types did not significantly differ in their associated substance use reductions, and only family therapies were associated with significantly greater substance use reductions than was seen in the no treatment control arms of studies. Usual practice was associated with a substantially but not significantly lower effect than no treatment (0.24 v. 0.45).
With the evidence to hand it is not possible to reach definitive conclusions about the most effective treatment approaches for youngsters with different substance use problems and histories. Generally the substance use outcomes of the various distinct treatment types represented in the available studies did not significantly differ, though family therapies had clearly been the most effective.
To the question, ‘Does treatment work at all?’, it is possible to draw some conclusions. Four of the treatment types identified in the analysis were significantly more effective than no treatment across the studies which made these comparisons: generic counselling; cognitive-behavioural therapies; motivational enhancement therapies; and psychoeducation. Other types including family therapy, behavioural therapy and combined cognitive-behavioural and motivational therapies, had not been compared to no treatment by the cut-off date for the analysis. More comparisons (though also more vulnerable to bias) were made possible by computing pre- to post-treatment substance use reductions associated on average with each treatment type, adjusting these for known differences between participants and other features of the studies. Generally, reductions after active treatments were greater than after no treatment, but only for family therapy was this difference statistically significant.
Answering the question, ‘Which treatment works best?’, is more difficult. Many treatment types have not been compared head-to-head within the same study, and others have been compared in just a few studies, leaving considerable uncertainty about which are the most effective, not least because what they were compared against differs. Accepting these limitations, family and motivational enhancement therapies significantly improved on their various comparators. Other types of cognitive, behavioural and motivational therapies also improved on their comparators, but not to a statistically significant degree.
These patterns were largely replicated when comparisons were made between the pre- to post-treatment substance use reductions associated with each treatment type, adjusted for differences in studies and participants. Across both types of analysis, family therapies returned the most convincing and consistent evidence of comparative effectiveness. However, the evidence was incomplete: family therapies have not directly been compared with several other treatment types, and when they have, too few studies have been done to be confident about the results. It may be relevant to their apparent success that family programmes have a more extensive basis in research than many of the other treatments in the analysis, and especially well-developed treatment protocols.
For whom treatment works best can partially be answered. The one variable clearly associated with substance use outcomes was the substance at issue. Whether based on post-treatment results or changes from before to after treatment, reductions in substance use were smaller for alcohol and other specific substances (eg, heroin and cocaine) than for cannabis, use of which appears more responsive to treatment.
Additional to these findings on effectiveness, service planners will want to consider the costs of implementing the different treatment types. For instance, in the US Cannabis Youth Treatment Study the cost per day of abstinence gained was significantly higher for ‘branded’ family therapies relative to cognitive/motivational and social network approaches.
commentary The clearest implication of the featured analysis was the primacy of family-based approaches for problem substance users of an age when their parents and home circumstances would still be strong influences limiting or promoting their welfare. In other words, the superiority of family-based approaches not only tentatively emerges from the evidence, but also ‘makes sense’, especially since approaches prominent in the research also intervene beyond the family in other aspects of the child’s life which affect the welfare-generating potential of the family.
Published in 2017, what seems the most recent synthesis of research findings on family therapy for substance using teenagers (free source at time of writing) focused on multidimensional family therapy, one of the family therapies included in the featured review. Substance use outcomes were included along with other problems such as delinquency and mental health. The analysis found that overall the family therapy produced significantly better outcomes to almost exactly the same degree as found in the featured review, a small effect size of 0.24 overall and 0.25 for substance use. Effects were however inconsistent across the studies. One of the reasons for the variation was that the family therapy performed especially well when the sample included a relatively high proportion of children with severe substance use problems (before treatment using substances on more than 64 out of the past 90 days) or whose behaviour them led to them being assessed as pathologically at odds with family and society.
The featured analysis was unusual in how comprehensively it sought relevant studies and the detail with which it analysed their results in the attempt to eliminate bias and explain differences in effectiveness. However, it was hampered by the lack of direct comparisons between types of treatments. Only if these are compared within the same trial can steps be taken (like randomly allocating participants to alternative approaches) to even out all other influences except the difference in the type of treatment. When comparisons are made across studies, known differences can be adjusted for, but only imperfectly, and many factors possibly making the results of one study differ from those of another will not be known. Unfold supplementary text for more on this issue.
Outside the time frame of the featured review was a multi-national European trial conducted in Belgium, France, Germany, The Netherlands and Switzerland contrasting multidimensional family therapy with usual treatment at the clinics in the study. Participants were children aged 13 to 18 diagnosed with cannabis abuse or dependence during the past year, and with a parent figure who agreed to participate in treatment and in study assessments. Over the follow-up year continued cannabis use was the norm but on several measures improvements were significantly greater for patients allocated to multidimensional family therapy. However, the family therapy’s advantages were inconsistent. Of the three countries where participant numbers were highest and statistically significant results had the greatest chance to emerge, out of 12 possibilities there were three such results, each for a different country.
In terms of substance use and crime reductions, in The Netherlands one-to-one cognitive-behavioural therapy was found no less effective than the family therapy, though another analysis reports significantly greater improvements in health-related quality of life among the family therapy patients and the multi-national report revealed that the proportion of patients dependent on cannabis at the final follow-up fell significantly more steeply among patients allocated to multidimensional family therapy. Also there were indications that children with the severest problems reduced their cannabis use more when allocated to multidimensional family therapy.
One methodological feature not adjusted for in the analysis was whether the developer of the approach being tested in a study was themselves involved in its testing. The so-called ‘researcher allegiance’ effect is a concern in several social research areas (1 2 3), where programme developers and other researchers with an interest in a programme’s success have been found to record more positive findings than fully independent researchers.
This is particularly an issue in respect of the ‘brand name’ programmes prominent among the family therapies, some of which have mainly been evaluated by their developers and others with a stake in their success. Several studies led by researchers with no explicit interest in the programme’s success have not found (1 2) ‘branded’ family therapies more effective than alternative programmes.
However, a synthesis of findings on the most prominent of the family therapies for teenage substance users found that studies led by the developers of the approach had similar outcomes to those led by independent researchers. But it was also the case that the variability of results was far greater in the studies led by independent researchers, leading to a range in the effectiveness estimate which though centre on a positive impact, did not exclude the possibility that in reality effects were negative.
The cost-effectiveness issue alluded to in the featured analysis is made more important by the expense of family therapies relative to other approaches. However, extra cost might be considered worthwhile if these therapies gain greater remission in substance use and other problems or reduce costs for society more than the alternatives. From the little we know on this issue, family therapies for problem substance using youngsters in general have not been shown to be a good investment according to UK guidelines, but the scales may be tipped in their favour for the more multiply and severely problematic individuals and families, and over a longer time scale than has been studied. Unfold supplementary text for summaries of some relevant studies.
Over a child’s lifetime it is possible that cost-savings will accrue to family therapies to such a degree that they become clearly more cost-beneficial than other types of therapies, since the impacts tend to persist while relapse is the norm after approaches which do not address the child’s family and wider environment.
Cost-effectiveness and return on investment might be improved by reserving expensive family and environmental approaches for youngsters least likely to improve with less extensive intervention. Britain’s National Institute for Health and Care Excellence (NICE) has recommended multi-prong programmes centred on the family for problem-drinking children who also have other major problems and/or limited social support, signalling their particular suitability for the most severely affected and multiply problematic youngsters. In line with this recommendation, studies including European trials described above suggest that family therapies might be particularly effective for more multiply problematic youngsters.
These suggestions are tentative, primarily because the analyses of the severest cases were not planned in advance, so could have capitalised on chance variations in outcomes, though they gain credibility due to their plausibility and the emergence of supportive findings in both Europe and the USA. However, no such doubt undermines the findings of a synthesis of research on one of the family therapies which ( above) also found the approach especially effective for the most problematic youngsters.
Thanks for their comments on this entry in draft to review author Emily Tanner-Smith of Vanderbilt University in the USA and Henk Rigter of the University of Leiden in The Netherlands. Commentators bear no responsibility for the text including the interpretations and any remaining errors.
Last revised 31 March 2017. First uploaded 22 March 2017
Comment/query to editor
Give us your feedback on the site (one-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates
MATRIX CELL 2013 Drug Matrix cell A4: Interventions; Psychosocial therapies