Send email for updates
Manuel J.K., Hagedorn H.J., Finney J.W.
Psychology of Addictive Behaviors: 2011, 25(2), p. 225–237.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Manuel at firstname.lastname@example.org. You could also try this alternative source.
Does implementing evidence-based psychosocial therapies actually lead to the intended practice changes and do these make things better for the clients? From this review, most clearly when the whole organisation is enrolled in the effort and training is bolstered by systematic and expert continuing supervision.
Summary The main aim of this review was to analyse research on the implementation of evidence-based psychosocial or behavioural (not pharmacological) treatments in specialist substance use treatment services to identify factors associated with more and less successful implementation. Where this seems useful and important, the data given in the review has been supplemented by Findings based on the original studies.
First the review identified which treatments are generally agreed to be 'evidence-based'. Among these are behavioural couples therapy, cognitive-behavioural therapy (including relapse prevention), contingency management, motivational enhancement/motivational interviewing, and 12-step facilitation treatment. Brief alcohol interventions are also widely viewed as evidence-based. Despite their research backing, there remains concern about whether findings from trials of these therapies will be replicated in routine community care settings. Often trials have optimised the chances of the treatment working, for example, by excluding patients with co-morbid disorders or who are so socially unstable that they might be hard to re-contact. Once in treatment, efforts not typical in routine care may be made to promote attendance and therapists may be unusually well qualified, trained and supervised.
In 2007 a survey found that nearly all (96%) of US specialist substance use treatment services deployed two non-evidence based approaches – counselling and 12-step treatment. However, some evidence based approaches were widely said to be used "often", notably relapse prevention (91% of facilities), cognitive-behavioural therapy (69%), motivational interviewing (56%), and contingency management (20%).
The review then identified 21 studies of how successfully (based on therapist and/or client outcomes) evidence-based psychosocial or behavioural approaches had been implemented in specialist substance use services. In all the studies, implementation efforts included some type of workshop or didactic training. Just four studies were able to use client outcomes to assess the success of these efforts.
The accounts of the treatment providers and therapists themselves indicated that implementation programmes had enhanced knowledge and understanding of the new intervention and willingness to adopt it or led to actual adoption. However, two studies found such accounts were unrelated to how well therapists actually conducted the new intervention (in both cases, motivational interviewing).
Twelve studies avoided relying on provider accounts by actually observing interactions (or records of interactions) between therapists and real or simulated clients. Some found training did improve therapist skills, but that these skills decayed over time. An exception was a study which found that fidelity of implementation and skill levels increased in the follow-up period after training in cognitive-behavioural therapy, but only among therapists assigned to continuing post-training support in the form of supervision based on tape recordings of sessions or, less notably, continuing access to web-based training. Based on these studies, it seems that skill gains are not uniform across providers, nor are they necessarily sustained.
Of the four studies which assessed client outcomes, three found that (relative to former or usual treatments) these improved after therapists had been trained in new interventions. One found fewer positive urine tests among US cocaine dependent patients treated by therapists newly trained and expertly supervised in a network-based therapy compared to normal treatment alone. Another US study also featured post-training supervision, in this case based on tapes of therapy sessions. It found that patients of therapists trained in and who practised a therapy based on motivational interviewing were less likely to report use of their main substance than patients counselled in the usual way at the service. Moreover, the patients of more skilled and competent motivational therapists were less likely to test positive for illegal drugs and expressed greater increases in motivation. In a US study of Multidimensional Family Therapy for adolescent substance users, abstinence rates improved following implementation of the new approach. Again the implementation strongly featured post-training expert supervision based partly on actual therapy sessions. The sole exception to the improvement in client outcomes was another US study of the implementation of cognitive-behavioural therapy compared to usual treatment.
Another four studies not only offered trainees support additional to workshops, but tested whether these enhancements led to the expected improvements. In respect of motivational interviewing, skills of US therapists improved to a clinically significant degree only when workshops were supplemented by feedback on actual performance with clients and/or expert coaching. However, it took both coaching and feedback to increase the degree to which four months after training real clients expressed the desire and intention to change their substance use and reduced the degree to which they resisted such changes. In the context of US services for adolescents offering comprehensive family and related interventions, problem cannabis users were referred to a study of the introduction of contingency management techniques in to the programme. For randomly selected therapeutic teams, a workshop was supplemented by intensive quality assurance featuring (among other elements) weekly expert case consultation and quarterly booster training. According to reports from the children and their parents, these enhancements did increase the extent to which therapists implemented cognitive-behavioural elements of the training (and according to the children these changes were sustained) but not the core contingency management procedures of ensuring urine tests for drugs were conducted and followed by sanctions or rewards.
In contrast, two further studies of enhancements to motivational interviewing training found no benefits. One replaced the usual two-day workshop with short training sessions over a more extended period featuring feedback on the therapists' performance with simulated clients. This led to no temporary or lasting improvement in skills. Similarly in respect of US military substance abuse counsellors offered feedback on their sessions with a simulated client and regular expert phone consultations additional to a workshop. In both studies what was striking was that trainees whose attitudes to treatment were not conducive to adopting a motivational approach benefited relatively little even from the extended training and supervision.
In six of the 21 studies implementation efforts extended beyond work with individual therapists to organisation or systems levels, including supervisors, programme directors, or staff representation in the training and other efforts. Overall, these six studies had successful outcomes and most agencies reported adoption of the new treatment. Therapists were more likely to adopt the new evidence-based approaches if they had been supported or mandated to do so by agency leaders or supervisors, though in one study supervision, and therefore possibly treatment fidelity, decreased over time. However, just one of the six studies assessed whether client outcomes had improved as a result of the system-wide or organisation-wide efforts. It found that youth in family therapy were more likely to be abstinent from substance use after the implementation programme had been completed.
The review highlights the need for more conceptually driven, organisationally focused (not just individual provider-focused) approaches to implementation. It also raises the possibility that, at least in some situations, it may be more effective to implement evidence-based practices or processes rather than evidence-based treatment packages. In this scenario, training would emphasise not distinct treatment approaches, but common treatment processes, such as promoting support, goal direction, and structure in treatment and in clients' life contexts, enhancing clients' involvement in new rewarding activities, and building their self-efficacy and coping skills. Such an approach to implementation enables the new learning and skills to build on and be integrated with current approaches rather than replacing these approaches, and promises a more individualised delivery of treatment.
Across the 21 reviewed studies motivational interviewing or motivational enhancement therapy and contingency management were the most widely implemented evidence-based psychosocial treatments and workshops were universally used as the basic training vehicle. Most studies found that therapist knowledge and skills improved after these workshops but often whether these improvements were sustained was not measured and when it was, sometimes the results were negative. The implication is that training should be seen as an ongoing process, with continuous opportunity for discussion and learning, rather than a time-limited activity.
However, when such enhancements were trialled they were not always successful, though relevant studies were few. One factor which may partly account for this variability and other differences in the impact of training is therapists' pre-training general clinical skills and their skills in relation to the particular approach being implemented. It may be that a minimal level of motivation and skill is needed for the successful adoption of motivational interviewing (and other evidence-based treatments). On the other hand, very high pre-existing levels of motivation and skills related to the new approach may leave little room for training to generate further improvement.
Reviewed studies and other literature suggest that the chances of a successful implementation are raised when the entire agency is the focal point, rather than individual therapists. This process may be most successful when it begins with an assessment of the needs of the agency and its clients, and a discussion of how a new intervention may best fit into or be adapted for the agency. Training in the intervention should include ongoing supervision, coaching, feedback from taped sessions, and opportunities for discussions regarding implementation barriers. Agency administrative support and ongoing supervision may not be a panacea for provider conflicts and time demands when attempting to implement new treatments, but they can facilitate discourse between agency leaders and staff regarding providers' apprehensions or conflicts about learning a new treatment. Such support may be necessary to overcome barriers to implementation.
commentary The four studies which actually assessed whether clients benefited from the trialled implementation efforts go the heart of why such efforts are commonly mounted so are described here in greater detail. None afford an entirely consistent and convincing demonstration that implementing evidence-based therapies improves substance use outcomes for the clients. The evidence seems strongest for motivational therapies and those based on the client's social network. The clear failure was cognitive-behavioural therapy, one of the most widely implemented and respected approaches to substance use problems.
A study of network-based therapy found fewer positive urine tests among US cocaine dependent patients treated by therapists newly trained and expertly supervised compared to normal treatment alone. However, in this study the new therapy was additional to normal treatment, and neither patients nor therapists were randomly allocated, raising questions over whether the improvements were due to extra therapist contact time or differences between patients and therapists, rather than the impact of the training.
Another US study also featured post-training supervision, in this case based on tapes of therapy sessions. It found that patients of therapists trained in and who practised a therapy based on motivational interviewing were less likely to report use of their main substance over the 12 weeks (during which they may have continued to receive usual treatment at the service) following the end of the four-week trial treatment phase. However, they were not retained in treatment any longer, nor were their urines any more likely to be test free of illegal drugs during the four weeks of treatment. In this study the motivational intervention replaced usual counselling and both patients and therapists were randomly allocated, increasing confidence that the results were due to the training and implementation of the new therapeutic approach.
In the same study, the patients of more skilled and competent motivational therapists were less likely to test positive for illegal drugs during the four-week treatment phase and expressed greater increases in motivation. These associations were very modest, and most consistent between the use of fundamental and advanced motivational interviewing skills and greater competence on the hand, and the degree to which clients expressed increased motivation to reduce or stop substance use over the course of a therapy session. However, these findings from the four-week trial treatment phase were not accompanied by greater abstinence rates from the main drug the client was using in the following 12 weeks, creating a puzzling configuration of findings: being allocated to the new therapy improved main drug abstinence outcomes after the treatment phase, but not to any greater extent the more competently and skilfully it had been implemented.
There was a somewhat mixed picture too in a US study of Multidimensional Family Therapy for adolescent substance users. The key finding was that more young people became abstinent from a range of substances over a nine month follow-up period if they had entered treatment following implementation of the new approach. However, there were no such improvements in the frequency with which they used substances and the clinical significance of the increased rate of total abstinence is unclear. This was a very long-term study. Young people who started treatment before the training did so from 20 to 50 months earlier than those who entered treatment after training and researcher-backed supervision had ended. It was between these two phases that abstinence outcomes had most clearly improved to a degree which might have survived more rigorous statistical testing taking in to account the number of comparisons made in the study. But over such a period much can happen; in particular, the nature of the adolescent caseload changed significantly. Lastly, as the authors warned, this was one effort at one clinic, the results of which may not generalise to others.
The sole exception to the improvement in client outcomes associated with training was another US study of the implementation of cognitive-behavioural therapy compared to usual treatment. Even when this therapy was additional to usual outpatient treatment it created no added value in terms of abstinence from substances in general or the client's focal substance. Intense training and post-training supervision based on videos of the therapists' therapy sessions, and the fact that therapists all achieved at least adequate competence and skill levels, make inadequacies in the training an unlikely cause of the failure of clients to gain from the implementation. Failure to actually practice cognitive-behavioural therapy after the training can also be ruled since this was monitored and there were no client outcome differences between a high fidelity and lower fidelity implementation. At least two possible reasons for the negative findings remain. One that clients in the treatment-as-usual arm of the study improved so much that there was little room for extra improvement, possibly related to the criteria for joining the study which included a willingness to quit substance use rather than just cut back and a degree of social stability and connectedness. Among other criteria, these excluded over 8 in 10 of the patients screened for the study at an outpatient treatment service. The other possibility is that cognitive-behavioural therapy is actually no more effective than therapy as usually practiced at the clinics in the study. Supporting this interpretation, a synthesis of studies comparing this therapy with other treatments for alcohol or drug problems did not find it clearly more effective than other similarly extensive and coherent approaches.
Last revised 10 May 2013. First uploaded 12 November 2011
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates
REVIEW 2006 My way or yours?
REVIEW 2005 The motivational hallo
MATRIX CELL 2017 Alcohol Matrix cell B4: Practitioners; Psychosocial therapies
DOCUMENT 2007 Drug misuse in over 16s: psychosocial interventions
REVIEW 2018 Alliance rupture repair: a meta-analysis