Effective dissemination: a systematic review of implementation strategies for the AOD field
Effectiveness bank home page. Opens new windowReview analysis

This entry is our analysis of a review or synthesis of research findings considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original review was not published by Findings; click Title to order a copy. The summary conveys the findings and views expressed in the review. Below is a commentary from Drug and Alcohol Findings.

Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text
Copy title and link | Comment/query |

Effective dissemination: a systematic review of implementation strategies for the AOD field.

Bywood P.T., Lunnay B., Roche A.M.
Adelaide: National Centre for Education and Training on Addiction, 2008
Unable to obtain a copy by clicking title? Try this alternative source.

Comprehensive Australian review garners the lessons from across health promotion and medical care on how best to improve practice by introducing research-based innovations, and evaluates their applicability to substance misuse.

Summary Australia's National Centre for Education and Training on Addiction acts as the country's centre for the development of the workforce responding to substance use problems. To help identify the best ways to narrow the gap between research-based knowledge and its application in practice, the centre undertook a systematic literature review of the most common strategies to enhance uptake of innovations in professional practice. Though most of the evidence derived from non-substance specific health and medical fields, the review sought to determine its applicability to substance use.

A literature search found 25 relevant systematic reviews plus 85 additional primary studies. These were used to assess 16 dissemination and implementation strategies in terms of changing professional practice and the impact of these changes on welfare and substance use. As well as rarely being focused on substance use, the evidence was typically of average quality and most studies were prone to some degree of bias. Nevertheless, findings from better quality studies indicated that some strategies can improve professional practice in sectors including:
• Preventive care; for example in the substance misuse sector, advice on smoking cessation or substance use in pregnancy or on the risks of alcohol or drug use more generally, and screening for actual or potential problems;
• Treatment; for example, methadone or nicotine replacement and psychosocial interventions like brief advice to risky drinkers, motivational interviewing and cognitive-behavioural therapy;
• Disease management; for example, management of patients on pharmacotherapies, of co-occurring psychological problems, or of chronic illness related to substance use such as hepatitis C and HIV infection and cirrhosis of the liver;
• Rehabilitation; for example, management of relapse in dependent clients; and
• Palliative care; for example, for clients with terminal illnesses related to substance use.

Of the 16 evaluated strategies, the four most successful across these clinical sectors are described below.

Interactive educational meetings are often conducted under the umbrella of continuing medical education. Examples include educational conferences, meetings, seminars, workshops, lectures and symposia. These resulted in small to moderate improvements in practice, particularly when participants were starting from a relatively low base, and sometimes (but inconsistently) were also shown to have improved patient outcomes. Interactive or personalised formats worked best. Improvements were most likely when meetings were followed up for example by telephone consultations, and participants were incentivised through mechanisms such as feedback on performance and medical education credits.

Educational outreach visits, also known as 'academic detailing', involve a trained health educator or specialist (typically a physician or pharmacist) visiting health care providers in their own workplaces to inform them about the evidence for a specific intervention or practice guideline with a view to encouraging its uptake. In general, these have improved practice in a range of settings and when they did so, the impacts were moderate to large. The few studies which investigated this generally found no impact on patient outcomes. Again, an interactive format and follow-up contacts helped.

Prompts and reminders (computerised or manual) prompt the practitioner to take certain actions or record key information when research suggests these are warranted by the situation and will improve patient management. They may be single-action systems such as prompts to deliver a brief intervention when screens indicate risky drinking, or may be elaborated in to decision support systems recommending a series of actions from a number of alternatives. Studies recorded mixed effects in respect of both practice improvements and patient outcomes. Practice improvements were commonly noted in preventive care, prescribing and disease management, but less so in diagnostic practice, while the few studies of patient outcomes rarely demonstrated significant positive effects. Systems worked best when the clinician could not suppress the prompt/reminder and was obliged to respond, even if only to acknowledge it.

Audit and feedback involves gathering information For example, from medical records, databases, patients or by direct observation. on a practitioner's performance, comparing this against a benchmark such as their peers' performance or clinical guidelines, and feeding back the results to the practitioner with or without recommendations. The rationale is that practitioners who are made aware of suboptimal behaviour will seek to improve. Research shows that commonly they do improve (though to a highly variable degree), but the few studies which measured patient outcomes found little or no impact. Impacts on practice were greatest among practitioners whose initial performance was least satisfactory, and when feedback was relatively intensive. Intensive feedback might, for example, entail feeding back verbally to each individual or prolonged feedback from a senior colleague. Non-intensive feedback might entail just written feedback, feedback to groups, from a less experienced colleague, or lacking personal incentives for improvement.

Cutting across the different types of dissemination strategies are some common features both of the strategies and of the contexts in which they operate which may facilitate or inhibit implementation. The most successful strategies provide clear and succinct messages, with simple, focused objectives, are reinforced by additional materials and support, and require small, practical changes. Effective messages are presented by reliable and credible sources (backed by research-based information) in an appealing and persuasive interactive format which encourages participation, are tailored to the local setting, and relevant to the needs of practitioners and clients. Systems or procedures are most likely to be implemented if they are accessible, demand little effort to comply, but require the practitioners to respond. Successful strategies are based on an assessment of the barriers to change, address change at multiple levels, including the individual practitioner, organisational structure and culture, and health system policy, and are sustainable over a prolonged period.

Contextual factors which may enhance effectiveness include identifying the need for change, communicating this to the target audience in a way which motivates change, providing resources and staffing to integrate change into existing systems, and ongoing monitoring of how well an innovation is being implemented to identify and correct 'drift'.

However, the reviewers cautioned that the evidence base was of only moderate quality, few studies assessed impacts on clients, and those which did found little or no benefits. This may have been because studies were typically conducted over a short time period; longer follow-up and reinforcement of change may be needed to detect such benefits. Since no single strategy proved effective in all situations, careful selection is required to ensure the best match with the topic or behaviour in question and the people and institutions targeted for change.

Findings logo commentary The featured review was comprehensive and seems to have truly deserved the term 'systematic' in its title, but was hampered by the quality of the evidence and its lack in the substance misuse sector. Even more hampered was an accompanying report which extracted cost-effectiveness data from the same set of studies reviewed in the featured report. It found that of the 25 reviews and 85 additional studies, just two reviews and 14 studies reported costs, and generally the methodological quality of the economic analyses was poor. Nevertheless it was concluded that educational meetings under the rubric of continuing medical education were generally effective and cost-effective, while educational materials alone were relatively cheap but also relatively ineffective.

The review focused on formally evaluated strategies, which have mainly been implemented at the level of the service or individual practitioner. As it acknowledged, beyond these are more macro health system and administrative structures which can themselves be agents for change. These and other factors also critically influence whether the reviewed strategies have a chance of making a difference. Some examples below.

Outside the review's scope were more macro, system-wide change efforts driven for example by national priorities and targets, backed by incentives or sanctions to promote those policies. Examples in Britain have included the numbers-in-treatment and waiting time targets which have improved treatment access. Similarly, in the US state of Delaware, state treatment recruitment and engagement targets promoted through financial sanctions and incentives applied to services resulted in an 87% increase between 2001 to 2006 in the average number of patients in treatment.

The Delaware study is an example of the harnessing of market mechanisms to public priorities. Other market mechanisms which contain costs but also require the demonstration of value and quality also have an impact. Most researched have been US managed care cost-containment mechanisms, and US quality-assurance accreditation systems, which seem to have had opposing effects on the quality of treatment for substance use problems.

Change strategies of any kind can only work well if the organisations and people at which they are targeted are willing and able to change and to sustain that change. Researchers at the Institute of Behavioral Research in Texas have been conducting a determined attempt to measure and improve organisational 'health' including ability to adopt new practices. In England they found that client participation in treatment was greater in services characterised by team working and mutual trust among staff, and which encouraged discussion and implementation of new ideas and procedures. One reason may be that staff in such agencies are personally most able to implement new learning gained in training. Across behavioural health services in general, agencies which prioritise achievement, individualism and self-actualisation rather than security are most likely to be open to change. However, more prosaic issues like high staff turnover and the 'churn' in organisations due to market forces and commissioning cycles have severely limited the sector's capacity for accumulating and implementing learning (see for example 1 2 3). For more on these issues click this link to trigger a search on the Findings site for studies related to organisational functioning.

There is also the possibly crucial but under-investigated issue of recruiting people whose personal qualities and 'training' in life as much as in their professions equip them for the tasks of relating (in prevention) to young people on highly sensitive issues, or (in treatment and harm reduction) to society's most stigmatised, damaged and disadvantaged members. Research has typically sought to to purge the data of the influence of the person(s) delivering the intervention, in order to isolate the impact of the intervention itself, rather than to investigate them as an active ingredient. Nevertheless, examples of their importance include studies of motivational interviewing, probably in the UK the most influential therapeutic approach for substance use problems. In one such study, client engagement was unrelated to 'trainable' features such as the frequency with which the therapist made statements seen as compatible (such as open questions) or incompatible (such as warnings) with the approach. However, engagement was strongly related to embodying the overall spirit of motivational interviewing and to more general social attributes including empathy, warmth, supporting the client's autonomy, and coming across as 'genuine'. Within (and only within) the kind of empathic, caring context socially skilled therapists were able to create, 'breaking the rules' by making interventions supposedly incompatible with motivational interviewing actually deepened the client's engagement, perhaps because the therapist came across as more genuine. Another study of motivational interviewing training found that initial gains in skills had waned two months later. However, this was not the case for the addiction and mental health clinicians who, even before training, had been more proficient than the other trainees would be after training. Not only did these 'natural experts' start from a higher level, they went on to absorb and retain more of what they had learnt. In contrast, within months much of the training was wasted when it fell on less fertile human ground. Among this set of trainees, given a choice between choosing the 'right' people who had not been trained, and the 'wrong' people who had, the right people would have been the better choice. A further example from Britain concerned brief interventions based on motivational interviewing versus simple advice and information, intended to prompt college students to curb their cannabis use. In this context, some interventionists were markedly more successful with motivational interviewing than simple advice, while others did slightly better with the more straightforward and familiar advice option. In contrast there was little difference in outcome between the approaches overall, normally the only finding to be reported in such studies. What made the difference it seemed was not the approach itself, but the combination of approach and individual.

In the treatment sector, many of these therapist qualities are among the 'common factors' which cut across specific therapies and seem responsible for much of the benefits of psychosocial therapies in general. Qualities such as optimism, empathy, and respect for the patient or client are not easily susceptible to change via the strategies featured in the review. In contrast, specific therapies can be codified and disseminated via these strategies, but are unlikely to have major impacts on patients; even when expertly implemented in well resourced research projects, generally they make little difference to outcomes. For more on these issues see for example the Findings analysis of a review of cognitive-behavioural therapies; click this link to trigger a search on the Findings site for other studies of the qualities of the therapist.

Factors such as institutional and personal receptivity to change, and personal and organisational qualities not easily susceptible to formal change strategies, may have been one reason why the review found that the impacts of these strategies – especially on patients and clients – were inconsistent and often minor. Nevertheless, innovations of the kind recommended in the featured review – sharply targeted, highly tailored and feasible in practice – can be integrated in to practice and make substantial differences to, for example, treatment uptake and aftercare adherence. One clear example was the system of simple, inexpensive prompts and patient motivators introduced at a US inpatient treatment centre which substantially improved aftercare attendance and helped sustain progress made during initial treatment. Another was the introduction in Philadelphia of a computerised system to link assessed client needs to local services addressing those needs. Not only did this improve 'wrap-around' service provision, it more than doubled the completion rate of the core treatment programmes. Innovations such as these to improve quasi-administrative procedures hold at least as much promise as the introduction of evidence-based therapies.

Thanks for their comments on this entry in draft to Ann Roche of the National Centre for Education and Training on Addiction at Flinders University in Australia. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Last revised 10 August 2009. First uploaded

Open Effectiveness Bank home page

Top 10 most closely related documents on this site. For more try a subject or free text search

STUDY 2013 Screening and brief intervention for alcohol and other drug use in primary care: associations between organizational climate and practice

DOCUMENT 2017 Drug misuse and dependence: UK guidelines on clinical management

REVIEW 2011 Implementing evidence-based psychosocial treatment in specialty substance use disorder care

REVIEW 2011 A policy-oriented review of strategies for improving the outcomes of services for substance use disorder patients

STUDY 2012 Innovation adoption as facilitated by a change-oriented workplace

REVIEW 2011 Integration of treatment innovation planning and implementation: strategic process models and organizational challenges

STUDY 2006 Matching resources to needs is key to achieving 'wrap-around' care objectives

MATRIX CELL 2020 Alcohol Matrix cell D3: Organisational functioning; Medical treatment

STUDY 2008 Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment

REVIEW 2005 The motivational hallo