Send email for updates


About updates
Review analysis

This entry is our analysis of a review or synthesis of research findings considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original review was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text The Summary conveys the findings and views expressed in the review. Below is a commentary from Drug and Alcohol Findings.

Title and link for copying Comment/query to editor

A policy-oriented review of strategies for improving the outcomes of services for substance use disorder patients.

Humphreys K., McLellan T.
Addiction: 2011, 106, p. 2058–2066.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Humphreys at knh@stanford.edu. You could also try this alternative source.

Improving performance of substance use disorder treatment systems is no easy matter and one prone to unintended consequences. All the more welcome then is guidance from leading US experts with top-level experience in the UK and the USA; their favourite tactic, rewarding services for patient progress during treatment, is featured in UK payment-by-results schemes.

Summary Two US experts who worked at top levels on drug policy for the current US president and have advised the UK government draw on scientific literature and their experiences to offer policy makers an overview of system-level strategies to improve the outcomes of services for substance use disorder patients. Their aim was to stimulate discussion rather than conduct a systematic and comprehensive review of the evidence.

Though they may be used together, the review divides the strategies in to process-focused quality improvement strategies which change how treatment staff work in the expectation that this will improve patient outcomes, and patient-focused strategies which concentrate on outcomes without specifying how those are to be achieved.

Process-focused quality improvement strategies

Substantial deficits in the quality of substance use disorder care in the USA and the UK include an excess of paperwork, insufficient time with patients, demoralised staff, lack of medically trained staff, dysfunctional organisational dynamics, and underuse of scientific evidence. Strategies under this heading seek to change organisational, financial and clinical practices to redress such deficits. These improvements will it is hoped translate into better patient outcomes, but this cannot be assumed. In general, the longer the time between receipt of services and outcome measurement, the less the outcome can be taken as indicative of the quality of those services.

Increasing licensure/credentialing requirements

Most US addiction treatment programmes have no physicians or nurses and the primary credential of many counsellors (to an extent also in the UK) is themselves being in recovery. It therefore seems logical that increasing credentialing requirements for staff will improve quality.

However, such stiffening would exacerbate staff recruitment and retention problems unless accompanied by inducements to make a career in the field more attractive. Also, credentialing has become an entrepreneurial activity; countless organisations develop and sell credentials which do not always assure quality. Finally, the evidence from hundreds of clinical trials is that counselling outcomes are not predicted at all by the counsellor's type or level of educational degree; a prestigious degree in medicine, for example, does not ensure that a physician can make an emotional connection to an addicted homeless patient or an alcoholic soldier suffering post-traumatic stress disorder.

Despite these caveats, legitimate credentialing and licensure could help weed out unsuitable individuals, some credentials are essential to clinical activities such as prescribing medication or taking blood, and more highly educated staff are particularly receptive to evidence-based practices.

Beyond these rather gross indicators of value, new credentialing policies are a weak lever for improving addiction service outcomes, though one which might be stronger if focused on specific clinical competencies rather than coursework or hours of training.

Measuring and/or incentivising evidence-based clinical practices

Evidence-based clinical practices which have been incentivised in the USA include screening primary care patients for drinking problems and retaining patients in specialist treatment for at least three months. US health-care systems have also begun to monitor the proportion of patients whose substance use disorder is identified, the proportion who engage early in care, and the proportion retained in care over time.

Such incentives can lead to dramatic improvements in hitting these targets. However, achieving them can bear at best a weak relationship to subsequent patient outcomes (see for example this Findings analysis), shown among others by US studies of targets like the three-month retention measure also adopted in the UK. Research has shown that treatment providers respond to incentives (important in itself), but has yet to clarify which provider practices should be changed to improve patient outcomes.

Another issue is whether anyone other than the providers themselves should be made aware of how closely they follow evidence-based guidelines. A recent trial found that giving treatment services feedback on their patients' progress had no effect on clinical performance, confirming that if there are no consequences from doing so, information on one's performance is generally disregarded. Performance assessments are best made public if the goal is to improve outcomes.

Improving managerial capacity and business practices

This strategy involves expert business consultants helping treatment services improve their management skills, knowledge and capacity. The highest-profile US efforts are the Network for the Improvement of Addiction Treatment (NIATx) and the follow-on Advancing Recovery project.

An intriguing strategy employed by NIATx is the 'walkthrough'. Managers attempt to access care in their own services from the patient's point of view – for many, an eye-opening experience which helped explain low rates of treatment entry and retention. Other management practices taught in these initiatives include careful analysis and allocation of funding and better development of a business case for new funding. In response, services reduced waiting times and increased retention, and these gains persisted after intensive organisational consultation had ended.

The Advancing Recovery project generally improved continuity of care and use of evidence-based pharmacotherapies. However, both initiatives probably attracted better-organised and led and more motivated services; effects are likely be less dramatic if such initiatives are applied to all services. Also, as yet changes in care processes have not been related to long-term patient outcomes.

Embedding substance use disorder care in a higher quality care network

An example of this strategy is the Obama administration's policy of medicalising care for substance use disorders. Through funding for screening and brief intervention in primary care settings and changes in public and private insurance, it has begun transplanting substance use care into general health care. This brings with it features which may improve patient outcomes, such as medically trained staff, availability of medications, financial incentives for quality, electronic health record monitoring of patients, and a broad culture of careful inspections and monitoring. Co-location should also facilitate access to supplemental medical services. Finally, integrated care coordinated by one's usual GP may be more accessible and less stigmatising than going to an 'addiction treatment programme', meaning it may also be possible to engage lower severity and more manageable patients.

However, this does not guarantee quality improvement. Medical care systems could re-allocate resources intended for substance use disorder care, and patients may be treated by less knowledgeable practitioners. Integration remains a promising idea in search of rigorous evidence rather than something which can be assumed to work.

Patient-focused strategies

Another class of strategies focus on the patient's actual outcomes rather than organisational or clinical practices. From this perspective, some of quality improvement strategies described above may be criticised for not focusing on what ultimately matters most, a point strengthened by research showing how changing care processes has often not translated into better patient outcomes and can have negative side effects. The patient-focused approaches described under this heading have been implemented less frequently in the addictions than process-focused strategies, so examples are drawn from treatment of other disorders.

Rewarding post-treatment outcomes

In some health care sectors providers are directly incentivised to produce specific long-term patient outcomes. An example from substance use treatment is the payment-by-results programme launched by the UK government, the first in this sector.

One issue is that what happens to a chronically ill individual after treatment becomes less closely linked to care quality over time. Measuring outcomes too long after treatment could demoralise providers held accountable for things over which they have little control. Another risk is that providers will exclude patients with a poor prognosis. Also, the cost of re-assessing patients after treatment can be considerable. When this task is assigned to clinicians, follow-up rates and data are poor and time is diverted from treating patients. It is better to use an independent outcomes monitoring team, but this requires a continuing resource commitment.

Rewarding in-treatment performance

Another approach is to reward services for outcomes attained during treatment. This resolves several problems with the previous strategy: finding the patient is easier, and what the provider does should bear a stronger relation to how the patient is doing. Of the outcome improvement approaches described in this paper, this is among the most promising and feasible, not only because it could improve care, but because it focuses clinicians' attention on something they can and should be responsible for throughout the care process.

One such experiment in the US state of Maine initially generated enthusiasm. Services were funded for raising the proportion of patients who by their final contact with the service had achieved outcomes such as abstinence or major reduction in substance use. Performance appeared to improve, but this seemed due to services treating fewer severely troubled clients. Patients may also have been less candid when their accounts of their substance use affected their clinician's income. The system was updated but the most recent evaluation again yielded disappointing results.

It is not necessarily that this approach cannot work, but that performance contracts need to reward objective outcomes such as urine test results and adjust these rewards in the light of the service's case-mix. Such a system has been trialled in US methadone clinics which were informed of their performance, but with no financial or reputational consequences attached, there is no evidence that this changed clinical practice or improved patient outcomes.

Some clinicians might say such approaches are not feasible because they require regular monitoring of patients' substance use. However, this should happen anyway as good clinical practice. Research is, however, still needed to establish the strength of the relationship between in-treatment progress and longer term, post-treatment outcomes.

Rewarding patients for attaining particular outcomes

Individuals with substance use disorders do respond to incentives and sanctions such as short jail terms or housing linked to positive drug/alcohol tests, or those given by contingency management treatment programmes which systematically levy material rewards or privileges for abstinence or other outcomes.

Paying patients can however be resisted by the public and sometimes care providers too on the basis for example that 'They ought to change for free like everyone else'. Non-financial rewards and stressing the public benefits of these schemes (such as safer neighbourhoods) can help. Changes induced by external rewards sometimes reverse once rewards end. Such schemes may however be useful in the early stages of treatment to promote progress and engagement with care.

Making the patient a customer with purchasing power

A radically different approach tried in the USA gave individuals early in the recovery process vouchers to buy whatever services they thought would aid their recovery. Examples included college classes, transport to work or to self-help group meetings, housing, dental care, work training and clothes for job interviews. In theory this should generate improvements via the mechanisms that drive efficiency and quality in commercial markets. Access to Recovery, as the programme is called, expanded both the number of organisations providing services and the number of recipients. More importantly, outpatients in Washington state who received vouchers stayed longer in treatment and were more likely to be employed than comparison patients.

'Personal health budgets' being piloted for health and social care in the NHS are analogous. In consultation with a health professional, chronically ill individuals are given a fixed pool of funds from which they can buy a range of services, assembling a care package tailored to their needs. Extension to the addictions has been discussed within government, but it is not clear whether this will be tried.

The authors' conclusions

Strategies policy makers can use to improve outcomes of substance use disorder treatment are often poorly developed with weak empirical support. Incentives for particular clinical practices can change what systems do, but it is less clear which changes translate into better patient outcomes; some such schemes have proved literally worse than doing nothing. Cases discussed in this paper in which care utilisation was incentivised, but outcomes did not improve, mean money was spent on care that was apparently not needed, which may have adversely affected other patients due to the diversion of resources.

Despite a small evidence base, bringing market forces targeting quality and effectiveness to bear on treatment systems – including directly rewarding outcomes – has significant practical and logical appeal. Particularly promising are initiatives focused on in-treatment performance rather than long-term post-treatment outcomes. Also promising are vouchers that give patients purchasing power for their chosen services, hopefully to be extended to substance use disorders, perhaps via the British health service's personal health budgets.

When implementing such programmes policy makers serve themselves and the field if they embed careful, realistic evaluations from the start. The required investment and level of collaboration are substantial, but justified by the potential payoff in knowledge about how to enhance care for life-threatening disorders.


Findings logo commentary This advice from two of the world's most respected addiction researchers, each also with top-level experience in government drug policy, usefully and lucidly sets out the options for making improvements to treatment systems. Exactly what those improvements are intended to achieve was left open, so its findings should apply to the ambitions of recovery-focused UK national strategies.

Realistically they observe that quality improvement initiatives often seem to bear little relationship to what really improves outcomes for the patient and society, because outcomes do not in fact improve at all or very much. Treatment is a relationship business; mechanistic pulling of levers may shift the gears but not touch the heart of the matter, resulting in surface changes which do not last, a common experience in contingency management programmes which reward and sanction patients for desired behaviour or non-behaviour.

As the authors explain, in turn this means that credentials and training courses completed may mean little when it comes to forging engagement- and outcome-enhancing relationships. Studies can, for example, isolate personality variables which when high or low mean that on average patients respond best to certain types of counselling styles. However, the complexity of each individual means that what is indicated for one aspect of their situation or character may be counter-indicated by another, leaving the sensitivity of the therapist to sort out the best approach. Tying their hands too tightly through detailed treatment manuals (no matter how expert the author) and supervision to ensure they stick to the script has in the case of motivational interviewing led to worse outcomes, not better.

One implication the reviewers do not touch on is that without appropriate recruitment, much of the quality improvement effort put in to training and supervision will be wasted. This was the message which emerged from a study of motivational interviewing training which found that initial gains in skills had waned two months later. However, this was not the case for the addiction and mental health clinicians who, even before training, had been more proficient than the other trainees would be after training. Not only did these 'natural experts' start from a higher level, they went on to absorb and retain more of what they had learned.

UK payment by results schemes

Though featured under schemes which reward post-treatment outcomes, nationally agreed outcomes for pilot payment by results schemes in England often specify in-treatment and treatment exit measures, placing them partly within the during-treatment payment schemes favoured by the featured review. One of the problems it identifies with post-treatment payment schemes – the resources needed to re-contact and reassess patients – is sidestepped by using routinely collected criminal justice and treatment records which do not require contact with the patient. Reports from the pilots suggest these schemes can both be feasible and generate innovations focused sharply on achieving results.

The British schemes attempt to balance in-treatment and treatment exit measures against longer term crime and relapse indicators, generally choosing to place greater financial weight on the longer term. This means services must wait many months – in respect of some measures nearly two years – to receive much of their funding under the scheme, a cash-flow problem which requires counter-measures if services are to survive. Arguably too, as the featured reviewers comment, services are placed at financial risk for outcomes over which they may have little control because they are so far from the time when they had direct influence over the patient.

The alternative of weighting in favour of during treatment and treatment end measures falls foul of the observation in the featured report that these are often loosely related to the longer term recovery the system is trying to generate. For example, 'successfully' completing treatment free of dependence and of opiate and crack cocaine use did mean that over the next four years more patients in England appeared to have avoided relapse, but the difference of 57% versus 43% who did not successfully complete was not as large as would be expected if successful completion correlated strongly with lasting recovery.

Such schemes are classed by the reviewers as "patient-focused strategies" – not to be confused with patient-centred practice in the sense of basing treatment objectives and methods on the patient's preferences and priorities. Being patient-focused in this sense is threatened by payment by results schemes because these pre-set the treatment destination in detail without reference to what the individual patient wants, and in a way services cannot afford to ignore because their financial survival depends on meeting the criteria for payment. Local schemes could still create a space for the patient's ambitions in their payment criteria, but this is not a required element or one included in the national outcomes schema, nor one which sits easily within a system predicated on observable outcomes the public and their representatives recognise and are willing to pay for.

As a commentator on the featured review pointed out, schemes which make payment contingent on either process or outcomes entail a regulatory overhead which could eat in to whatever efficiencies are achieved at the services concerned. For example, the reviewers rightly caution that "performance contracts need to reward outcomes that are objective (eg, urine testing) and case-mix-adjusted". This means some authority has to assess the severity of the caseload in terms of the resources needed to achieve the intended outcomes, and assess whether those outcomes have been achieved in ways which go beyond merely asking the patient. When funding, jobs and organisational survival ride on these assessments, leaving them entirely to the people and organisations at threat may stretch their integrity too far. This concern spawns new regulatory requirements and possibly new regulatory bodies which must, as the Audit Commission described, be capable of overcoming complexity to deliver valid and meaningful measures if disputes, demoralisation and wastage are to be avoided.

In UK payment by results schemes the most visible result has been the setting up of central assessment centres (or LASARS), which have a key role in setting tariffs based on patient severity and verifying outcomes. These say the Gaming Commission should be independent both of treatment services and the commissioners of those services to make them less vulnerable to pressure to manipulate the figures (or at least the suspicion that this is happening), meaning a new body with its own overheads, which itself may require regulation. An independent assessment centre also places another step in the journey to accessing treatment during which access may falter. The plus side may be more efficient assessment, better treatment placement, and the potential for long-term case management to start at the assessment stage, but these possible advantages could have been achieved by centralising within existing structures.

British practitioners and managers seeking to improve their practice have available to them the web site of the Substance Misuse Skills Consortium, an independent initiative led by treatment providers to harness the ideas, energy and talent within the substance misuse treatment field, to maximise the ability of the workforce, and to help more drug and alcohol misusers recover. Commissioners of services have been offered guidance from the National Treatment Agency for Substance Misuse, England's special health authority tasked to improve the availability, capacity and effectiveness of drug misuse treatment.

Thanks for their comments on this entry in draft to Keith Humphreys of Stanford University in the USA, crime and drugs consultant Russell Webster, and Annette Dale-Perera of the Central and North West London NHS Foundation Trust in England. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Last revised 28 September 2012

Comment/query to editor
Give us your feedback on the site (one-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates


Top 10 most closely related documents on this site. For more try a subject or free text search

HOT TOPIC 2016 What about evidence-based commissioning?

STUDY 2011 Performance-based contracting within a state substance abuse treatment system: a preliminary exploration of differences in client access and client outcomes

DOCUMENT 2013 Delivering recovery. Independent expert review of opioid replacement therapies in Scotland

DOCUMENT 2016 Coexisting severe mental illness and substance misuse: community health and social care services

STUDY 2011 Therapist effectiveness: implications for accountability and patient care

STUDY 2014 Assessing the availability of and need for specialist alcohol treatment services in Scotland

DOCUMENT 2010 The Patel report: Reducing drug-related crime and rehabilitating offenders

REVIEW 2011 Implementing evidence-based psychosocial treatment in specialty substance use disorder care

STUDY 2012 Innovation adoption as facilitated by a change-oriented workplace

STUDY 2010 A randomized controlled study of a web-based performance improvement system for substance abuse treatment providers