This entry is our analysis of a study considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.
Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text
Copy title and link
| Comment/query | Tweet
Jones A., Pierce M., Sutton M. et al.
Addiction: 2017, 113(2), p. 279–286.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Jones at andrew.jones@manchester.ac.uk.
Substance use treatment commissioned on a payment-by-results basis in England has been linked to higher rates of in-treatment abstinence and non-injecting than other commissioning models, but lower rates of treatment initiation and completion. Is this enough to support the policy?
Summary Payment by results is based on the principle of paying organisations to achieve set outcomes in the form of benefits for patients or clients, as opposed to paying for the delivery of services. In April 2011, the Department of Health selected eight commissioning areas in England to participate in payment-by-results drug and alcohol recovery pilot projects. Early evaluation findings suggested poorer performance in participating areas on some outcome measures. But, the evaluation being confined to the early stages of implementation (the first nine months) meant the authors couldn’t rule out under-performance due to a period of organisational flux.
Commissioning services based on patient outcomes, as opposed to the services delivered, was trialled and evaluated in eight English local authorities.
Findings for drug using patients included higher rates of in-treatment abstinence and non-injecting compared with other areas, but lower rates of treatment initiation and completion.
This study adds to a limited evidence base on the effects of payment by results. The evidence does not directly support payment-by-results schemes as policy for recovery from drug use problems.
Focusing on the drug misusers rather than problem drinkers, the featured study provided a more complete account of the impact of payment by results, extending the preliminary analysis of the pilots in two ways: firstly, by examining longer-term effects two years after the introduction of the programme; and secondly, by considering a wider range of outcomes including changes in the rate of subsequent re-presentation for treatment (indicative of relapse), rates of drug-related deaths, recorded offending, and behavioural change during treatment.
From April 2012 to March 2014, eight commissioning areas in England participated to varying degrees with the payment-by-results programme. These were compared with 143 non-participating areas free to operate under their own commissioning models.
Though participating areas were allowed some flexibility in the design of the payment-by-results scheme, including variation in the proportion of payment attached to outcomes and freedom to include locally designed payment metrics, their schemes were required to adhere to a national outcomes framework. Payments were aligned with three nationally-specified domains:
• progression towards abstinence from presenting drug(s) of dependence;
• reduction in offending;
• improved health and wellbeing.
The evaluation was based on an analysis of data from the National Drug Treatment Monitoring System, Office for National Statistics mortality database, and Police National Computer criminal records, between 2010 and 2014. This provided the researchers with a picture of recovery outcomes in participating and non-participating areas both two years before and two years after the introduction of the programme.
The data derived from:
• 154,175 adults in treatment in the two years before the introduction of the programme (10,716 in participating areas, 143,459 in non-participating areas)
• 148,941 adults in treatment after the introduction of the programme (10,012 in participating areas, 138,929 in non-participating areas)
About two-thirds used opiates, just under 30% crack and just over 30% cannabis (drug use categories were not mutually exclusive). The outcome measures were: rate of treatment initiation; waiting time; treatment completion and re-presentation; substance use; injecting; housing status; fatal overdose; and acquisitive crime. At issue was whether these indicators improved more in payment-by-results areas after these schemes started than over the same period in comparison areas. If they did, it would be consistent with payment by results having the intended beneficial effects on patients and communities.
After payment by results was introduced, clients in participating areas were less likely to be injecting whilst in treatment, and more likely to report abstinence from illicit substances and alcohol. At the latest within-treatment assessment, the proportion who reported no use of illicit substances or alcohol increased in participating areas (22% to 27%) and decreased (23% to 22%) in non-participating areas.
Among clients who successfully completed treatment within six months, the proportion who did not then return for treatment within a year increased in participating areas (82% to 90%), and also increased in non-participating areas, but to a much more modest degree (85% to 86%).
Overall, clients in payment-by-results areas tended to wait longer to access treatment, were more likely to experience an unplanned discharge from treatment, and were less likely to successfully complete their treatment or complete without later having to return for further treatment.
The proportion of clients who actually started being treated after their initial attendance decreased substantially in participating areas but increased in non-participating areas. When one participating area which seemed to be an outlier was excluded from the results, the relative decrease was no longer observed.
Among those who started a treatment intervention, the proportion who waited less than three weeks decreased in participating areas but increased in non-participating areas. Unplanned discharges increased in participating areas and decreased in non-participating areas.
The proportion of clients who successfully completed treatment within 12 months decreased in participating areas (34% to 24%) and increased in non-participating areas (27% to 29%). Correspondingly, the proportion of clients who both successfully completed treatment within six months and did not subsequently re-present to treatment within 12 months decreased (20% to 14%) in participating areas and increased (16% to 17%) in non-participating areas.
There was no statistically significant difference between participating and non-participating areas with respect to the resolution of housing problems, the rate of acquisitive crime, or drug-related deaths.
The proportion who did not inject four weeks prior to follow-up did not change substantively in participating or non-participating areas.
Drug addiction recovery services in England that are commissioned on a payment-by-results basis are associated with higher rates of abstinence and lower rates of injecting while in treatment than other services, but lower rates of treatment initiation and completion.
Successful treatment completion has been seen as critical indicator in England for several years. So, regardless of participation in payment by results, this could have informed the treatment priorities of all areas included in the evaluation. However, “Participating areas performed significantly worse than non-participating areas for this measure, recording a significant and substantive decrease. This suggests a failure of the [payment-by-results] systems to encourage improvements in primary patient recovery outcomes.”
The findings do not directly support the implementation of payment-by-results schemes as policy. However, as a considerable proportion of commissioning bodies in England are known to have either adopted or have plans to adopt a payment-by-results system, the findings have “wide and significant implications for this treatment sector”.
commentary Completion of treatment (without return) is the key indicator of recovery from substance use problems chosen by the UK Government. Seeing that “participating areas performed worse in a number of … recovery-specific outcome domains [including these], despite their apparently greater emphasis”, suggests that the attempt to promote recovery by ‘paying’ for it seems to have backfired.
The authors themselves concluded that the featured evaluation could find “no clear link between introduction of a [payment-by-results] scheme and improvements in treatment outcomes”, findings “consistent with evidence from the [wider] evidence base, which has generally only shown improvements in indicators of processes to be (weakly) linked to the introduction of [payment by results]” (1 2 3).
Earlier findings on payment by results in the eight commissioning areas indicated that the funding model was having the opposite effect to that intended – clients in participating areas were significantly less likely to complete treatment, and significantly more likely to decline to commence treatment. An Effectiveness Bank commentary at the time suggested that payment by results being counterproductive could depend on whether: (a) fewer clients successfully completing treatment were counteracted by more clients staying in rather than dropping out of treatment; and (b) in the longer term the schemes could be shown to have performed better than in the first year of operation. The current paper was able to address the latter part of this, showing what happened in the two years after the payment-by-results programme was introduced. Again, clients in payment-by-results areas were less likely to successfully complete their treatment, and less likely to complete without returning for further treatment. Although among a subset who completed their treatment in six months, the authors could show a greater increase in participating areas in the proportion not returning for treatment within a year, this finding was based on a post-hoc analysis which necessarily increased the opportunity to find a favourable result.
One concern before the implementation of the payment-by-results programme was of so-called ‘gaming’, where treatment services in participating areas would favour the selection of clients more likely to achieve the prescribed outcomes. This in basic terms could create a climate where organisations were ‘chasing the money’, and subsequently ‘chasing completions’. However, this didn’t appear to have been the case. Not only did rates of completions fall in areas participating in payment by results, but there was reportedly a relative increase in client complexity in participating areas. The assessment and referral system known as LASARS introduced an additional layer of bureaucracy – delaying treatment since the initial assessment was for the purpose of setting payment tariffs rather than promoting recovery. Areas commissioned on a payment-by-results basis did record more favourable outcomes for non-injecting and abstinence while in treatment, suggesting a positive impact, but perhaps only because this extra hurdle meant that compared to other areas, more motivated would-be patients made it through to treatment.
The full Department of Health evaluation (due in 2015) had not been released at the time this entry was written.
For more on payment by results in the UK and on treatment commissioning in general, see this Effectiveness Bank hot topic.
Thanks for their comments on this entry in draft to Russell Webster (whose web site substantially focuses on payment by results) and Chris Lee, a Public Health Specialist for Lancashire County Council. Commentators bear no responsibility for the text including the interpretations and any remaining errors.
Last revised 26 January 2018. First uploaded 22 August 2017
Comment/query
Open Effectiveness Bank home page
Top 10 most closely related documents on this site. For more try a subject or free text search
REVIEW 2017 An evidence review of the outcomes that can be expected of drug misuse treatment in England
MATRIX CELL 2018 Drug Treatment Matrix cell E2: Treatment systems; Generic and cross-cutting issues
STUDY 2009 The Drug Treatment Outcomes Research Study (DTORS): final outcomes report
DOCUMENT 2017 Drug misuse and dependence: UK guidelines on clinical management
DOCUMENT 2013 Community loses from failure to offer maintenance prescribing in prisons
HOT TOPIC 2016 What about evidence-based commissioning?
STUDY 2011 Interim methadone treatment compared to standard methadone treatment: 4-month findings
DOCUMENT 2017 2017 Drug Strategy