Send email for updates


About updates
Hot topic

Below is one of our selection of Hot Topics, important issues which sometimes generate heated debate over the facts or their interpretation. Click the GO button or the Blue title to trigger a customised search for relevant Findings logo documents. Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text

Title and link for copying Comment/query to editor Other hot topics

What about evidence-based commissioning?

Recently dramatically re-shaped in the UK, commissioning is the process of identifying needs within the commissioning body’s target population, and of developing policy directions, service models, and the market in services, to meet those needs in the most appropriate and cost-effective way. Despite its importance, this process is relatively unevidenced. Ticking the filter term “Commissioning services” towards the bottom of the Effectiveness Bank’s subject search page limits the main search to just those documents concerned with commissioning; generally just a few are left.

One reason for the lack of studies is that usual ways of evaluating services are not feasible for evaluating local networks of services. Large numbers of communities cannot easily be assigned at the toss of a coin to one type of commissioning process versus another. The alternative of finding naturally occurring comparisons where each community is the same except for its commissioning processes founders on the fact that these grow organically from the community.

These features mean evidence is in short supply and often has to interpret ‘messy’ real-world examples subject to multiple influences rather than deliberately changing practice to see what happens. In turn that means the results may not be caused by commissioning processes, but by other differences between times and places with one type of commissioning process and those with another.

Before getting to the evidence, first this hot topic sets the scene by looking at how services are commissioned, how this has changed, the fears about the consequences, and the safeguards to stop these happening. Then we turn to the research evidence, especially that on new payment-by-results funding mechanisms. From what has been said, it will come as no surprise that there are more questions than answers – but the questions are fundamental, setting the agenda for assessing where we have got to and are going.

All change in England

The data that we do have from the UK (dipped into below) may be seen as an argument for reforming structures, and in England that has happened on a grand scale. In 2014 this new landscape was outlined by the English drugs field’s membership body in a publication intended to help treatment services make their case for investment to commissioning bodies subject much more than previously to local political priorities.

From April 2013 national expertise, specialist national services and advice and support have been provided by Public Health England, which has absorbed the National Treatment Agency for Substance Misuse. As the task facing treatment has swelled and diversified, the financial resources are shrinking Locally the treatment budget formerly administered by that agency has been allocated to local authorities to help fund their new public health responsibilities, including the prevention and treatment of alcohol and drug problems. Clinical commissioning groups consisting of GP practices take care of commissioning NHS-funded clinical services for the local population, while criminal justice treatment-support funding is now under the control of the police and crime commissioners, and prison health services (including drug and alcohol treatment) have become the responsibility of NHS England, formerly known as the NHS Commissioning Board.

Gluing it all together is the role of local health and wellbeing boards, multi-agency groups responsible for the overall strategic direction for improving health and well-being in their areas and for coordinating NHS and local government efforts, taking over the role played by multi-agency drug and alcohol action teams which had focused on substance use and had more specific expertise in that area. The boards’ initial priorities were generally public health and health inequalities, with a distinct focus on generic preventive strategies rather than more ‘downstream’ addiction problems which had already developed to the point of justifying treatment. Whatever their focus, in practice four years on from their initiation in 2012, most boards still seemed “some way off driving the big issues”. Drug and alcohol action teams also had a commissioning role, now largely subsumed into the broad public health functions of local authorities, depriving commissioning of the accumulated knowledge and joined-up thinking built in to well-functioning teams. The anticipated result is a loss of focus on and knowledge of substance use and treatment in local strategic and commissioning bodies at the same time as national leadership has been diluted by absorption in Public Health England.

In these respects, England can be seen as reaping the consequences of the lack of local political and administrative ownership of substance use service delivery, a lack encouraged by a ring-fenced, centrally allocated budget and a centralised delivery structure in a “hub and spoke” configuration, leaving local infrastructure and critique underdeveloped and multi-agency partnerships “unable to protect investment because they were largely locally disconnected”. Locally there was little defence against the disinvestment which seems to have followed ( below) withdrawal of protected central funding.

In contrast, Scotland’s commissioning structures remain based on the equivalent to what in England were drug and alcohol action teams. But there too, a recent change in funding mechanisms arising from the transfer of responsibility for drug policy from justice to health departments has raised concerns about reduced resources. Associated with this change, funding earmarked for drug and alcohol services will fall from £69.2 million to £53.8 million in 2016/17. Government expects health authorities to make up the shortfall from health budgets, which are due to increase by 6.5%, but whether they will remains unclear. As taken further in England, this represents a move away from protecting substance use budgets, and instead leaving local funding bodies to decide how much to allocate this sector from more general income sources.

Change of direction for treatment to ‘recovery’

Commissioners also face a radical change in objective from the top, which if taken on board locally would usually preclude simply continuing with established models of service provision. Adoption of ‘recovery’ as an overarching principle for addiction treatment (see this recovery hot topic) entails extending the commissioning horizon beyond treatment episodes restricted in space (as at a clinic) and time, to the world in which the patient lives and must fully return after treatment, and to their entire life course.

Precise definitions of ‘recovery’ are lacking, but the broad themes of what for UK administrations counts as recovery are clear: some of the most marginal, damaged and unconventional of people are to become (as the Scottish drug strategy put it) “active and contributing member[s] of society” and variously abstinent from illegal drugs and/or free of dependence. Yet at the same time as the task facing treatment has swelled and diversified, the financial resources to commission services and forge links with other sectors are shrinking, and those other sectors themselves face financial pressures.

Fears over ‘disinvestment’

Diminishing per-patient funding for addiction treatment and falling total funding predate the current austerity era which took root in 2010 following the banking crisis of 2008. Though it blipped upward in 2006/07, per-patient spending on drug addiction treatment in England including local contributions and central funding has as far as can be calculated been on a downward trend since 2002/03, when it was about £3861 in 2008/09 prices. By 2008/09 it had fallen to £2756, about 29% less. After peaking in 2006/07, the total spend also started to fall. The freeze in central funding which followed almost certainly led to further per-patient, inflation-adjusted funding decreases.

Though disinvestment is not new, according to major service providers, by 2015 it had reached the point where instead of trying to determine how best to get value for money – entailing a focus on ‘value’, which might demand spending more – England had descended towards commissioning systems where money only counts, and the cheapest way to provide services wins out under the pressure of budget cuts.

A well-informed and powerful central advocate could identify such developments and support or pressure improvements, but according the former head of the National Treatment Agency for Substance Misuse says Public Health England “has disinvested” from the local presence which characterised his agency, “limiting not only its ability to promote and share best practice, but also the local intelligence it previously provided which enabled Home Office and Department of Health to understand what was really happening on the ground.” If lowest-common-denominator poor practice is taking hold, no one centrally may know until the consequences become apparent, perhaps as in the recent increase in drug-related deaths. A ‘see no evil’ scenario is also apparent locally, particularly in the lack of local data on drug-related deaths, and the narrowing down to successful treatment completion as the indicator of recovery, one only loosely related either to post-treatment poisoning deaths among patients treated for opiate dependence or to lasting remission from drug dependence.

Also gone since 2015 is the central eyes and ears and voice of the drugs field in England in the form of the now dissolved sector membership charity, DrugScope. When it was available to aid intelligence-gathering, DrugScope participated in an audit requested by the Department of Health to check how devolution to local authorities had affected commissioning and funding plans for drug and alcohol services in 2014/15 and beyond. A commentary from DrugScope highlighted the plans of a substantial minority of areas to reduce funding in 2014/15 and 2015/16, and the fact that already in 2013/14 commissioners were concerned about the “significant financial pressures they were under and the potential impact this was having on all services”.

Since then the financial screw has tightened on local authorities and will tighten further on their public health allocations year on year, forcing more radical service reviews and including merger of drugs and alcohol with other service sectors. At the same time priorities seem to be shifting to alcohol and preventive initiatives rather than illegal drugs and treatment, a move towards a public health agenda closer to the traditional remits of local authorities and of their key decision-makers in this sector, directors of public health.

In 2015 the consequences of these processes became visible through a survey of treatment services across England and interviews with senior staff. It was the third such report, and “While the first found no evidence of deep and widespread disinvestment, in its second year the survey found that many respondents were experiencing or anticipating substantial funding reductions. This trend continues in 2015, with a considerable proportion of both community and residential providers reporting a reduction in funding.” The future was likely to be gloomier still as budget cuts accumulated, meaning “challenges around resourcing safe and high quality services clearly remain”.

Linked to the drive to save money and yet produce recovery outcomes is the typically three-year retendering cycle through which commissioners seek to reshape services and/or provide them more cost-effectively. In England in 2015 nearly half the substance use services which responded to a survey had been through tendering or contract re-negotiation in the previous year and half expected to do so in the year ahead. This so-called ‘churn’ is a major diversion from service provision, leading to much of a three-year cycle being spent getting up to speed or preparing for possible de-commissioning, though many services felt the result was improved service delivery.

Safeguards and incentives

Provided outside the ambit of the NHS constitution and regulations safeguarding patient choice and competition, commissioning of addiction treatment services seems effectively unregulated. The market mechanism of patients voting with their feet to go to what for them are better quality services is often not an option, increasingly less so as mega-services take over in local areas, offering to do everything for the commissioners with consequent cost-savings.

There are, however, continuing financial incentives for local authorities to maintain and increase drug treatment numbers, and the proportion of patients who The outcome is uncertain, but could be quantity at the cost of quality successfully complete treatment, defined as the number of patients who left treatment free of drug(s) of dependence and did not return within six months as a proportion of the treatment caseload. Those who score well on these measures will to this degree (other non-drug related measures also have an effect) earn a larger share of the dwindling national pot of public health money.

Public health commissioners and planners will also be held accountable via the public health outcomes framework for England. Among the indicators of their performance is successful completion of drug treatment, the number of alcohol-related admissions to hospital, and the proportion of people entering prison whose substance dependence treatment need had not been addressed in the community.

Putting these influences together, local authority politicians and commissioners could be forgiven for aiming to maintain or increase numbers in and leaving drug addiction treatment while at the same time cutting the spend per patient and doing what they can to prevent rapid treatment re-entry. The outcome is uncertain, but could be quantity at the cost of quality. However, if disinvestment takes the form of poor quality services rather than their absence, in the treatment sector this should be picked up by the Care Quality Commission, which in 2014 set out its plans to assess whether residential and non-residential services are “safe, effective, caring, responsive and well-led”.

British commissioning systems fall short of expectations

Here we turn to evidence on commissioning and on how to improve it, much of which in the UK portrayed the shortfalls of the systems which predated the restructuring and cutbacks outlined above. Evidence-based commissioning as an ideal has always been at the mercy of politics (national and local) as well as the vagaries of budgets. Very few areas ever developed a system truly planned in a holistic, evidence-based fashion.

These weaknesses emerged in investigations in to alcohol services by the Department of Health up to March 2011, which found many areas did not have a clear, shared vision for reducing alcohol-related harm, and that alcohol strategies were often out of date or being rewritten.

Three years later the charity Alcohol Concern surveyed plans and reports from commissioning authorities in 25 areas, including 15 topping the league of alcohol-related harm. Almost all the documents mentioned alcohol, though many would not it was judged have been considered comprehensive enough to meet guidelines from Public Health England. Though included among the 15 high-need areas, four made no recommendations about tackling alcohol-related hospital admissions and seven each none about identifying and advising risky drinkers or about treating dependent drinkers. Public Health England had stressed the development of recovery-orientated systems integrating peer support and mutual aid with professional services, yet these were “scarcely mentioned”, suggesting a “discontinuity between evidence and actions”.

In Scotland in 2009 an audit of local drug and alcohol service provision systems found these poorly informed by the problems to be addressed and what works in addressing them, and in respect of drugs, unclear about what ‘value for money’ consists of. Specifically in relation to the commissioning of advocacy support for drug users, intended to equalise the power relation with services and help patients negotiate their treatment, in both England and Scotland, in 2010 national rhetoric had yet to be consistently reflected in commissioning decisions on the ground.

Also in 2010, exhaustive consultations in the south west of England revealed that procedures for commissioning offender alcohol interventions were unclear, contested, and badly under-resourced. In England’s prisons, an inquiry conducted in 2009 and 2010 found that drug treatment commissioning and funding structures had led to a “fragmented system” offering limited choices in the types of treatment and broader social support available, while across 2004 to 2009 prison inspectors reported that alcohol services present a “depressing picture” of “very limited” services, which leave offenders with poor prospects on release. Scottish prisons feature a range of alcohol-related interventions, but in their assessment published in 2011, health service researchers were concerned that many prisoners who could benefit from such interventions were being missed.

Commissioning processes do make a difference

How far things have changed and will change as in response to the recovery agenda, budget cuts and reorganisations, is as yet unclear, but what is clear is that for good or ill, commissioning processes do make a difference.

England has seen the slashing of waiting times for drug addiction treatment and a steady improvement in the proportions of patients leaving treatment at least for the time being free of dependence – both associated with explicit national drives expressed through local commissioning. Similar influences had led to increases in the proportion of patient who stayed in treatment at least 12 weeks or successfully completed before then. Alcohol treatment waiting times too have shortened in England, and in Scotland waits for both alcohol and drug treatment fell in line with a national target implemented through local commissioning structures.

In the very different public service environment of the USA, waiting times have also been a target. One national US programme halved waiting times for addiction treatment and extended retention partly by fostering a self-sustaining inter-service improvement network and a performance-analysis system linked to funding. In the US state of Delaware, state authorities effectively incentivised services to improve patient recruitment and engagement, while in Washington patients given vouchers to purchase recovery services stayed in treatment longer and were more likely to gain employment. Evaluation of an early example from the state of Maine in the USA of a system which approached payment by results ( next section) offered plusses and minuses. From 1992 services were told their current funding may depend on how well they had done the previous year, including indicators of how far patients had improved while in treatment. Effectiveness as assessed by these measures did improve, but other elements of the contract seem to have led services to focus on delivering only contracted services, withdrawing the extra inputs previously provided.

Is payment-by-results the answer?

If there are new structures and objectives for commissioners, so too is there a new mechanism in the form of payment-by-results – paying organisations, not to deliver specified services, but (in its pure form, via whatever acceptable service mix they choose) to achieve set outcomes in the form of benefits for the patients or clients. Though this seems a sure-fire shortcut to value for money, the National Audit Office has warned that payment-by-results contracts are “hard to get right, which makes them risky and costly for commissioners”, and that there is a risk of a “negative impact on value for money”.

Despite the complications and concerns described below, it is important to remember that if adequately assessed, the great advantage of paying for results is that it takes the guesswork out of wondering whether mandated or incentivised quality improvements really do make a difference. It is perfectly possible to extend retention, introduce new evidence-based therapies, or, as in this US study, to incentivise their competent and complete implementation, without this in turn improving substance use outcomes.

Getting outcome measures right is critical

Choosing the right outcome measures is critical to the success of such schemes. In a reciprocal process, this requirement is in turn affecting treatment objectives and structures. The concrete, measurable and collectable outcomes required by the schemes are bound to become not just proxies for the ultimate objective (recovery), but make-or-break sub-objectives for services whose survival depends on achieving them. If in reality the measures inadequately represent the desired recovery objective, funding based on them will send treatment services chasing in the wrong directions and punish and reward them for the wrong reasons.

When it came to making outcomes concrete enough to be used to pay English drug services, recovery through employment as envisaged in national strategies was notably lacking, perhaps a recognition that implementing this transformational vision would be a stretch when the resources to elevate patients from near the bottom rungs of society to at least near the average have been stripped back. Practicalities if nothing else mean English schemes often specify in-treatment and treatment-exit measures rather than post-treatment recovery indicators, and the post-treatment indicators are confined to routinely collected criminal justice and treatment records which do not require recontacting and reassessing patients. Such measures bear a loose relationship to lasting remission from dependence. A more comprehensive and explicitly recovery-oriented set of measures were proposed in 2012 by drugs field experts and a team from the not-for-profit Social Finance organisation to underpin investment in services.

Ironically, the English pilots placed a premium not on the long-term contact presupposed by the recovery vision and associated understandings of addiction, but on discharging patients who then are not seen again for at least a year. The individualisation stressed by recovery advocates also seems at odds with the payment mechanism. Local schemes could create a space for the patient’s ambitions in their payment criteria, but this is not a required element or one included in the national outcomes schema, nor one which sits easily within a system predicated on observable outcomes the public and their representatives recognise and are willing to pay for. Instead schemes pre-set the treatment destination in detail without reference to what the individual patient wants, and in a way services cannot afford to ignore because their financial survival depends on meeting the criteria.

Success criteria seem more suited to services geared to improvement rather than prevention of deterioration. How many people do not die, keep their homes, retain custody of their children, or stay clear crime, are not assessed, and it is hard to see how they could be. Among the typically multiply problematic caseloads of publicly funded treatment services, funding criteria focused on substance use risk distorting service provision away from meeting what for the individual are their most pressing needs, perhaps forcing them to conform to the implied view that their substance use is their primary problem and primary identity. That risk is even sharper outside recovery-oriented payment-by-results schemes, where services are judged primarily on their generation of people no longer dependent on drugs or using heroin or crack.

Levelling the playing field costs

Beyond choice of outcomes are some more general issues faced by any such scheme. Even if outcomes are ideal and could be directly and accurately measured – a task which has expensively occupied teams of researchers – what led to them would remain unclear, particularly since patients commonly traverse several treatment services and modalities before sustainably overcoming dependence. Giving all the credit to the last episode ignores the contribution of predecessors which paved the way for ‘its’ successes.

Another potential drawback was exemplified in the Delaware study cited above, where the pattern and pace of improvements suggested services did respond to financial incentives, focusing effort where rewards were greatest, but also that they did just enough to harvest those rewards without trying to do more to help their patients. It seems to reinforce concerns that (like contingency management incentives for the patients) such systems engender a mentality of doing just enough to get the money; the rewards can become the objective, not the patient’s progress.

Treatment entry processes too must change – and not necessarily for the better – due to the requirements that the new payment mechanism has convincingly unbiased ways of taking in to account the ‘degree of difficulty’ posed by a service’s case-mix and of measuring and recording the results. When funding, jobs and organisational survival ride on these assessments, leaving them entirely to the people and organisations at threat may stretch their integrity too far. In UK schemes, the most visible result has been central assessment units (or LASARS), which have a key role in setting tariffs based on patient severity and verifying outcomes. These, say the Gaming Commission, should be independent both of treatment services and their commissioners, inserting another step in the journey to accessing treatment. A keen observer of the process has raised concerns about the diversion of resources to administration and to this extra step, which means a patient’s “first contact is not with a helping service but a payment system”. The plus side may be more efficient assessment, better treatment placement, and the potential for long-term case management to start at the assessment stage.

Early results of payment-by-results in England

Evidentially, payment-by-results in health and social care of any kind is a leap in the dark. A review of reviews could find no evaluations which reported on patient outcomes, and a review specific to drug and alcohol treatment could find “little evidence that [pay-for-performance] is effective in improving client outcomes”. When Russell Webster, a leading UK commentator on such schemes, reviewed the literature, he found “consensus” about the evidence base – consensus that is “not able to give a clear indication as to whether payment by results works”, and that “unexpected, often perverse, consequences are commonplace”.

Evidential uncertainty and the risk of counterproductive effects are presumably among the reasons why the English schemes were evaluated pilots. Analysed by the Department of Health, initial results to February 2013 for the 6582 patients being treated for drug rather than alcohol problems showed consistent gains only in the proportion who while in treatment said (via forms completed by staff) they had stopped using their problem substance(s). Against the same comparators, the proportion exiting treatment free of dependence – a measure closer to the government’s recovery ambitions – was worse in the pilots. Other measures were seemingly unaffected. For the 3081 patients whose problems were mainly with alcohol, things seemed worse: no indication that the pilots had elevated abstinence rates and the proportion exiting treatment free of dependence was lower than in the rest of England and lower than in the same areas before the pilots. “Mixed” was the document’s characterisation of the results; “disappointing” might also have been justified – but these were early days.

The government analysis was followed in 2015 by that of researchers, including some involved in creating and maintaining the key data source, the National Drug Treatment Monitoring System. Their peer-reviewed journal article was based mainly on patients with heroin and crack cocaine problems treated in payment-by-results areas in the first year of the schemes compared to earlier years and other areas. The analysis confirmed the Department of Health’s key finding – that the proportion of patients successfully exiting treatment (defined in this case as no longer using heroin or crack and no longer dependent) was lower in scheme areas. The difference was slight but statistically significant and consistent across various comparisons. Also slight but significant was the higher rate of patients declining to start or engage with treatment after contact with the treatment system. Both results can be considered contrary to the government’s recovery agenda.

For the researchers the probable reasons were that services were keeping patients longer in treatment than they did without payment-by-results incentives in order to cut down on the number who return to treatment after leaving, Services kept patients longer to reduce financial losses due to returns to treatment an event which loses the service money. Though the opposite seems to have been the intention in government, this effect might well be positive. Why more would-be patients turned treatment down is even less clear. The researchers’ speculation that “linking payment to recovery-based outcomes ... is likely to have changed the nature of treatment” seems to hint at the disturbing possibility that being a unit of currency for the service sours the treatment relationship, the outcome Russell Webster foresaw when he noted that pre-treatment assessment about determining the price on the patient’s head meant their “first contact is not with a helping service but a payment system”.

More questions than answers

It will be clear to the reader that the questions about the English schemes are multiple and far-reaching, but answers are in short supply. Our Matrix Bites commentaries on seminal and key studies have posed these questions: Surely a charity or health service should not need external incentives to strive to do the best for its patients? Yet without these, would services stay unstretched within acceptable-quality comfort zones? Are pre-set objectives desirable, pushing services to deliver on national and local priorities, improving comparability across services, and preventing them glossing over their shortcomings? Or do they stifle patient-centred practice, preventing treatment objectives being based on the patient’s priorities? Maybe all the above? Does the no-return-for-a-year criterion incentivise services to make sure their patients’ recovery lasts, or tempt them to counterproductively place hurdles in the way of treatment re-entry? Where does it leave long-term continuing care of the kind advocated by some authorities on recovery? These issues are raised against the backdrop of an existing treatment system which has itself been widely criticised for failing to deliver recovery outcomes. Is that criticism justified; could the pilot mechanisms do any worse?

How to find out more

Run these hot topic searches to see what internationally or in studies confined to Britain has been discovered by evaluators about how commissioning in general and payment-by-results in particular work, and how they might work better. See also this UK-based resource pack on payment-by-results offering access to research, comment, and an interactive program to help commissioners and providers decide whether payment-by-results might be effective for commissioning a particular service.

Thanks for their comments on this entry to Russell Webster, UK-based author of a blog on payment-by-results, Sara McGrail, independent commentator on British drug policy, and David MacKintosh, formerly of the London Drug Policy Forum and now of the Community Safety Team at the Corporation of London. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Last revised 18 May 2016. First uploaded 01 November 2011

Comment/query to editor
Give us your feedback on the site (one-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates