Send email for updates
Brucker D.L., Stewart M.
Journal of Behavioral Health Services and Research: 2011, 38(3), p. 383–397.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Brucker at firstname.lastname@example.org. You could also try this alternative source.
In 2007–08 the US state of Maine introduced a new scheme directly linking funding for outpatient treatment services to performance in terms of waiting times and retention, but financial and service delivery impacts were negligible. Were the incentives too weak, or were services already doing as well as they could?
Summary US single state agencies are public bodies responsible for the coordination of substance abuse services in states and territories, in particular for clients who cannot fund their own care. Most do not provide services directly, but purchase from community-based systems of care. Performance-based contracting is one way they can try to improve quality. It involves offering direct financial incentives to service providers as long as they meet pre-determined levels of performance on defined indicators.
Maine was one of the first states to implement such a system, providing an opportunity to examine the relationship between services being paid in this way and their performance, in this case on measures of access to and retention in treatment. Performance-based contracting was introduced in 1992 to shift the publicly funded treatment system from a focus on outputs to outcomes. Results were mixed; providers reduced certain services, yet reported some better outcomes. Possible 'cherry picking' and 'gaming' were suggested by inconsistencies between client data reported for the payments system and treatment service medical records.
To address these criticisms, Maine restructured its system, with effect for outpatient services in financial year 2008 (1 July 2007 to 30 June 2008). The aim was to foster efficiency and quality of service by tying performance to actual payment level. First agencies were financially rewarded or sanctioned for exceeding or missing the number of 'units of service' they were contracted to provide, an attempt to prevent cuts in patient numbers. Then 'access' or waiting time targets were set at different levels for standard and intensive outpatient services: typical Specified as the median, meaning the middle value in the ranking from greatest to least delay. times from first phone contact to assessment were to be no greater than five and four days, and from assessment to the start of treatment, no more than two and one week.
Retention targets too were set: standard services were to retain at least 50% of new patients for at least four days and 30% for at least three months. Operationalised as 90 days. Corresponding targets for intensive services were 85% for at least four sessions and 50% to complete treatment. For Maine, 'completion' meant that the patient had achieved at least two thirds of their treatment plan Plans were to include objectives specific to the client, for example: abstinent during treatment; significant reduction in problem use; willingness to voluntarily seek continued care as necessary; participation in self-help. before an agreed discharge.
By exceeding or undershooting these targets, services stood to gain/lose 9% of their contracted fee. Assessments were made quarterly and payments adjusted the following quarter.
In financial year 2008, around 5000 adults started treatment at the 17 services in the new payment scheme. Data from them in respect of retention was contrasted with the same services the year before the new scheme was implemented, and in respect of waiting times, with that from around 4000 patients at services not in the scheme. Typically, patients were white single men not in full-time employment with a drink problem, though intensive services saw slightly more patients (around 4 in 10 of the caseload) whose primary problem was opiate use. Over half also had mental illness diagnoses.
Generally services were not subject to substantial financial gains or losses as a result of the payments scheme. Over financial year 2008 they were budgeted to receive $3,531,364 and could gain $238,099 by exceeding performance targets, but only 19% of the incentive money was paid out. Across the year, payment adjustments for each agency ranged from about a loss of 7% of their contracted fee to a gain of 7%, averaging virtually zero and typically Median level of payment, meaning the middle value in the ranking from greatest loss to greatest gain. a loss of just 1%.
Based on raw figures unadjusted for caseload and other factors, non-intensive programmes in the payment scheme were significantly more likely to hit their waiting time targets than those outside the scheme: in respect of assessment, 61% versus 52%, and 92% v. 85% in respect of time to treatment. However, the reverse was the case for intensive services: corresponding figures were 69% v. 78%, and 86% v. 93%. The pattern of non-intensive services doing better if in the payment scheme, but intensive services doing worse, was generally similar in respect of average days patients had to wait for assessment or treatment.
Contrasting their pre-scheme (financial year 2007) to post-scheme (2008) performance, after joining the payment scheme both standard and intensive services recorded worse retention on all measures, often substantially and significantly worse. For example, from 40% of patients staying in standard programmes for three months the figure fell to just 24%. From 53% completing intensive programmes, the figure fell to 46%.
But further analyses revealed that these findings were not due to being subject or not to the payment scheme. Once other factors Number of older patients; sex; white v. non-white ethnicity; married/cohabitating or not; full-time employed or not; co-occurring mental illness or not; health insurance v. none (a proxy for other unmeasured socioeconomic factors); referred from criminal justice sources or not; agencies with large versus smaller caseloads. had been taken in to account, on no measure of either waiting time or retention was being in the scheme associated with a statistically significant difference.
While being in the payment scheme made no apparent difference, retention was significantly related to other factors. For example, across both assessed years (2007 and 2008), significantly more likely to stay for three months at non-intensive services were women, criminal justice referrals, those also mentally ill, or employed full time. Less likely to stay three months were patients whose primary drug was cocaine or those at agencies with large caseloads. In the year when the payment scheme was in operation, full-time employees were also more likely to complete their treatments at intensive services than other patients, as were white patients.
As a whole, the results presented here suggest that as implemented in Maine in financial year 2008, performance-based contracting had only minimal effects on agency reimbursement and no effects on time to assessment, time to treatment, patient participation, length of stay, or completion of treatment. Financial and service delivery impacts were negligible.
Why this non-impact? First, whether or not subject to the new payment scheme, agencies as a whole were already doing well on waiting times for assessment and treatment. Also the financial consequences of meeting or not meeting targets were in practice very small and limited even in worst/best cases to 9% of base funding, perhaps insufficient to move agencies to adopt policies and procedures to improve performance.
On the other hand, there was no evidence that (unlike the earlier scheme) the new scheme led agencies to limit admissions to the most promising patients, a particularly important finding. Also, implementing the scheme meant that, with providers, the state agency could improve accountability and reinforce organisational focus on access to and retention in treatment. Just implementing a performance-based contracting system which is both operationally effective and accepted by services is an important achievement.
In interpreting these findings it should be borne in mind that they are limited to the last contact the patient had with the service. Post-treatment data are not collected at state level.
commentary It is tempting to say that the Maine scheme was accepted by services because it apparently demanded little of them they were not already achieving, so made little difference financially. Across health care, patients often do not comply with remedies which require significant lifestyle change. Many even fail to regularly take pills which could prolong their lives. Patient resistance sets limits to the degree of compliance the treatment service can achieve, even when this is specifically targeted. Unrealistic targets may either lead services to cheat or to penalties so severe that what may be the only accessible treatment service in an area is curtailed or closed down, an attempt to improve services which would end up making things worse for the patients.
The Maine scheme was included in a review by US authors of ways to improve performance of substance use disorder treatment systems. Despite Maine's unpromising experience, they favoured the same types of schemes based on during and in treatment measures (but including substance use, not just attendance) because these most closely reflect the influence of the treatment, focus the service on patient progress indicators which it should in any event be monitoring, and do not require following up patients.
Maine's attempt included criteria similar to some on which funding has been made partly contingent in Britain, like 12 weeks retention (in Maine, three months) and successful treatment completion, in both jurisdictions entailing a planned discharge, plus other elements which differ. In Britain some criteria are applied at the level of a local treatment system (such as numbers in effective treatment, entailing 12 weeks retention or planned discharge) and others (as in Maine) at the level of an individual service, the latter most notably in the form of pilot payment by results schemes in England.
As in Maine, nationally agreed outcomes for the pilots often specify during treatment and treatment exit measures. Partially overcoming a limitation noted in the featured study, British schemes also attempt to balance during/end treatment measures against longer term crime and relapse indicators, generally choosing to place greater financial weight on the longer term. Onerous follow-up requirements are sidestepped by using routinely collected criminal justice and treatment records which do not require contact with the patient. Reports from the pilots suggest these schemes can both be feasible and generate innovations focused on achieving results.
However, this means the services in the pilot schemes must wait many months – in respect of some measures, nearly two years – to receive much of their funding under the scheme, a cash-flow problem which requires counter-measures if services are to survive. Arguably too, as the US reviewers commented, services are placed at financial risk for outcomes over which they have little control because they are so far from the time when they had direct influence over the patient.
The alternative of weighting in favour of during treatment and treatment end measures falls foul of the fact that these are often at best loosely related to the longer recovery the system is trying to generate. For example, 'successfully' completing treatment free of dependence and of opiate and crack cocaine use did mean that over the next four years more patients in England appeared to have avoided relapse, but the difference of 57% versus 43% who did not successfully complete was not as large as would be expected if successful completion correlated strongly with lasting recovery.
Above all, such schemes have left little or no room within their structures for patient-centred practice in the sense of basing treatment objectives on the patient's priorities. Instead they pre-set the treatment destination in detail without reference to what the individual patient wants, and in a way services cannot afford to ignore because their financial survival depends on meeting the criteria for payment. Local schemes could still create a space for the patient's ambitions in their payment criteria, but this is not a required element or one included in the English national outcomes schema, nor one which sits easily within a system predicated on observable outcomes the public and their representatives recognise and are willing to pay for.
Maine at first experienced what was suspected to be 'gaming' in the form of returns from services which did not match their own clinical records. Partly to avoid this, schemes which make payment contingent on client progress or outcomes often entail a regulatory overhead which could eat in to whatever efficiencies are achieved at the services concerned. For example, it has been argued that rewarded outcomes should be objective (eg, urine testing) and case-mix-adjusted. This means some authority has to assess the severity of the caseload in terms of the resources needed to achieve the intended outcomes and assess whether those outcomes have been achieved in ways which go beyond merely asking the patient. When funding, jobs and organisational survival ride on these assessments, leaving them entirely to the people and organisations at threat may stretch their integrity too far. This concern spawns new regulatory requirements and possibly new regulatory bodies which must, as the Audit Commission described, be capable of overcoming complexity to deliver valid and meaningful measures if disputes, demoralisation and wastage are to be avoided.
In English payment by results schemes the most visible result has been the setting up of central assessment centres (or LASARS), which have a key role in setting tariffs based on patient severity and verifying outcomes. These, say the Gaming Commission, should be independent both of treatment services and the commissioners of those services to make them less vulnerable to pressure to manipulate the figures (or at least the suspicion that this is happening), meaning a new body with its own overheads, which itself may require regulation. An independent assessment centre also places another step in the journey to accessing treatment during which access may falter. The plus side may be more efficient assessment, better treatment placement, and the potential for long-term case management to start at the assessment stage, but these possible advantages could have been achieved by centralising within existing structures. Alternatively, as in some drug pilots, the treatment services themselves can be trusted to act professionally in allocating clients to different need levels and treatments. Subject to clinical audit and review, this has been the method adopted by the NHS mental health payment-by-results scheme.
British practitioners and managers seeking to improve their practice have available to them the web site of the Substance Misuse Skills Consortium, an independent initiative led by treatment providers to harness the ideas, energy and talent within the substance misuse treatment field, to maximise the ability of the workforce, and to help more drug and alcohol misusers recover. Commissioners of services have been offered guidance from the National Treatment Agency for Substance Misuse, England's special health authority tasked to improve the availability, capacity and effectiveness of drug misuse treatment.
Last revised 29 September 2012
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates