Send email for updates
Under the spotlight in this hot topic is an issue which bothers everyone from patients and their families to services and the commissioners of those services – why we see major differences between the outcomes of treatment services, and what can be done to make a treatment organisation more effective. The answers range from monitoring and further training to tighter contracting of services, and the promising ‘walk-through’ tactic of experiencing your own service from the client’s point of view. These may help, but in the end we conclude there is no substitute for commitment to the patients and clients you are there to help. Without this improvement systems may not even be tried and if they are, the service may do the least possible to meet the requirements of those exerting pressure on it rather than the needs of its patients.
Whole organisations are not easily manipulated by researchers to see if outcomes improve, so a major source of clues to improvement mechanisms comes from simply documenting relationships between characteristics of organisations and how well they do. This kind of data comes with the important rider that a characteristic can be linked to outcomes for reasons other than cause and effect. If this is the case, then engineering more of that characteristic will not necessarily improve things for clients and patients.
From the UK we have two studies have shown us that clients may engage best with treatment when services foster communication, participation and trust among staff, have a clear mission, but are open to new ideas and practices, and that non-conformist drug workers who value hedonism and stimulation may help socially excluded clients improve most because their values match those of their clients.
Streamlined procedures can get people in to treatment quickly, and make a difference to how many can benefit from the treatment. UK studies have found that rapid entry to methadone treatment enables more people to engage with treatment (as the longer someone waits, the less likely they are to start treatment), and reduces the amount of time they are at risk from illegal heroin use.
A comparative study of three private methadone clinics in Sydney in the 1990s highlighted the importance of good organisation for improving treatment effectiveness, as well as an ethos of individualised treatment. Methadone dose too seemed important for suppressing illegal heroin use, but dose was itself related to the service’s orientation to its treatment programme. Highest doses were prescribed by a service which acted as a ‘methadone dispensary’, while patients preferred the individualised treatment orientation of a service which on average prescribed moderate doses.
However, what service providers can offer is often subject to funders’ requirements. In one US study, being constrained by funders in terms of services and ability to individualise treatments was the clearest negative factor, and quality accreditation the clearest positive in what made for a good service, with good treatment outcomes for patients.
Engagement with specialist substance use treatment is also an opportunity for often marginalised patients to access primary care and mental health services. This opportunity can be facilitated or constrained by the organisational climate. The results of a 1995 US survey of unit directors and clinical supervisors in 618 outpatient drug treatment programmes found that when staff are overburdened, and quality standards are de-emphasised, access to primary care and mental health services can be compromised.
Within non-specialist services, it is critical that staff feel treating patients with substance use issues is legitimately their business. This message emerged from the classic British studies conducted by Alan Cartwright and colleagues of the Mount Zeehan alcohol treatment unit in Kent. From the early 1970s their government-funded Maudsley Alcohol Pilot Project opened up an agenda centred on the therapeutic alliance between patient and helper. In this and other work, the researchers found that the helper’s commitment to working with drinkers (a key factor in alliance formation) depended on the workplace environment, including whether it engendered the feeling that this was a legitimate role supported by their organisation. Staff who felt that working with drinkers was ‘not my business’ could not be trained into being committed therapists. Cartwright’s studies turned the focus on the messages staff received about the organisation’s priorities as expressed in its policies, resource allocation, and the perceptions induced in staff about whether working with drinkers was a valued and worthwhile use of their time.
Here we turn the smaller but arguably more significant corpus of studies which have tried changing organisational practices and/or the commissioning context to see if these changes actually do improve performance – though performance is sometimes measured by proxy indicators such as retention in treatment rather than remission of substance use problems. In the process we will come across a bind which needs untangling – that services most in need of reform are likely also to be those unaware of this and not ready to improve.
There is tentative evidence that the introduction of performance contracts can improve provider performance and the effectiveness of services. These incentivise providers to shift their thinking from delivering units of service regardless of their impact on outcomes, towards delivering services which actually improve the clients’ condition and by doing so earn the incentives or satisfy the requirements of the contract. This is the way all substance use services were commissioned in the US state of Maine from 1992. Over a four-year period, effectiveness improved significantly across outpatient and residential programmes, though not for underrepresented or ‘difficult’ client groups. Though the contract specified required outcomes, it also required a certain amount of services to be delivered. Prior to the introduction of performance contracting, providers delivered more services than were contractually required; afterwards, they delivered close to the contracted amount. It seems providers were focusing on delivering the core services they were paid for, in place of delivering extra or ‘optional’ services clients previously had access to.
Adherence to a treatment protocol is a way of ensuring that treatment is standardised across the service, promoting consistently high quality. However, a tension arises if uniformity is enforced in the face of patient differences, as explored in this Effectiveness Bank hot topic about the individualisation of care. Local conditions too need to taken into account. Researchers monitoring the implementation of a structured behavioural therapy in rural Midwestern America found that therapists being trained in how to implement a new treatment intervention wanted to know how to implement the protocol in their specific setting, with their specific clients, under their specific organisational conditions. Before this, the therapists seemed to lack interest in the new treatment during training. Only when the research team tailored the implementation by responding to local conditions did therapists become willing and able to invest time and energy in learning to execute the protocol.
Administrative advances have enabled caring services to create and take opportunities to be caring in practice. A study of US services observed that intake assessments of patients’ needs were largely redundant paperwork which led nowhere in terms of meeting those needs. Linking assessments to a computerised guide to local welfare and medical services transformed them into a practical route to obtaining services matched to assessed needs – and treatment completion rates doubled.
Programme monitoring can be used to assess the quality and fidelity of interventions – how true they were in practice to the intended model. In this study of family-focused prevention programmes delivered in health care settings, programme fidelity improved over time, and a key conclusion was that fidelity assessment data can be successfully used to generate ongoing improvement in the quality of programme delivery.
Another way to hold up a mirror to a service is to use ‘walk-throughs’ – for senior staff to place themselves in the patient’s shoes and feel what is and isn’t working, rather than imagining all is well. That’s what staff did at 327 US services, as well as role-playing a relative of the client. An analysis of the ‘walk-throughs’ – which started with the first phone or other contact and extended to the early stages of treatment – showed that the role-players experienced poor staff engagement and impersonal interactions, shortcomings in equipment, administrative procedures and premises, poorly communicated information, burdensome and repetitive processes and paperwork, including lengthy intake interviews focused not on the client’s needs, but those of the agency, and failure to provide for clients with complex lives and problems. Extended to another 12 US areas, walk-throughs by senior staff became the key tactic for identifying service delivery problems and improving clinical processes.
Often the mirror through which an organisation can see reflected its strengths and weaknesses is constructed by an external agency to which treatment services must or choose to submit themselves for accreditation, but a review by two of the world’s most respected addiction experts judged accreditation a weak lever for improving outcomes. Instead the US experts favoured engendering motivation for change by subjecting agencies to market forces, of which in the UK the most prominent examples are ‘payment by results’ schemes which pay services for achieving certain outcomes. Such schemes are intended to force services to be effective or face financial penalties. They can force change, but sometimes only that required to gain the externally imposed carrots and avoid the sticks.
For any service improvement programme to work, the service has to engage in it and take it seriously. Australia’s internationally recognised addictions workforce development agency has pointed out that first an organisation has to want to change, yet the factors which mean a service is functioning poorly and failing its clients may also mean it is unaware of these deficits and that it neglects ways to improve.
In the end it seems, it comes down partly to a service ethos which cares enough to take care that what it does and how it does it really does help its clients and patients
One way to square this circle has been trialled in the USA, using staff surveys to alert the service to how its staff see it and how this compares with other services. Faced with the evidence, senior staff from agencies which scored as less open to change and to staff suggestions were the ones most likely to commit to change. However, the study did not assess whether they actually followed through on this commitment.
Ideally health services and charities whose mission is to serve will willingly open themselves to influence and scrutiny and embrace improvements, but the ones doing least well in that mission are probably also the ones least likely to take those steps. External pressure seems the solution, yet the same organisations may react by doing just what is needed to satisfy their funders or inspectors (which may bear a loose relationship to patient welfare) rather than engaging in a sustained improvement programme focused on the needs and aspirations of their actual and prospective patients. The market mechanism of patients voting with their feet is often not an option, increasingly less so as mega-services take over in local areas, offering to do everything for the commissioners. In the end it seems, it comes down partly to a service ethos which cares enough to take care – that what it does and how it does it really does help clients and patients.
Run this search for evidence that the organisation does matter, and for clues to how to make it more effectively service its therapeutic objectives.
Last revised 15 May 2016. First uploaded 01 August 2011
Comment/query to editor
Give us your feedback on the site (one-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates
HOT TOPIC 2016 What about evidence-based commissioning?