Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment
Effectiveness bank home page. Opens new windowResearch analysis

This entry is our analysis of a study considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.

Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text
Copy title and link | Comment/query |

Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment.

Hoffman K.A., Ford J.H., Choi D. et al.
Drug and Alcohol Dependence: 2008, 98, p. 63–69.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr McCarty at You could also try this alternative source.

Placing staff in the clients' shoes was the key tactic in this national US treatment improvement programme which more than halved waiting times and increased retention without limiting patient numbers.

Summary In a joint attempt to improve the quality of substance use treatment services, the US government and a major US philanthropic foundation co-funded the Network for the Improvement of Addiction Treatment. Treatment providers bid for funding from either source to help improve efficiency and access to and engagement in their services by:
• reducing the waiting time between the client's first treatment-seeking contact and the start of treatment;
• cutting the number of missed appointments ('no-shows');
• increasing the number of clients admitted to treatment; and
• increasing the proportion of treatment-starters retained for at least four treatment sessions.

Interventions were tailored to each service but developed using the same model. It entails identifying key problems and possible solutions by involving service users through focus groups, satisfaction ratings and advisory boards, and by 'walk-throughs' during which staff role-play a client and experience existing procedures from their point of view. Possible solutions to each problem are briefly piloted and evaluated, then rapidly revised and retested until effective strategies are identified or the change cycle is abandoned, a process driven through by a suitably skilled staff member with the authority to make the required changes. Apart from substantial For each site initially $100,000 a year for 18 months or three years, and in the replication study $100,000 over 18 months for the 13 of the 15 sites awarded a full grant. grants, each site was phoned weekly and visited quarterly by a 'coach' trained (usually not specifically in respect to alcohol and drug treatment) in process improvement, who helped the service learn and apply the rapid improvement cycle. Participating services could also learn from each other via the project's web site and met annually to share successes and failures, the intention being to build a self-sustaining improvement network. Centrally developed systems helped services monitor the impacts of the changes they piloted.

The featured report provides longer term data from 15 Six services did not provide sufficient data. Services which did contribute data consisted of seven non-residential services, four intensive non-residential services, and four residential services. of the 21 non-residential and residential services awarded grants in 2003, plus data from a further 14 Two of the awarded organisations did not provide sufficient data. Services which did contribute data consisted of 10 non-residential and four intensive non-residential units. of the non-residential services awarded grants in a second round of funding in 2005. An earlier report on the initial funding round had found significant improvements over the first 15 months. By the end, treatment started a week sooner (down from on average 19.6 days to 12.4) and another 11% (up from 62% to 73%) of clients who attended an initial treatment session went on to attend at least three The proportion attending at least four sessions also improved (from 54% to 60%) but not to a statistically significant degree. in total. An appendix to the main study did not reveal any obvious trends in the numbers of admissions. Concerns that early gains might not be sustained (there were slight reversions by the end of the 15 months) were addressed by collecting another 20 months of data. Documented in the featured report, the results showed that overall The non-intensive, non-residential services actually significantly improved further on nearly all measures. the services maintained improvements in waiting time and retention; there was no significant weakening, nor any further improvements.

Results were similar when the same procedures were tested on the new set of second-round non-residential services. Over the 18-month observation period, waiting times fell from on average nearly 31 days to 19, while at each yardstick (from the first treatment session to the second, third or fourth) another roughly 10% of clients were retained. By the end of the period, nearly 68% of clients attended at least four treatment sessions compared to 57% at the start. Also reported was a near halving in the proportion of appointments missed from 22% during the first three months to 13% in the last three, and a modest increase in admissions from 21 a month per service to just over 23.

Among the first round of services, strategies to reduce waiting times included on-demand scheduling and next-day admissions. Other admission improvements included simplification of intake and assessment processes, longer openong hours, elimination of redundant paperwork, cross-training, and enhanced telephone responsiveness. Among the changes used to enhance retention in care were reminder calls, changes in appointment times, motivational interviewing training, and introducing clients to their counsellors before the first treatment session.

Beyond these specifics, reports from the services and interviews with staff suggested that networking and annual meetings built strong and persisting collaborations between services in different regions, enabling services to replicate changes found successful elsewhere. Change processes initiated by the project fostered an overall customer service culture at the agencies, to which walk-throughs (when staff role-played clients) were a major contributor. How revealing these were became apparent during the funding application process. For some of the initial grants, this required services to conduct a walk-through of their admissions process, with staff taking on the role of a prospective client and a member of their family. The results were reported back to the funding agency. An analysis of 327 such reports revealed poor staff engagement with clients and impersonal interactions, shortcomings in equipment, administrative procedures (such as poor phone systems) and premises (which often lacked privacy), inconsistent or badly communicated information, burdensome and repetitive processes and paperwork (including lengthy intake interviews focused not on the client's needs but those of the agency), failure to provide for clients with complex lives and problems, and an inability to schedule intakes in a timely manner.

The authors concluded that the changes stimulated by the initial round of funding had become embedded in the services and sustained even after funding ended, while short-term findings from second-round services enhanced confidence in the replicability of the procedures and their ability to create improvements in treatment access and retention. However, there was substantial variation between services in the degree to which they were able to engineer improvements, variations which seemed attributable to the nature of the services rather than the nature of their clients.

Findings logo commentary Projects like the Network for the Improvement of Addiction Treatment which aim to change the culture of services should help overcome the limitations of change driven purely by targets or financial incentives, which encourage services to do only what is required to satisfy target-setters or funders, rather than whatever is required to improve the service for its users. Reassuringly, improvements in waiting times and retention were not gained by simply cutting down on admissions, though the aim to actually increase these was at best achieved only modestly, partly (the authors suspect) because it would not bring extra money to the services.

The value of networks of the kind established by the project seemed evident also in another US study, which found that by far the single factor most closely related to whether criminal justice and allied treatment services adopted research-based quality improvements, was the degree to which they networked and carried out joint activities, especially with other treatment programmes. Also related were training opportunities and management stress on quality. The implication was that the most fertile ground for quality-improving innovation in the drugs/crime sector is likely to be an active network of not very large service providers and criminal justice agencies with managements committed to quality improvements.

Unfortunately the featured study was unable to test whether improved access and engagement translated in to further improvements in the resolution of substance use problems. In general people in need of addiction treatment do better if they get it and if they participate more fully in that treatment, but the relationships are often loose. Studies often find that treatment participation and retention With the notable exception of retention in substitute prescribing programmes. are unrelated or only poorly related to post-treatment substance use. Initiatives which improve engagement may have no noticeable effect on outcomes.

Another major question mark over the project is (as the researchers acknowledged) the representativeness of the services which made it through the application process. They were a small minority Across both rounds 595 applications resulted in just 38 awards being made. even of the services which felt they were close enough to meeting the criteria to apply. Successful applicants were heavily pre-selected for their potential ability to implement and document service changes and their commitment to customer-focused improvements. Ten of the 23 initial awards and all those in the second round were made partly on the basis of the service's performance in actually implementing a key improvement procedure, a walk-through of their admissions process. Despite this sifting, there remained substantial variation between the selected services in the degree to which they were able to engineer improvements. The authors remarked that agencies already under stress, with inconsistent leaderships or unstable finances, often abandoned change efforts. Even with the project's support and funding, several were unable to adequately monitor access and retention, a prerequisite for data-driven improvements. Such difficulties even after thorough sifting raise considerable doubts over whether most services would be able to profit from a similar exercise. It may also be relevant that the project was a nationwide one, meaning that services were presumably rarely in the position of offering good ideas to local competitors. Improving financial health is it seems The project' web site at is headed "Financial Strength in a changing World" and the campaign is described as one aiming to "Improve services, reduce costs, increase revenue". a major motivator for services to engage in the project's improvement process, but one which could also impede it due to competitive pressures.

These features may have atypically enhanced the project's impact. Set against this is the probability that pre-study sifting had an opposing effect, limiting the scope for improvements because selected services were already doing relatively well. The application process was designed to identify services receptive to innovation and with the infrastructure, organisation and stability to see it through. Even without help of the kind provided by the project, in England such services were associated with greater participation in treatment by their clients.

The baseline from which the study sought to measure improvements would have been raised further among services required to conduct a walk-through exercise as part of the application process. Focused on the admissions process, this almost certainly substantially reduced waiting times even before the study started monitoring the services' performance. These services started the study with a waiting time of on average just eight days and then made no further improvements. Starting from a lower baseline, services which had not been through this exercise more than halved waiting time from 23 days to 10. The figures seem to demonstrate the power of explicitly and systematically addressing admission procedures via a walk-through, even without the funding, networking and technical assistance available later in the project. Including the pre-study walk-through, it seems a fair assumption the project as a whole actually led to something nearer a 60% reduction in waiting times rather than the 37% it was able to document.

Related to this is the degree to which even these heavily selected projects would have been able to implement all the changes targeted by the project without its substantial financial and technical support. A subsequent phase of the project will test whether services do as well with only modest or no financial awards and different levels of technical and networking assistance.

Services which want to profit from the project's experiences will find a practical guide to the process improvement procedures tested in the study at the project's web site, along with ideas for solutions to common problems.

Thanks for their comments on this entry in draft to Dennis McCarty of the Oregon Health & Science University. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Last revised 02 September 2009. First uploaded

Open Effectiveness Bank home page

Top 10 most closely related documents on this site. For more try a subject or free text search

STUDY 2009 The Drug Treatment Outcomes Research Study (DTORS): final outcomes report

STUDY 2012 Delivering service quality in alcohol treatment: a qualitative comparison of public and private treatment centres by service users and service providers

STUDY 2011 Performance-based contracting within a state substance abuse treatment system: a preliminary exploration of differences in client access and client outcomes

STUDY 2010 A randomized controlled study of a web-based performance improvement system for substance abuse treatment providers

STUDY 2012 The forgotten carers: support for adult family members affected by a relative's drug problems

STUDY 2008 Does following research-derived practice guidelines improve opiate-dependent patients’ outcomes under everyday practice conditions? Results of the Multisite Opiate Substitution Treatment study

REVIEW 2011 Alcohol-use disorders: diagnosis, assessment and management of harmful drinking and alcohol dependence

MATRIX CELL 2018 Drug Treatment Matrix cell D3: Organisational functioning; Medical treatment

STUDY 2005 Abused women gain more from holistic counselling

STUDY 2012 Innovation adoption as facilitated by a change-oriented workplace