Send email for updates
Marsh J.C., Shin H-C, Cao D.
Evaluation and Program Planning: 2010, 33(2), p. 81–90.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Marsh at firstname.lastname@example.org. You could also try this alternative source.
From the comprehensive treatment process data collected by a major national US study emerges the important lesson that retention in itself is not an active ingredient in post-treatment outcomes but reflects influences such having one's needs met (especially important for women) and developing a good relationship with the service and your key worker.
Summary The US National Treatment Improvement Evaluation Study collected data between 1993 and 1995 from a nationally representative sample of treatment programmes funded by the US government, serving mainly vulnerable and underserved populations including minorities, pregnant women, young people, public housing residents, welfare recipients, and those involved in the criminal justice system. It remains one of the few treatment effectiveness studies to have collected detailed information from clients on their relationships with the treatment provider and their receipt of services.
The featured analysis used this data to test whether stays in treatment and drug and alcohol use a year after leaving were (as in other studies) related to the intensity of the following types of services/processes as reported by patients and clients when they left treatment:
• Access services like child care and transportation intended to improve access to treatment.
• Substance use counselling and other services (12-step meetings and medications for alcohol/drug problems) intended to directly reduce substance use.
• The service-needs ratio assessed the degree to which the needs for services Family and life skills services such as parenting, domestic violence counselling, family services, assertiveness training, life skills, family planning, and non-medical pregnancy services; health services including AIDS prevention and medical pregnancy services; mental health services; and concrete services such as schooling, job skills, housing, help collecting benefits, training in English, help getting alimony/child support. patients expressed when they started treatment were met by actually receiving those particular services.
• Client-provider relationship: whether a positive therapeutic partnership had developed between the treatment service (in particular the client's key worker) and the client, assessed via ten questions, Whether client had seen a treatment plan; helped develop the plan; agreed with treatment; adhered to treatment goals; whether important provider existed; time spent with primary provider; length of sessions with primary provider; agreement with primary provider; understanding by primary provider; whether primary provider speaks preferred language. to most of which the client could indicate magnitude or degree.
The analysis aimed to unpick For example, it was expected that longer retention would help reduce later drug use because it gave more time for needed services to be received, and that receiving needed services would incline patients to stay longer. Women generally have greater needs and more need for transport and child care in order to access services so these might be more influential for women than for men. not just the presumed effects of these variables, but also how they worked, and whether effects and mechanisms differed for men and women and were affected by other characteristics of the client or service.
The source study had interviewed 6593 patients when they started treatment. Of these, the featured analysis included 3027 from 59 services who had completed all intake, treatment discharge, and follow-up interviews, were not in prisons or jails, and had expressed some need for services at treatment intake. Mainly they were problem drug users who may also have been drinking heavily. A year after leaving treatment they had told researchers the number of days in the last 30 on which they had used the five most frequently used substances: alcohol; cannabis; crack cocaine; cocaine powder; and heroin. These use days were summed to give an index of the intensity of their polydrug use. The fact that the variables presumed to influence substance use were measured a year before means that they could have had a causal influence, but the study was unable to be sure of this because other influences could not be excluded.
Models were constructed of how mechanisms and influences interrelated to influence retention and post-treatment substance use. One was made for the full sample, but the same model could not be applied to men and to women – the processes were too different.
For the full sample, post-treatment substance use was significantly lower when a greater proportion of the patient's needs had been met during treatment, they had a better relationship with the treatment service/keyworker, and they had stayed in treatment longer. Further upstream of the causal chain model, facilitating access to treatment by providing transportation and child care led to receipt of a greater range and intensity of substance abuse counselling and other treatment services, which in turn led to an improved relationship between service and client. Improved relationships then seemed to lead to patients staying longer in treatment, and both were related to having more of one's needs met.
Among the 'fixed' factors affecting these processes were that residential treatment led to longer retention and more counselling and other treatment services being received. The more educated the client the less their post-treatment substance use, while the greater use was before treatment the greater it tended to be afterwards.
As assessed by surveying programme administrators, some other organisational factors did not emerge as influences in the model, including whether the service was accredited, the typical intensity of counselling scheduled (as opposed to actually received by each individual) for patients, and the number of extra services Academic training, vocational training, medical, psychiatric, or pregnancy services. provided on-site.
For men the structure of the model was very similar to that for men and women combined, with the exception that time spent in treatment was no longer related to post-treatment substance use. For women this was also the case, and there was another difference both from the full sample and from men, reflecting an apparently more pervasive influence of transportation and child care access services. These not only appeared to facilitate access to more types of substance use counselling and other treatment services, but also When the criterion for declaring a finding as statistically significant was relaxed to a one in ten rather than a one in twenty chance of it bring a chance finding. Similarly relaxing the criterion for men did not lead to the same relationships emerging in the model. led to an improved client-provider relationship and a greater proportion of needs being met. Another difference for women was that a better client-provider relationship had no direct influence on post-treatment substance use, though it did influence substance use indirectly via a greater proportion of needs being met figure.
Several services and service delivery mechanisms – client-provider relationship, access services, substance use counselling and other treatment services, and services matched to needs – individually and collectively contributed to retention in treatment and reduced post-treatment substance use. While mechanisms affecting outcome are fundamentally the same for men and women, access services are particularly important for women, and a positive client-provider relationship is directly related to reduced substance use for men but only indirectly for women.
The study's modelling techniques helped establish not just which factor is related to another, but also which might be cause and which effect. These analyses suggested that a positive client-provider relationship leads to better matching of services to needs and to longer treatment, but not the other way round. The implication is that when providers develop a constructive relationship with clients, they are able to more effectively identify and meet service needs and encourage clients to remain in treatment. These findings are consistent with a review which concluded that in substance use treatment, client-provider relationships consistently predict retention but less consistently predict substance use outcomes.
Another question is whether the client-provider relationship is therapeutic in its own right, or primarily a vehicle for enhancing access to and impact of specific services. Modelling results suggest this relationship is both directly connected to reduced post-treatment drug use and indirectly related via greater need-service matching and longer treatment stays.
These linkages do however differ for men and women. The client-provider relationship is directly therapeutic for the total sample and for men, but among affects post-treatment substance use only via receipt of needed services, which for women more strongly predicts outcomes than the relationship and more strongly than for men. This may be because women start treatment with more needs, so addressing those is especially important.
The analysis was also consistent with receipt of transportation and child care leading to receipt of more substance use counselling/treatment services, which led to a higher quality client-provider relationship. It seems these access services may be a necessary precondition for receiving more substance use counselling, which in turn gives clients the opportunity to connect with their provider and work on treatment goal-setting, planning and bonding, which in turn means more needs are met, encouraging or enabling longer retention. For the total sample and for men, the result is to reduce post-treatment drinking and drug use.
Access services were identified in the models as especially valuable for women, perhaps because they are more likely to face barriers to treatment entry including lack of transportation and problems with child care. Not only does providing these facilitate access, it also improves treatment in ways which improve substance use outcomes.
Remaining in treatment for more than three months has been considered a robust predictor of reduced post-treatment substance use. The featured analysis found that when other variables are taken in to account and findings for men and women analysed separately, retention is no longer directly related to substance use, but acts via improved matching of services to needs. The implications are that treatment duration may in previous studies have served as a proxy for receipt of services.
These findings emerged from a resicted sample of treatment programmes and from a restricted sample of the patients who started treatment in those programmes; they may not be representative to all substance users in need of treatment. Findings from the source study derive mainly from public sector programmes serving lower-income groups and may not generalise to other programmes and caseloads. They also date from the early '90s, since when treatment services have changed, though perhaps not in ways which invalidate the core conclusions. Finally, the statistical techniques used to construct the models can eliminate some theories about how treatment works, but not confirm that the final model actually reflects cause and effect. This inherent limitation of the techniques is compounded by the fact that the main treatment process information was all collected at one time point (discharge from treatment); ideally the presumed causes would have been assessed before the presumed affects.
commentary Logical, intuitively appealing and in line with other research as the conclusions of the analysis are, it is important (as the authors acknowledge) to remember that the most which can be said is that the cause-effect explanations are consistent with the data. It cannot definitely be said that this is in reality how the treatment process worked for these US clients and services in the early '90s. Only studies which deliberately varied, for example, the quality of the client-provider relationship (while keeping everything else constant) could confirm that directly or indirectly this relationship causes the resulting changes in retention and substance use. Such studies are however not very feasible because they would mean deliberately consigning some patients to a substandard treatment experience and possible substandard outcomes in a situation where lives are at risk from relapse. Failing such studies, careful and sophisticated analyses such as the featured study offers, based on relatively comprehensive data from real-world treatment services, can help elucidate important variables affecting the success of addiction treatment. More methodological considerations below.
Findings from this analysis were foreshadowed by an analysis from the same source study focused on receipt of services matched to needs. It found that receiving services matched to need was associated with greater reductions in illegal drug use generally and use of the drug(s) in relation to which the patient had sought treatment. This was the case for each of the needs separately (except for mental health) and for the extent to which each individual's overall needs had been addressed. The strongest links were with housing and vocational help and among patients at residential services, where these particular needs were most likely to be addressed. Matching services to needs was linked to improved outcomes partly (but not entirely) via a link with increased retention. However, these associations were confined to the half of the patients with multiple needs across at least four out of the five domains.
More generally, research is supportive of the attempt to match the intensity and type of help to patients' needs, but studies are few and usually the impacts on substance use have been moderate. Research is strongest in regard to providing inpatient care and professional psychotherapy for patients with distinct but not disabling psychiatric problems who also have fewer 'recovery resources' in the form of employment opportunities and a supportive family. The relative prominence of research on psychiatric severity and psychotherapy may be a function of the comparative lack of investment in meeting patients' needs for housing and employment, which are also more difficult to engineer. Despite the difficulties, studies do suggest that providing such services improves outcomes in the targeted areas and also in respect of substance use problems.
A Findings review has explored the impact of (among other forms of practical help) the access services found so influential in the featured analysis – transport and help with the child care. It concluded that practical help to overcome access obstacles directly improves retention and also shows that the service is responsive and caring. Transport seemed most important for impoverished populations required to attend methadone services daily for supervised consumption. Direct help in the form of a driver and vehicle worked best, probably because it provides an escort and structures the patient's day. Providing transport was found to augment efforts to link patients to external agencies such as housing and employment services. The review also noted that for many women, child care is essential if they are to be attracted to and retained in treatment, especially in long-term residential care, but may not be used if it is unfamiliar or seems to threaten the mother's custody of the child. Beyond child care and transport, flexible and realistic opening hours and attendance requirements meant patients with unpredictable lives were not set up to fail, and allowed others to maintain normal family and working lives.
It is unclear whether characteristics of the patients before they started treatment influenced both retention and outcomes and treatment processes in ways which could have created what looked like a causal relationship between processes and outcomes. This is especially relevant in respect of variables not entirely under the control of the treatment service. For example, clients who were particularly motivated or well placed to overcome their dependence might have developed better therapeutic relationships, been more diligent in attending the service's counselling sessions, and attended 12-step meetings outside the service. They may also have stayed in treatment longer and done better in terms of controlling their substance use after treatment, but perhaps mainly because of their pre-existing attributes rather than treatment processes. Similarly, the measures of child care and transportation services were measures of the receipt of those services, not just their provision by the treatment agency. Receipt may reflect not just organisational concern to help patients come to treatment, but also the eagerness of the patient to ask for these services so they can attend.
Such considerations may explain (for example) why the counselling sessions scheduled by the treatment service and whether it provided on-site access to ancillary services (reflecting the organisation's provision, not their use by the patient) had no impact on retention and substance use a year after leaving treatment, while whether the patient actually attended counselling sessions and whether they actually accessed needed ancillary services (potentially reflecting their impetus to get better) did have direct or indirect impacts.
The restricted sample is also a concern. About 85% of patients starting treatment completed intake interviews for the source study. From this interviewed sample were eliminated another 2179 who did not complete subsequent interviews or expressed no needs at intake. The remaining sample constituted about 57% of all patients starting treatment (from whom were then excluded those in prison or jail). Loss to the sample was unlikely to have been entirely random. In so far as it was related to the factors used to account for treatment outcomes, estimates of the effects of these factors will be biased. For example, assume (not unrealistically) that patients who developed a very poor relationship with their providers not only tended to leave treatment very early but also failed to complete all research interviews. In this scenario the potential impact of this relationship on retention – estimated on the basis of the patients who did complete all interviews – will have been underestimated.
Last revised 12 March 2013. First uploaded 12 March 2013
Comment/query to editor
Give us your feedback on the site (one-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates
STUDY 2002 The grand design: lessons from DATOS