Drug and Alcohol Findings home page in a new window Background notes

The Drug Treatment Outcomes Research Study (DTORS): final outcomes report


Unless otherwise referenced, this account is based both on the featured report and on earlier reports (1 2) describing the initial sample.

How representative was the sample?

DTORS effectively updates the NTORS study which in 1995 recruited 1075 clients. Methodologies were similar: problem drug users approached usual treatment services in the usual way and their progress was tracked regardless of whether they remained in treatment. However, differences in the way the samples were recruited demands caution in comparing one study's findings with the other. NTORS limited itself to methadone prescribing and inpatient detoxification or residential rehabilitation services, then seen as the major treatment modalities, and selected a typical set of such services with the proviso that they were in high drug use/service areas and had high caseloads. The resulting sample was not, nor was it intended to be, representative of all treatment-seekers. Moreover, the initial interviews were conducted by staff responsible for the respondents' treatments, not (as in DTORS) by researchers who assured them of confidentiality, a difference which could have affected the answers patients felt it advisable to give, and one seen as a weakness in the NTORS methodology. In contrast, DTORS aimed to recruit 3000 drug users presenting for treatment at all the structured drug treatment services in 100 randomly selected drug action team areas throughout England. Had the selection process worked as intended, the study would have started with what could be relied on as a truly representative sample of treatment-seekers, not just at prescribing or residential/in-patient services, but also in day-care and non-residential counselling.

In the event, DTORS too was unable to ensure a representative sample or achieve its 3000 target, and nearly three quarters of the sample could not be reinterviewed at the one-year follow-up. Especially in respect of the initial sample, this still left the study with the largest and probably the most representative drug treatment sample ever recruited in England. However, interpreting the findings must take in to account the sometimes substantial difficulties in recruiting and retaining them in the study.

The problems started when 26 of the 100 randomly selected drug action team areas refused to take part in the study and had to be replaced, while work in six more could not be completed. This left the sample short of a third of the intended areas, compromising the intent to select areas at random. Then some treatment services in those areas were refused approval or were unable to take part for other reasons, while problems securing contracts for fieldworker staff meant 40 NHS agencies – about 10% 342 agencies were eventually recruited; how many were NHS services is not specified. of all agencies in the areas – were unable to participate. The intention was to interview the sample during a four-week window but this was extended to five and then seven weeks. In addition, the study started recruiting clients seen first in the fortnight before the sampling window who had returned for further treatment, though few Personal communication from the DTORS team, January 2010. were added by this means. Depending on how these decisions to extend the window were taken and by whom, the result could have been to bias the sample in ways which could not be adjusted for by simply adjusting the results for the length of the window.

The representativeness of the initial sample also depended on treatment staff introducing the study to new clients and gaining permission for a researcher to contact them. The sampling window at each agency was widened and the period during which the windows operated seems to have been extended from the end of 2006 to March 2007, yet still far fewer attendees were recruited than expected. Though all were supposed to be asked to join the study, an unknown number were not. It is easy to imagine that staff might have been selective about introducing this extra complication in to delicate initial encounters with clients who seemed wary about confidentiality and unsure about entering treatment, a selectivity which could have systematically biased the sample. Of those who were identified as eligible by agencies, about two thirds (64%) were interviewed; most of the remainder refused to participate in the study.

In the end the study started with 1796 treatment-seekers interviewed at 342 treatment facilities across 94 drug action team areas. The extent to which the study could attempt to adjust for the possible sources of bias identified above was very limited. In respect of missing drug action team areas and individuals not asked to join the study, not at all; in respect of missing treatment agencies, only in terms of overall throughputs of clients and proportions referred from the criminal justice system. All these potential sources of bias might have been mitigated had the final sample been able to be weighted to make it match more closely the profile of treatment starters routinely recorded by the National Drug Treatment Monitoring System. But some DTORS participants could not be matched to that database, and others who could be matched had presented a quite different profile The cited report comments: "Another possibility considered was to calibrate to variables such as whether the client took heroin or crack, or had injected drugs. This information is available on the NDTMS database, so population characteristics could be calculated. However, not every responding individual could be matched to the NDTMS database.* Furthermore, although information on these variables could be assigned to each responder based on their interviews, there were large mismatches between the answers they gave to the question in their interview and their information on the NDTMS database. For example, only 82% of clients gave the same answer to the question on the use of heroin. Because of these mismatches, calibrating to these variables would not necessarily reduce bias and would have the effect of increasing the standard errors of the estimate."
* Roughly half of the reporting agencies supplied clients that do not appear on NDTMS. While some of these were matched up later, there was still a large proportion of DTORS respondents not on NDTMS.
to DTORS researchers than to the treatment staff responsible for collecting information for the national database. According to the detailed report on the initial sample, the upshot was that the only adjustments made were to cater for the degree to which DTORS' participants differed from the national profile in terms of age group, gender and whether they identified as 'white' or not. However, other reports (1 2) indicate that referral source and main drug were adjusted to match the national database profile. If they were not, or if the adjustments could not be relied on, in some respects (criminal justice referral; cannabis, crack and cocaine powder as main problem drug) the resultant bias might have been significant.

From this starting point emerged the later follow-up samples. Findings from these were subject to further layers of of adjustment Using standard weighting techniques. to attempt to correct them for the half of the starting sample not reinterviewed in time for the first follow-up, and the fact that nearly three quarters (72%) of the sample were not interviewed at the final follow-up. With such major loss of subjects to the study, the adequacy of these adjustments becomes crucial. Effectively they assumed that the missing people would have responded to treatment in the same way as people who (within the limits of the data available to the study) seemed similar, except that they could be followed up. But if two people who in every respect the study was able to measure are similar, except that one was available for follow up and the other was not, the suspicion is that they actually did differ in ways the study did not measure. These differences might also importantly relate to how well they responded to treatment; having in effect to estimate three quarters of the outcomes in this way adds substantial uncertainty.

Inevitably in such studies, assumptions have to be made about what would have happened if treatment had not been available. Implicitly (and in the case of the economic calculations, explicitly) the reports assumed that without treatment to seek, the drug users would have carried on as they had done before treatment. Perhaps, but also perhaps not; a third were under some form of criminal justice pressure, many of the rest would have been under other forms of pressure, and motivation to change their situation was high. But without treatment doors to go through to actualise their motivation, respond to life's pressures, and gain the support so many appreciated, it seems unlikely that their progress would have been as great as it was.

The nature of the caseload

Reports on the initial sample show them mainly to have been unemployed (77%) white men aged between 25 and 44 with often multiple and longstanding (71% had previously been in treatment) problems. The drugs with which they were experiencing problems were mainly heroin (72%), crack (47%) and alcohol (24%). Asked what they now hoped to gain through treatment, nearly three quarters (72%) said stopping all drug use and about half (49%) to sort their lives out. Two in five had recently been or still were living in unstable accommodation. Most (73%) had committed revenue-raising offences in the past year and about a third (35%) had been referred to treatment by the criminal justice system, generally (55%) having been ordered by the courts as part of a Drug Rehabilitation Requirement, though for a third (32%) treatment was a condition of bail. Of the 37% who had recently injected, nearly half (48%) admitted sharing injecting equipment. Over a quarter (28%) had received psychiatric treatment and nearly one in ten (9%) had overdosed in the past three months. About half (49%) had children under 16 years of age; three quarters of these parents lived apart from their children.

What did patients and staff think?

The featured report was supplemented by a report based on in-depth interviews with small, illustrative rather than representative This is not intended as a criticism. The samples were never intended to be representative but to enable the full range of factors, influences, views and experiences associated with the treatment of users of tier 3 and tier 4 services to be explored. samples of 32 treatment staff and 44 treatment-seekers who completed the second round of follow-up interviews. How sparse this sample was can be appreciated by the fact that just one client had been treated in inpatient detoxification and two in GP-run prescribing programmes.

In line with findings from the full sample, their accounts suggested that people arriving at treatment services via the criminal justice system were no less motivated than other treatment-seekers. There was also a consistent theme that delivery of a rounded and individualised service catering for the multiple needs of the clients was seriously impeded by:
• caseloads which over the years had risen and stayed longer in treatment, trends possibly due to national targets operative at the time;
• similar problems caused by the increased volume of criminal justice referrals, some of whom (despite the comments above) were felt to be unmotivated;
• a competitive environment which meant clients were not referred to other agencies for fear of the referring agency losing the money attached to high caseloads;
• poor partnership working with mental health services due to different understandings of the impact of drug use on recovery from mental illness;
• restricted access to accommodation.

Did society save money?

In financial terms, a further report estimated the net benefits for society associated with treatment-seeking and the degree to which each £ spent on treatment saved and improved patients' lives. The answer to the latter question was very little, and though cost-savings are likely, there were major difficulties and questions over their estimation.

This economic sub-study suffered from all the limitations of the parent study above. Additionally, it had to contend with very poor data on some of the costs treatment may have saved, and an inability to fully cost all the processes entailed in seeking treatment such as the involvement with the criminal justice system which for a third of the sample resulted in the current referral and treatment contact. The analysis also made the same questionable assumption as its predecessor NTORS that stolen/defrauded money and goods were lost to society, rather than simply transferred from one member of society to another. Since crime costs were the biggest single element in the calculations, and since a large proportion of these costs are due to these questionable losses, this assumption will have made a major contribution to the cost-savings side of the equation. In the absence of any data, the sub-study decided to assume that without seeking treatment, the drug users' situations would have remained unchanged, almost certainly not the case for many. Finally, offending behaviour reported for the last four weeks of a follow-up period was assumed to be representative of the entire period since the last interview. But if (as some of the data suggests) offending was falling in the first six months after treatment contact, this assumption would underestimate the level of offending and overestimate how much it had fallen. To an extent this may have been counterbalanced by an underestimation of offending preceding the baseline interview.

The product of these uncertain figures was an estimate that over about a year, seeking treatment saved about 0.05 (one twentieth) of a life year adjusted for the health-related quality of life of the patients, at a cost in treatment of £4531. This means that one quality-adjusted life year is saved (assuming all the assumptions and figures are valid) at a cost of £90,620, considerably in excess of the £30,000 yardstick commonly cited for cost-effective medical treatment. Underlying this shortfall were at best modest improvements in the physical/mental health and functioning components Physical function, limitations in role due to physical or emotional problems, the effect of pain on normal work and activities, general health, vitality, impact of physical or emotional problems on social activities, and mental health. of the quality of life measure. In terms then of the health-related quality of life of the patients assessed over this first year, treatment seems hard to justify. However, its financial standing is rescued by adding the savings experienced by society as a whole in the costs of publicly provided health and social care services and crime. With these roughly £11,000 savings per patient in the mix, the headline calculation is that in year one, treatment saved 2.5 times more than it cost, close to the 3 to 1 ratio estimated by the NTORS study in the mid-90s.

How did patients fare in respect of national reintegration objectives?

The study period just predated the 2008 English national drug policy which adopted reintegration through employment (enabling treatment exit) as its theme, so can be seen as a baseline report on how steep was the hill that policy had to climb. The answer seems to have been, very steep.

The first foothill to climb was that treatment seekers themselves did not prioritise this objective. At treatment entry most prioritised ending drug use. For half their goals did include (if generally not as a priority) "Sort life out/get it together", but just 1 in 5 specified employment as a way of sorting their lives out, and for just 1 in a 100 was this a primary goal. This was despite the fact that over three quarters (77%) were unemployed. It could be that they saw it as too early in treatment to contemplate such a goal, except that for 71% it was not actually early in their treatment careers because they had been in treatment before.

However, the question asked of the patients was about treatment goals, not life gaols. Unless services present themselves as employment facilitators, patients have little reason to see this as a goal to be achieved by going to those services. Consistent with this interpretation, three to five months later just under a fifth recalled receiving employment-related help from any source; presumably fewer still would have received this help from the treatment service. With few patients aiming for employment progress, few being offered help to progress, plus for many an unappetising CV, it is no surprise that little progress was made: 9% employed at baseline barely rose to 11% at three to five months and 16% at about a year, but the high proportion not followed up casts doubt on whether any progress was made at all, or whether it was just that employed people were easier to find and reinterview. Little progress was made too in laying the foundations for stable employment in terms of improved mental health and housing. The former would have been impeded by poor partnership working with mental health services, the latter was for some a major barrier to life changes, reportedly made intractable by the unavailability (physically or because of housing priorities) of suitable housing. In general ( above), in-depth interviews with clients and staff suggested that individualisation of treatment in response to broader client needs and aspirations was limited.

Is three months retention a threshold for lasting improvement?

The proposition that three months is a threshold at which the chances of lasting recovery take a step up is built in to the English national outcome monitoring system and in to the definition of "effective" treatment on which local funding largely depends. Research support generally for this proposition is questionable, and DTORS also questions its validity.

The main US research cited as backing this proposition in fact found no such threshold but a gradual improvement with longer stays, and then only clearly in long-stay residential rehabilitation programmes. In the UK, DTORS' predecessor NTORS is the other main support, but if there was a three-month threshold, it applied only to long-stay residential rehabilitation programmes, not to the bulk of UK treatment options. Even in respect of those programmes, the chosen threshold was simply the longest period tested in the calculations, and the only one which would normally have enabled treatment completion. For the prescribing and non-residential counselling These were included in the US study only and no threshold remained after adjusting for the variety of such services. services which together form the bulk of UK treatment options, neither study found a anything which even looked like a three-month threshold.

DTORS too offered little backing to a three-month threshold. Using all the follow-up interviews including late first follow-ups, the analysts plotted patients' outcomes at the follow-up interview against how many days they had spent in treatment since the previous interview. None of the plots relating to intensity of drug use, Proxied by past-week spend on drugs overall and on heroin or crack. crime Proportion of respondents committing acquisitive crime; average income from offending. or psychological wellbeing Mental health component of the health-related quality of life measure. showed a step up at three months, and all continued to improve up to five to six months then flattened out, though the pace of improvement The analysts argued that "These findings support the validity of the national performance indicator of retention in treatment for at least three months, but suggest potential value in longer measures of retention than currently employed as well as the need for treatment facilities to focus on a continuing process of change". may have been greater in the earlier months. The apparent point of diminishing or no further returns at six months could have been a methodological artefact. Outcomes after cumulative treatment of this duration could not have reflected the initial progress of patients interviewed at three to five months and would instead have substantially reflected that of patients whose interviews could not be conducted until after five months. It could be that this (presumably) hard-to-reach group were simply not doing as well as the people interviewed earlier.

Does treatment reduce crime by reducing drug use?

A puzzling finding cast doubt on whether the impact of treatment on drug use accounted for reductions in crime. This causal chain seemed supported by the way that over the follow-up period drug use, spending on drugs, crime, and income from crime, all fell, and generally the more so the more time a patient had spent in treatment. The obvious explanation was that crime was driven by the 'need' to offend to support drug use, and that once drug use was curbed, the need for crime and crime itself waned. The figures were consistent with this explanation. Before treatment, offenders typically spent £188 more per month on drugs then they legally earned, a shortfall expected to be filled by income from crime. But within three to five months, the same people were typically earning £140 more per month than their current drug spend, so had less need to resort to crime. The same processes should have resulted in a clear correlation between the criminal income of each participant at different stages in the study and the extent of their drug use. However, this was not the case: "Treatment appears to be associated with significant reductions in income from offending. However, no direct correlation with levels of drug use was distinguishable within these data". Depending on the exact measures found not to be correlated, part of the explanation may be that offending and drug use (though still substantial) had already been reduced before the study, either on the initiative of the treatment-seeker or because they were restricted by criminal justice supervision.

How did the different treatments compare?

Though as explained above, the study was unable to compare the effectiveness of the different treatment modalities, nevertheless it did compare the progress of patients in these modalities. Any progress differences found should be understood as possibly having as much to do with the nature of the caseloads as with the effectiveness of the treatments. Several outcomes were found to be unrelated to treatment modality: most measures of drug use reductions; improvements in employment; increase in legitimate income; level of acquisitive offending. Continued engagement in patterns of drug use which risked opiate overdose was more likely among patients who had entered prescribing programmes and less likely among those who had entered counselling programmes, possibly because the opiate-type drugs being prescribed meant that any further opiate use 'on top' counted as a risk. Being in (but not having been in and left) a specialist prescribing programme was associated with stable accommodation. Improvements in mental health were less among patients in GP-based prescribing programmes, and mental health tended to better among residential rehabilitation clients.

Crack users: hard to treat?

Of the two substances on which the English national drug treatment policy is focused, dependence on cocaine in the form of crack is traditionally considered harder to treat than heroin dependence, partly due to the lack of an effective medication, but also due to the instability inherent in compulsive use of the drug. However, while medications have yet to prove their worth, in respect of psychosocial treatments for stimulant dependence in general and crack in particular, it is less the case that 'nothing works', more the case that, within reason, everything works for some people to some degree and for some time.

DTORS seemed to support this understanding, though the analysis to date has focused on the undifferentiated 44% of treatment-seekers who had used crack in the four weeks before seeking treatment. While it seems all or nearly all 47% of the sample considered their crack use a problem. these considered their crack use a problem, just 1 in 8 (12%) of treatment-seekers considered it their primary problem. Had these 200 or so people been separated out in the analysis, they might have been shown to have different prospects to people whose crack use was subsidiary to use of (generally) heroin. Supporting this speculation is the fact that people who were using crack together with heroin were more likely to stop using crack than other crack users.

Across all crack users, by the first follow-up point only 15% recalled receiving a crack-specific intervention, and whether they had was unrelated to whether crack use ceased or continued. Despite this lack of targeted attention, crack users did as well as anyone else in terms of short or longer-term retention In fact, over the first 12 weeks a higher proportion were in treatment because slightly more had started treatment. in treatment, employment, increase in legitimate income, accommodation, and increase in the proportion of parents living with all their children. If anything, crack seemed easier to give up than heroin; 53% of crack users had stopped using by the first follow-up and 61% by the second, respectively 9% and 12% higher than the corresponding figures for heroin and heroin users. Crack users also reduced their spending on drugs to a greater extent than non-users, perhaps because they were spending more to begin with. There was however a suggestion of a bifurcation, with the minority of crack users who did not stop continuing to use at undiminished levels.

Were criminal justice clients different?

In general the answer to this question was, 'No, or only slightly'. Predictably treatment-seekers referred or coerced in to treatment via criminal justice routes had recently been more heavily involved Over the previous four weeks criminal justice referrals (a third of the total sample) were more likely to report shoplifting, house burglary or bag snatching and earned significantly more from their offending. in some forms of crime, a differential which might have been greater had their activities not been curbed 36% of criminal justice referrals had been in prison or some other form of custody compared to just 8% from other referral routes. by criminal justice supervision. However, once other factors had been taken in to account, at no point in the study was the level of acquisitive offending greater among criminal justice referrals. In other ways too they seemed remarkably similar to other treatment-seekers. As many (about 7 in 10) had previously been in treatment, and their current motivation and readiness for treatment was as high as the predominantly self-referred remainder of treatment-seekers. A third said that they would not have come to treatment without the pressure resulting from their legal involvement, but many more (over half) said they would have come anyway. In-depth interviews with a small sample of clients also suggested their motivation to do well in treatment was as great as the rest of the sample. Again not surprisingly, on average criminal justice referrals reported somewhat greater external pressure to enter treatment, and in fact slightly more did so, but after this point they were retained in treatment as well as other patients.

Such small differences as there were suggested that criminal justice routes were netting a slightly more problematic caseload, related possibly to the disruption of their lives by recent spells in custody or under other forms of criminal justice supervision. Compared to the remainder, 13% more were not stably housed, 8% fewer were in employment, education or training, they were on average less well educated, and far fewer (13% v. 28%) of the parents among them lived with all their dependent children. Apart from a slightly greater prominence of crack use, their drug use and drug problem profiles were similar to those of other treatment-seekers and at follow-up they had reduced their drug use to about the same degree. Reflecting criminal justice populations in general, the criminal justice referrals were slightly more likely to be from black ethnic minorities and to be male.

Thanks for their comments on these notes in draft to Michael Donmall of the National Drug Evidence Centre at the University of Manchester and others on the DTORS research team. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Comment on these background notes Return to main entry
L10 Web Stats Reporter 3.15 LevelTen Hit Counter - Free PHP Web Analytics Script
LevelTen dallas web development firm - website design, flash, graphics & marketing