EFFECTIVENESS BANK BULLETIN 27 April 2009
EFFECTIVENESS BANK BULLETIN 27 April 2009The entries below are our accounts of documents selected by Drug and Alcohol Findings as particularly relevant to improving outcomes from drug or alcohol interventions in the UK. Entries were drafted after consulting related research, study authors and other experts and are © Drug and Alcohol Findings. Permission is given to distribute these entries or incorporate passages in other documents as long as the source is acknowledged including the web address http://findings.org.uk. However, the original documents were not published by Findings; click on the Titles to obtain copies. Free reprints may also be available from the authors; if displayed, click Request reprint to send or adapt the pre-prepared e-mail message. Abstracts are intended to summarise the findings and views expressed in the study. Below are comments from Drug and Alcohol Findings. Links to source documents are in blue. Hover mouse over orange text for explanatory notes.
If you have not found what you want you could:
● Try a subject or free text search instead. Searches include bulletin entries and all other documents on this site.
● Try browsing other bulletins or back issues of the magazine.
● Try searching the libraries of
Alcohol Concern or
DrugScope (opens new window).
● Documents are regularly added. Use the e-mail update service to monitor additions.
● Return to the home page.
Click HERE and enter e-mail address to receive alerts of new bulletins
Scotland's auditors spotlight lack of system and clarity in drug policy ...
Delaware commissioners engineer innovative and engaging services ...
Persistent and practical efforts promote engagement in self-help groups ...
Remarkable prevention outcomes from primary school good behaviour strategy ...
Drug and alcohol services in Scotland.Audit Scotland.
Audit Scotland, 2009.
Scotland's national audit body assesses value for money from drug and alcohol services. It found systems poorly informed by the problems to be addressed and what works in addressing them, and in respect of drugs, unclear about what 'value' consists of.
Abstract Audit Scotland is a statutory body which provides services to the Auditor General for Scotland and the Accounts Commission. Together they seek to ensure that the Scottish Government and public sector bodies are held to account for the proper, efficient and effective use of public funds. The featured report helps fulfil this function in respect of Scotland's policies on drug and alcohol use. By gathering new information and collating this with existing data, The auditors analysed published information on services, reviewed national documents, and collected and analysed expenditure data from all NHS boards and councils. Police forces were unable to identify their costs, so instead these were estimated from information collected on the activities undertaken by all Scotland's forces. Focus groups were conducted with people who have problems with drugs and alcohol, directly affected families, local drug and alcohol partnership support staff, and voluntary and private sector service providers. The audit team also interviewed staff and reviewed documents from agencies commissioning or providing drug and alcohol services. the study aimed first to identify how much the public sector spends on services specifically 'labelled' as concerned with drugs and/or alcohol, then to assess whether the uses this money is put to are determined by evidence of need or of what works.
The study noted that the impact of drug and alcohol misuse in Scotland is widespread. Individuals, and society more widely, are affected in terms of health, child protection, crime, community safety, housing, employment and social exclusion. Related death rates are among the highest in Europe and have doubled in the past 15 years. Alcohol problems affect many more people and cause three times the deaths related to drug misuse. Drug and alcohol misuse are problems across the whole of Scotland but particularly affect deprived areas. In response the Scottish Government recently launched new strategies for drugs and alcohol: The road to recovery: a new approach for tackling Scotland's drug problem in May 2008 and Changing Scotland's relationship with alcohol: a framework for action in March 2009. Multi-agency partnership working is a core element in these strategies. In 2007/08, the public sector spent £173 million on drug and alcohol services in Scotland: £84 million on drug services, £30 million on alcohol services, and the remainder on joint services.
The auditors found that funding arrangements are complex. Projects can have a number of funding streams, each with different timescales and reporting criteria, creating difficulties for those planning and providing services. Across Scotland the range and accessibility of drug and alcohol services varies. The Scottish Government has not set out minimum standards in terms of range, choice and accessibility that service users and their families can expect to receive. Spending decisions are not always based on evidence of what works or on a full assessment of local need. Given the scale of drug and alcohol problems in Scotland and the range of agencies involved, clarity of roles and accountability is essential. It is important for the Scottish Government to set out the direction and the roles and responsibilities of partner agencies and how performance will be assessed.
Key recommendations for the Scottish Government were that it
should:
• set clear national minimum standards for drug and alcohol services including their range, quality and accessibility; receive assurance that these are implemented in line with set timescales; and ensure performance is regularly monitored and publicly reported;
• clarify accountability and governance arrangements for the delivery of drug and alcohol services in Scotland and set out clearly the responsibilities of all organisations and partnerships involved in planning or delivering these services.
Key recommendations for public sector bodies were that they should:
• ensure that all drug and alcohol services are based on an assessment of local need and regularly evaluated to ensure value for money. This information should then be used to inform decision-making in the local area;
• ensure that service specifications are in place for all drug and alcohol services and set out requirements relating to service activity and quality. Where services are contracted, this specification should be part of the formal contract;
• set clear criteria of effectiveness and expected outcomes for the different services that they provide and undertake regular audits to ensure services adhere to expected standards;
• use the Audit Scotland self-assessment checklist in the report to help improve the delivery and impact of drug and alcohol services through a joined-up, consistent approach. The checklist sets out some of the high-level practical issues around drug and alcohol services raised in the current report. NHS boards, councils, police forces, prisons and the voluntary and private sectors should use it to assess themselves against each statement as appropriate and assess the strength of all relevant partnership arrangements.
The report comes at a time of heightened concern over whether British drug policies are delivering value for money, much of which hinges on what 'value' consists of. In the mix are: reduction of medical harms; social benefits and in particular crime reduction; social cost savings; recovery of social and psychological functioning; abstinence from illegal drugs; abstinence from these and also from legal substitutes; more recently, reintegration with a focus on employment and associated welfare benefits savings; and various combinations and weightings of these objectives. As a result, the report is hamstrung, able to determine more completely than ever before the size and nature of the 'money' side of the equation, but unable to assess 'value': "There is no direction from the Scottish Government on what money for drug treatment and care services should deliver". A secondary problem is that however value is defined, there is inadequate data to assess the degree to which it has been achieved.
The situation is clearer in respect of alcohol policy, partly because abstinence (though an objective for many individual problem drinkers) is not on the radar as a social objective, leaving various forms of harm reduction (medical and social) as the clear primary objectives. In contrast to drug policy, the report is able to say, "There is direction from the Scottish Government on what additional money for alcohol services should deliver".
Nevertheless the light shone on the money side of the equation is welcome. For the first time national and local funding has been aggregated, yielding an estimate that the identifiable spend in 2007/08 on alcohol services in Scotland was about a third of that on drug services. Assuming joint services were equally split, the proportion would rise to just over half. In either case, there is a striking disparity between the relative spend and the relative size of the problems being addressed.
The Scottish Government's response to the report will also draw on an earlier investigation of alcohol and drug action teams – local partnerships responsible for coordinating the delivery of drug and alcohol services in their areas.
In England in 2008 the National Audit Office released its report evaluating work by the Department of Health and the National Health Service to address the health effects of alcohol misuse. Currently the same body is conducting a similar exercise in respect of illegal drugs, in line with a commitment in the English drug strategy to "conduct a study to evaluate the effectiveness and value-for-money of Drug Action Teams". This report will follow up the recommendations made in 2004 by the Audit Commission, which assesses the work of local public bodies.
Last revised 13 April 2009 Comment on this entry Give us your feedback on the site (one-minute survey) Back to contents list at top of page
Reducing alcohol harm: health services in England for alcohol misuse STUDY 2008
The primary prevention of hepatitis C among injecting drug users REVIEW 2009
Needle and syringe programmes: providing people who inject drugs with injecting equipment REVIEW 2009
The Drug Treatment Outcomes Research Study (DTORS): final outcomes report STUDY 2009
Giving the silent majority a voice IN PRACTICE 2004
Is your measure of success what matters to the client, or what matters to everyone else? OFFCUT 2003
Identifying cost-effective interventions to reduce the burden of harm associated with alcohol misuse in Australia REVIEW ABSTRACT 2008
The costs and consequences of three policy options for reducing heroin dependency STUDY 2007
The grand design: lessons from DATOS KEY STUDY 2002
Improving public addiction treatment through performance contracting: the Delaware experiment.McLellan A.T., Kemp J., Brooks A. et al. Request reprint
Health Policy: 2008, 87, p. 296–308.
Instead of telling addiction treatment providers what to do to qualify for funding, the US state of Delaware set recruitment and engagement targets and largely left the methods up to the services. Result: more and more engaging treatment without stifling innovation.
Abstract In the USA, until 2002 Delaware's Division of Substance Abuse and Mental Health negotiated traditional contracts with addiction service providers, reimbursing them for the costs of approved treatment activities. In 2002, all outpatient programmes were switched to contracts which instead rewarded the meeting of performance standards in terms of the recruitment and engagement of patients. Certain core services Such as the use of the Addiction Severity Index (ASI) for assessments, group, individual and family counselling, screening and referral for HIV, other infectious diseases and mental illnesses, and evidence of cultural competence in the delivery of these services. commonly accepted as necessary for adequate care of public sector addiction clients were specified in advance, and providers were asked to identify at least one evidence-based practice Most chose motivational interviewing and/or cognitive-behavioural therapy. and establish their ability to implement it. They were also required to take on any patients seeking treatment which met the criteria for their programmes. Beyond these basics, services were more or less free to meet the performance targets however they thought best. The most fundamental target was to recruit sufficient patients to run at 80% (later 90%) capacity. Payments to services were cut if they failed to meet these targets, and further incentive payments could only be earned once recruitment targets had been met. These further payments were made on the basis of the proportions of patients attending a set minimum of treatment sessions at different phases of a programme, and the number who satisfactorily completed Defined as active participation in treatment for a minimum of 60 days, achievement of major treatment goals, and submitting a minimum of four consecutive weekly urine samples free from illegal drugs and alcohol. the programme.
During a six-month run-in period, training was offered to providers, and provider-commissioner meetings established closer relationships and identified administrative barriers to meeting the targets. To meet these, all services learnt one or more of the selected evidence-based practices. Beyond this they adopted several methods All streamlined admission procedures, reducing the data collection burden and focusing early sessions on meeting patients’ needs and promoting engagement. All also extended their hours of operation and some opened satellite offices. Several altered their physical environments to make them more attractive and inviting. to make it easier for patients to access the service and to encourage their participation, creating more ‘user-friendly’ services which required less effort from, and were more attractive to, the patients. Some also rewarded staff for helping the service meet its targets.
One service failed the requirements and withdrew from the contract. The remainder increased their capacity and rapidly increased the proportion of treatment slots which were filled, resulting overall in an 87% increase between 2001 to 2006 in the average number of patients in treatment. If anything, services were extended to a more severely problematic caseload. Recruitment targets were met without resort to recycling the same patients through multiple episodes of care. There was also considerable progress in encouraging patients to attend treatment sessions
The researchers concluded that properly designed, programme-based contract incentives are feasible to apply, welcomed by programmes and may help set the financial conditions necessary to implement other evidence-based clinical efforts, furthering the overall goal of improving addiction treatment.
Readers working in English drug treatment services will be keenly aware of the relevance of these findings to current arrangements which make funding contingent on the numbers of patients who satisfactorily complete treatment or stay for at least 12 weeks.
The main interest of the study is that it featured a payment system which (beyond certain basics) did not mandate certain activities or quality standards, but instead left it to the services to decide how to meet recruitment and engagement targets. This strategy could be applied regardless of the particular targets to which payments are linked. The authors report that it was widely appreciated as respectful of the staff’s judgment, responsibility and clinical expertise. Instead of establishing uniform pre-set programmes, the result was to stimulate innovation and creativity. Services also had much to gain from sharing their ideas and experiences, so the system encouraged collaboration rather than competition. After the study ended this was extended through new contracts which rewarded services for arranging post-detoxification transfer to outpatient rehabilitation.
A key limitation is that the study was unable to test whether improved recruitment and engagement really did help resolve substance use problems more widely and more fully than would have been the case without those improvements. In general people in need of addiction treatment do better if they get it and if they participate more fully in that treatment, but the relationships are often loose. In particular, studies often find that treatment participation and retention With the notable exception of retention in substitute prescribing programmes. are unrelated or only poorly related to post-treatment substance use. Initiatives which improve engagement may have no noticeable effect on outcomes. The study was also unable to eliminate the possibility that other quality improvement initiatives were at least partly responsible for the observed recruitment and engagement trends. Neither is it clear to what extent data provided by services was verified. With funding contingent on hitting targets, some truth-stretching is a possibility. However, improvements such as in opening hours, refurbishment of premises, and opening new clinics, are concrete and hard to falsify. Services which were attempting to make themselves look better mainly by manipulating the figures would be unlikely to invest in such improvements. As the authors observed, from the perspective of a potential patient, the new system has more treatment venues, better proximity to the most needy populations, more convenient hours of operation, and refurbished facilities.
Despite the gains, the study also revealed the limitations of target-driven funding. The pattern of attendance improvements at different phases of treatment suggests that services tried to meet the targets and gain the incentive payments, but did not attempt further improvements. Improvements were most rapid in respect of recruitment targets which attracted the great bulk of funding. Together these trends suggest services responded to the financial incentives, putting in the greatest effort where the rewards were also greatest, but not necessarily seeking to excel in these or in other ways. If, as seems likely, improvements were driven by the funding, it begs the question of why services dedicated to the welfare of people with substance use problems had not already done was what needed to maximise participation in their programmes. A potential side effect of any such payment system is to reinforce a culture of doing what it takes to gain rewards for staff and service owners and managers, undermining the motivation to make life as good as possible for as many patients as possible. The post-2002 system in Delaware did much to align patient and service interests, but alignment will always be imperfect unless services place patients and potential patients at the core of their concerns.
Thanks for their comments on this entry in draft to Dr A Thomas McLellan of the Treatment Research Institute in Philadelphia, USA. Commentators bear no responsibility for the text including the interpretations and any remaining errors.
Last revised 15 April 2009 Comment on this entry Give us your feedback on the site (one-minute survey) Back to contents list at top of page
Continuing care research: what we have learned and where we are going REVIEW 2009
Relating counselor attributes to client engagement in England STUDY 2009
Matching resources to needs is key to achieving 'wrap-around' care objectives NUGGET 2006
Recovery management and recovery-oriented systems of care: scientific rationale and promising practices REVIEW 2008
Dual diagnosis add-on to mental health services improves outcomes NUGGET 2004
The grand design: lessons from DATOS KEY STUDY 2002
Brief interventions short-change some heavily dependent cannabis users NUGGET 2005
Psychosocial interventions for people with both severe mental illness and substance misuse REVIEW 2008
Aftercare calls suit less relapse-prone patients NUGGET 2005
A randomized controlled trial of intensive referral to 12-step self-help groups: one-year outcomes.Timko C., DeBenedetti A. Request reprint
Drug and Alcohol Dependence: 2007, 90, p. 270–279.
Even in a largely 12-step oriented programme, this US study showed that persistent and practical efforts can modestly strengthen 12-step group involvement after treatment and improve outcomes.
Abstract Some of the data in this entry derive from an earlier report on the same study.
This study implemented and evaluated procedures to help clinicians make effective referrals to 12-step mutual aid groups. In this randomised controlled trial, 345 individuals with substance use disorders At intake, substance of choice for 46% of patients was alcohol, 36% cocaine, and 22% cannabis. Other substances accounted for less than 10% each of the sample. 42% of patients were using more than one substance. starting a new non-residential treatment episode The intensive 28-day programme combined cognitive-behavioural and 12-step elements and explored the interpersonal consequences of substance misuse. Treatment was abstinence-based with patient activities (eg, therapy oriented toward relapse prevention, psychoeducation) scheduled each weekday. were randomly assigned to a standard referral or intensive referral to 12-step groups. Patients reported on their group attendance and involvement and on substance use at baseline and six-month and one-year follow-ups. Standard referral entailed patients being given a schedule for local 12-step meetings and being encouraged to attend. Intensive referral involved counsellors linking patients to 12-step volunteers and using 12-step journals to check on meeting attendance.
Compared with patients who received standard referral, patients who received intensive referral were more likely to attend and be involved with 12-step groups across the first and second six months of the follow-up period, and improved more on alcohol and drug use outcomes over the year. Specifically, across both follow-up periods, after intensive referral patients were more likely to attend at least one meeting per week (70% versus 61%), were more involved with the groups, experienced greater reductions in alcohol and drug problems, and were more likely to be abstinent from alcohol and other drugs (51% versus 41%) chart. Analysis suggested that intensive referral improved alcohol and drug use outcomes by strengthening involvement in 12-step groups and associated activities. Attendance at groups was associated with substance use outcomes, but even after attendance was accounted for, stronger involvement remained associated with better outcomes.
The authors concluded that intensive referral was associated with improved 12-step group attendance and involvement and substance use outcomes. To maximally benefit patients, they recommended that treatment providers should focus 12-step referral procedures on encouraging broad 12-step group involvement, such as reading 12-step literature, doing service at meetings, and developing an identity as a group member.
This study is not the first to have shown that counselling targeted at raising 12-step attendance and involvement can have the intended effects and thereby improve substance use outcomes, but it does seem the first major study to have shown that this can be the case, even when the surrounding programme itself promoted 12-step involvement. Other studies discussed below.
While this study was concerned with the intensity of the referral procedure, US studies have shown that interpersonal style is also relevant. In one previously analysed by Findings, alcohol dependent patients admitted for inpatient detoxification were randomly allocated to one of two types of continuing care advice. The first was five minutes of highly directive advice during which patients were told they had a significant drink problem and that abstinence was very important, and were then unambiguously advised to get as involved as possible in AA/NA aftercare. The second was a one-hour session which also advised abstinence and AA attendance, but in the tradition of motivational interviewing, avoided being explicitly directive and asked patients to choose their own aftercare preferences. Overall there was no difference in later drinking or heavy drinking. However, among patients less committed to 12-step groups, the motivational interviews led to better drinking outcomes, while the more committed did worse. In contrast, patients already set on attending AA did better with brief direct advice, worse after the motivational interview. A later study involved patients who had responded to ads to join a study of skills-based alcohol therapy, most of whom had no prior experience with AA. It showed that both intensity and style have an impact. The basic therapy included a few minutes during which patients were instructed to attend AA and those who did so were encouraged to continue. In the same manner but much more extensively, other patients were instructed to attend and over a third of therapy time was devoted to discussing and encouraging involvement in AA. This had the intended effect of intensifying post-treatment AA involvement and (thereby it seemed) somewhat increasing abstinence rates, though alcohol-related problems or heavy drinking days were unaffected. In contrast a motivational-style attempt to encourage AA involvement (the message was "ultimately it is up to you" and cons as well as pros were rehearsed) which was also relatively extensive had no more impact either on involvement or drinking than the few minutes incorporated in the basic therapy. It is important to remember that these studies were conducted in the US context, where 12-step groups are so much 'part of the scenery' that (as in the Project MATCH alcohol treatment trial) directive advice to attend them may not feel directive at all to the patients. In Britain the results might be quite different, though the principle that some styles of communicating will suit some people but not others, and in particular that people already committed to 12-step involvement will react best to direct and congruent advice, is likely to be universal.
In the featured study, few patients were excluded, relatively few refused to participate, and follow-up rates were high, suggesting that given the same interventions, the results would be applicable across the clinic’s entire caseload of US ex-military personnel. Nevertheless, for several reasons the results might not be replicated in routine practice. Intensive referral counsellors were trained and supervised by the research team based on audio-taped sessions, and continued to be monitored through tapes and through post-session counsellor and patient checklists. What this was compared to was intended to approximate standard referral procedures at such clinics, but was also closely controlled via a standardised script to avoid overlap with intensive referral. The intention (and the result) was to limit the discretion of the counsellors and to create a sharp divide between the two referral options. As a result, the study illustrates what happens when intensive referral is implemented to perhaps an unusually high degree of consistency, and comparator counsellors are fettered from exercising discretion depending on the needs and willingness of the patient to engage with post-treatment 12-step groups. The study shows that, in these circumstances, intensive referral does make a difference, but not necessarily that it would if comparison counsellors were allowed to exercise discretion, nor that it would if intensive referral were routinely implemented. Neither do the findings necessarily support across-the-board intensive referral. This was least effective for the quarter of patients who had previously attended 12-step groups the most, presumably because many would have continued to attend regardless.
Given these circumstances, what requires explanation is not just why intensive referral had the impacts it did, but why it did not have greater impacts. It involved (unless the patient had already started attending the groups) the counsellor calling a mutual aid group volunteer during each of three counselling sessions to arrange, then and there, for them to meet up before going to a meeting together. The comparison merely involved handing over a list of meetings and encouraging attendance. Yet in the first six months there were no statistically significant differences in attendance rates. However, during that period intensive referral patients were somewhat more deeply involved in 12-step activities. Greater involvement might explain why over the entire 12 months slightly more of these patients attended the groups, and why in the last six months of the follow up they attended on average slightly more meetings (46 v. 37). Modest as it was, this degree of enhanced involvement nevertheless resulted in somewhat higher abstinence rates and steeper reductions in substance misuse problems.
The context of the study may explain why neither involvement nor outcomes were greatly affected by intensive referral. Virtually all the patients were already familiar with 12-step groups and the core treatment programme was in any event infused with a 12-step orientation. Three sessions of intensive referral were perhaps merely the icing on the cake, most patients having already made up their minds whether to get involved in the groups based on their previous experiences and daily activities at the clinic. How far then the results might generalise to countries like the UK with less pervasive 12–step traditions is unclear. It could be that intensive referral would actually have a greater impact, a speculation supported by its being most effective with the less frequent prior attendees. It might however have to be applied in a more discriminating fashion to avoid alienating patients set against this form of continuing support. On the other hand, persistent advocacy of 12-step involvement might be both inappropriate and ineffective in treatment programmes which are not based on 12-step philosophy, including many UK programmes.
We know from the NTORS study in England that (mostly heroin-addicted) patients who regularly attend 12-step groups after residential/inpatient treatment are much more likely to sustain abstinence from alcohol and opiates. Whether this was the cause or the result of their abstinence, or both were a reflection of some other attribute, is unclear. In NTORS, over half the services actively encouraged 12-step group attendance after treatment, but unless patients had already attended such groups, they were very unlikely For example, just 8% in the last three months of the one-year follow-up period. to do so after treatment ended. In Britain, where experience with 12-step groups cannot be assumed, in-treatment 'tasters' of what the groups are like may be needed, as well as sensitively applied active referral.
Thanks for their comments on this entry in draft to Christine Timko of the Veterans Affairs Health Care System in California, USA. Commentators bear no responsibility for the text including the interpretations and any remaining errors.
Last revised 23 December 2009 Comment on this entry Give us your feedback on the site (one-minute survey) Back to contents list at top of page
The grand design: lessons from DATOS KEY STUDY 2002
Continuing care research: what we have learned and where we are going REVIEW 2009
Day hospital and residential addiction treatment: randomized and nonrandomized managed care clients STUDY 2007
Aftercare calls suit less relapse-prone patients NUGGET 2005
Peer-based addiction recovery support: history, theory, practice, and scientific evaluation REVIEW 2009
Are we right to spend more? IN PRACTICE 1999
Self help: don't leave it to the patients THEMATIC REVIEW 2005
The Drug Treatment Outcomes Research Study (DTORS): final outcomes report STUDY 2009
Self-financing resident-run houses maintain recovery after treatment NUGGET 2008
Effects of a universal classroom behavior management program in first and second grades on young adult behavioral, psychiatric, and social outcomes.Kellam S.G., Brown C.H., Poduska J.M. et al. Request reprint
Drug and Alcohol Dependence: 2008, 95(suppl. 1), p. S5–S28.
In their first years at school, Baltimore pupils formed teams which could earn prizes and praise for good behaviour; 14 years later many fewer young lives were marred by substance-related problems, threatened by smoking, or on track to cause serious social problems.
Abstract In the first long-term randomised trial, the Good Behavior Game was tested in first- and second-grade (ages 6–8) classes in primary schools run by the US city of Baltimore beginning in the 1985–86 school year. The featured report documents impacts on substance-related and other problems roughly 14 years later, when the children were aged 19–21. Click here for more on the research and for a link to the Good Behavior Game manual used in the study.
The game is not a lesson as such, but a way of managing whole classes during lessons. It aims to socialise children to the role of being a school pupil and to reduce aggression or disruptive behaviour, which are known to be related to later substance abuse and dependence and antisocial behaviour. Children are divided into teams which can win prizes depending on the good behaviour of the team as a whole. In the featured study, teams did not compete against each other; each could independently gain rewards. Class teachers used the research team's assessments of their pupils (themselves largely based on the teachers' ratings) to assign them to three teams with the same numbers of boys and girls, and of aggressive/disruptive or shy, socially isolated children. The good behaviour rules (such as not talking out of turn or leaving your seat without permission) teams had to adhere to win prizes were displayed to the class. During a game period, a mark was placed on the chalkboard next to the name of a team whenever one if its members broke a rule. Teams won if they chalked up no more than four marks by the end of the game period. At first teachers announced the start of game periods, which occurred at no set time but initially for 10 minutes three times a week. Praise plus tangible rewards such as such as colourful stickers or rubbers were awarded immediately after the game. Over the school year, game time increased to three hours a week at more varied and unpredictable times, periods became unannounced, and rewards delayed until the end of the day or week. Rewards changed from tangible objects to valued activities intrinsically related to the classroom setting, such as extra quiet time to read during the school day. In preparation, Good Behavior Game teachers Comparable time and attention were devoted to control class teachers but without a focus on classroom behaviour management. were trained for 40 hours, their teaching was monitored, and they received continued mentoring.
The most stringent test of the game involved comparing children who had been in eight Good Behavior Game classes with comparable children from six control classes at the same schools who had not experienced the game. Game classes were also compared with classes in similar schools where the game was not implemented. Additionally, it was implemented twice in succeeding intakes of first-grade children who were taught using it in the first two years of primary schooling. Details in background notes.
Three quarters of the first intake of children were interviewed Interviewers did not know whether interviewees had been in game or control classes. by telephone when aged 19–21, using a standard questionnaire which yields mental health diagnoses based on the US classification system known as DSM-IV. The general pattern was that among young men, and especially those who in their initial school years had been rated as the most aggressive or disruptive, exposure to the Good Behavior Game had substantially curbed the development of proscribed and anti-social behaviours. Substance abuse and dependence were among those most strongly affected. However, among children (including girls) not predisposed at an early age to developing these behaviours, the game made less or no difference. Neither did it affect the development of pathological anxiety or depression, relatively unrelated to the aggressive and disruptive tendencies the game was intended to manage. Effects were greater and more consistent in the first set of pupils whose teachers were freshly trained and subject to continued monitoring and mentoring, suggesting that these supports may be needed There are other possible explanations, including the fact in the second set of pupils, teachers' ratings of early maladjustment were less closely related to later young adult behaviour, meaning that interventions designed to address these early signs might also have been less closely related. to maximise implementation and benefits, and in particular to focus benefits on the pupils most in need. For detailed findings see background notes.
Based on the significant results from the first set of pupils, the authors derived estimates of the proportions of children who would as young adults have developed problems with versus without experience of the game as tested in the trial – that is, consistently applied over the first two years of primary schooling by freshly trained and continually supported teachers chart. Estimates were that the game would: halve the proportion of boys later meeting criteria for drug abuse or dependence (from 38% to 19%); across both genders, do nearly as well in cutting rates of alcohol abuse or dependence from 20% to 13%; reduce the prevalence of regular smoking among boys from 17% to 7%; across both genders, cut the proportion exhibiting serious and pervasive antisocial behaviour from 25% to 17%; and halve this rate (from 86% to 41%) among boys whose early classroom behaviour indicated they were most likely to develop this behaviour.
These are some of the most substantial effects ever recorded from a school-based prevention programme. Unusually, the study's design was able to test whether effects persisted through to young adulthood. There are some key points about both the intervention and the findings. First, unlike most other school programmes, the Good Behavior Game does not occupy precious curriculum time. It is a way of managing a class while teaching the school's usual curriculum. To the degree that it works, teachers can expect to be able to teach less disruptive classes more effectively.
Secondly, partly because there is no 'subject' content, the intervention intercedes at the level of how the pupil relates to the social world around them and vice versa. The result is a range of beneficial effects. These are most easily documented for the minority of youngsters most likely to develop unhealthy relationships, but the benefits should extend to their friends, families, neighbours and colleagues, and to the broader society which is relieved of responding to proscribed and/or antisocial behaviour. The strategy is consistent with the observation that typically children develop a constellation of mutually aggravating problems, related the further back one looks to a shared set of factors affecting children's mental and physical well-being. Among these is a positive school environment, found in other studies to be strongly related to substance use.
Thirdly, and again unusually, the study directly measured problem substance use outcomes, not use as such, so sidestepped the criticism that experimentation with substances is relatively normal behaviour sometimes falsely dubbed a problem. However, this raises its own problems, primarily to do with the questionable applicability For example, alcohol abuse could be diagnosed solely on the basis of recurrent legal problems. In the USA, at the ages many former pupils were interviewed, any drinking would have been illegal, raising the possibility that exactly the same drinking pattern might in a year or two no longer be diagnosed as abuse. Some of the criteria for alcohol dependence also have limited applicability to youngsters who may not (for example) see the need to limit their drinking, so might truthfully deny drinking more or for longer than they intended, and also may have varied understandings of questions about needing to drink more to get the same effect. Dependence diagnoses presuppose the significant health or psychological problems which lead the user to repeatedly wish to and/or try to cut back, problems which may not yet be evident to someone aged 19–21. Similar issues arise in respect of drug-related diagnoses. For both drug and alcohol abuse, the same use pattern which in less troublesome youngsters might not attract a diagnosis, might well do so among young people characterised by disregard for the law and reckless, attention grabbing and aggressive behaviour. to young people of diagnostic criteria developed largely from experience with adults in clinical settings, issues which have been raised in respect of both alcohol and drug diagnoses. It is, for example, possible that the game led to fewer children meeting abuse criteria because it curbed antisocial tendencies, not because it did anything to curb substance use as such. Also the diagnoses This was not the case for regular smoking. were to do with ever having met the relevant criteria. By definition this is not necessarily indicative of a continuing problem, particularly as major life changes after the end of compulsory schooling (such as starting to drive, getting a job, starting a family, leaving home) could lead to transition in to and out of diagnostic categories, regardless of any changes in substance use.
However, the Good Behavior Game also led to 'real world' impacts of the kind which would be expected from the outcomes in the featured report, lending substance to its diagnoses. Another report from the same study found According to the young adults' accounts in response to the question, "Have you received services from [various service providing sectors] for problems with behaviours, feelings, drugs or alcohol?" that in both the first and second sets of pupils, 19–21-year-old young men previously exposed to the game said they had significantly less often had contact with services intended to deal with behavioural, emotional, or substance use problems. Again the greatest impact was among those most prone in their early years to developing these problems. Of most interest in the current context were reductions in drug treatment interventions. Without the game, around a tenth of the former pupils recalled this kind of intervention; with the game, this figure was more than halved, though the results fell short of statistical significance. Among the first set But not the second. of pupils the game was also associated with halved risks of thinking about or attempting suicide. Finally, stepping beyond the youths' own accounts, reductions were found among high-risk youth in official records of violent or criminal behaviour.
Among other studies of the game, the most relevant is a recent trial in the Netherlands. This followed up primary school pupils in classes randomly allocated the game to age 13, possibly too young to expect substantial impacts on substance use. Nevertheless it did find a significant reduction Statistically significant across the entire sample but not when parental smoking was controlled for due it seems to reduced sample size because some children did not supply this information. in smoking. There were also hints of a reduction in drinking, which reached statistical significance only in respect of past-week drinking, the most serious level assessed in the study. See background notes for further considerations relevant to whether results would be similar in other schools and cultures, risks of harnessing pupil peer pressure, and how the game works to prevent disrupted classes propelling aggressive and disruptive boys in to seriously antisocial and problematic teenagers.
The Good Behavior Game is not the only early school intervention to have shown promising effects across a range of behaviours, nor is it a complete solution to adjustment problems and developmental inequality. The team behind the Baltimore studies have themselves combined it with interventions to develop fundamental intellectual competencies such as maths and critical thinking. As measured up to about age 13–14, these curbed the incidence of smoking and use of drugs like heroin and cocaine Due to small numbers using these substances, despite statistically significant findings the authors caution against reading too much in to these findings. but not cannabis use or drinking.
The principles embodied in the game (such as harnessing positive peer pressure, everyone can win, spotting and rewarding good behaviour, setting achievable objectives, mechanisms for internalising this reward structure, children setting their own rules, counteracting counterproductive cliques) do however seem a valuable element and can be widely implemented. The game itself has been found feasible and effective in terms of classroom behaviour control in British schools. In broader form, these principles are embodied in the strand of personal, social, health and economic education (PSHE) which aims to create a climate in the classroom within which sensitive issues (including disruptive and aggressive behaviour) can be explored openly and honestly without fear of ridicule or betrayal of confidence, based on standards which the children themselves have helped generate. They can also be found in the SEAL (social and emotional aspects of learning) curriculum widely used in British primary schools.
Where these UK initiatives differ from the Good Behavior Game is in their rejection of approaches based purely on a mechanical system of rules, rewards and sanctions, seen as failing to encourage pupils to learn social and emotional skills or take responsibility for their own behaviour. Though not a substitute, the game might have a place within such an approach. But in schools which have successfully created this healthy climate, it may be superfluous. In Baltimore, in respect of preventing later aggression, the game was most influential in grade one classes characterised by disruption and aggression, where existing classroom management was presumably less adequate. When teachers and/or pupils had created a more congenial early years climate, the game was less influential and possibly ineffective.
Where the game scores, at least in its initial Baltimore application, is in lending itself to consistent application aided by its being very concrete and easy to codify in a step by step manual. Consistently maintaining a climate may be more important, but is less easy to do without wholesale and sustained change across all levels of authority in the school.
Thanks for their comments on this entry in draft to Adrian King of the InForm consultancy. Commentators bear no responsibility for the text including the interpretations and any remaining errors.
Last revised 28 September 2009 Background notes Comment on this entry Give us your feedback on the site (one-minute survey) Back to contents list at top of page
Early teaching boost pays off six years later NUGGET 2004
Confident kids ... like to party NASTY SURPRISES 2004
It's magic: prevent substance use problems without mentioning drugs HOT TOPIC 2010
Substance-focused initiatives not only way schools help prevent risky substance use NUGGET 2008
Prevention is a two-way process KEY STUDY 2001
Secondary school DARE ineffective without interactive extensions NUGGET 2003
