Drug and Alcohol Findings home page. Opens new window Effectiveness Bank bulletin 20 February 2013

The entries below are our accounts of documents collected by Drug and Alcohol Findings as relevant to improving outcomes from drug or alcohol interventions in the UK. The original documents were not published by Findings; click on the Titles to obtain copies. Free reprints may also be available from the authors. If displayed, click prepared e-mail to adapt the pre-prepared e-mail message or compose your own message. The Summary is intended to convey the findings and views expressed in the document. Below may be a commentary from Drug and Alcohol Findings.


Contents

Alcohol screening and brief intervention at the US health service for ex-military personnel

All the entries concern alcohol screening and brief intervention and derive from the US 'VA' health service for ex-military personnel. It trialled a computerised prompt to remind clinicians to consider advice or referral when a patient has screened positive for risky drinking. Where responding to such prompts was expected, they had the desired impact (entry 1) but not at other VA services (entry 2); leadership commitment seemed the essential ingredient. The third entry raised serious questions about the screening which determined whether prompts would appear. Finally, for convenience we repeat two earlier entries, one thoughtfully (entry 4) analysing how to measure the performance of these systems and the last (entry 5) a review from the same VA research team of how to implement primary care alcohol screening and brief intervention.

Electronic prompts mean more primary care patients get advised about drinking ...

Leadership commitment needed to implement electronic alcohol advice prompts ...

Most risky drinkers missed in primary care because patients underreport drinking ...

How to assess progress in implementing screening and brief alcohol interventions ...

What it takes to implement alcohol screening and brief intervention ...


Use of an electronic clinical reminder for brief alcohol counseling is associated with resolution of unhealthy alcohol use at follow-up screening.

Williams E.C., Lapham G., Achtmeyer C.E. et al.
Journal of General Internal Medicine: 2010, 25(suppl. 1), p. 11–17.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Williams at emily.williams3@va.gov. You could also try this alternative source.

When a patient has screened positive for risky drinking, up pops a computerised prompt to remind the clinician to consider counselling. In one service for US ex-military personnel, this resulted in nearly three quarters of patients being counselled and a hint of consequentially reduced drinking; at another, findings were negative. Why the difference?

Summary No health-care system has successfully implemented sustained, routine brief alcohol counselling. Using electronic medical records to remind clinicians to consider or undertake this work can increase provision of recommended preventive care, and providers at practices with clinical decision support systems are more likely to counsel patients with unhealthy alcohol use than those at practices without. However, no study has tested an electronic clinical reminder as a method of implementing brief alcohol counselling in the absence of other systematic supports or incentives.

The US 'VA' health care service for ex-military personnel offers an important opportunity to test such a system. Constituent services commonly deploy clinical reminders in electronic records in conjunction with national performance measures linked to financial incentives. The service nationally implemented annual alcohol screening in 2003, resulting in over 90% of all outpatients being screened for unhealthy drinking. The next step is to maximise the numbers then offered advice.

With this aim in mind, an electronic clinical reminder was developed to prompt clinicians at a VA service with eight associated sites to offer brief alcohol counselling when patients were recorded as having screened positive for unhealthy drinking on the AUDIT-C screening questionnaire. It was implemented without any other provider training or support. As well as prompting the service provider, the system offered them information about what constitutes evidence-based brief alcohol counselling, supported assessment of alcohol use severity, provided an aid to deciding whether to implement brief counselling or referral, and documented these actions in the patient's record. Though prompted automatically, providers could choose whether to open and act on these and any other prompts in respect of other conditions.

The study aimed to determine the proportion of positive-screen patients whose care providers did open and use Defined by entries in their medical record indicating that the provider had used the reminder between initial and follow-up AUDIT-C assessments and advised the patient to reduce drinking below recommended limits or to abstain, referred them to addictions or mental health services, and/or had such a referral refused. the reminder over a two-year period, and whether this use was associated with resolution (defined as screening negative plus at least a two point out of 12 reduction in score) of unhealthy alcohol use at follow-up alcohol screening, relying only on VA records. Findings were adjusted for differences between patients of the kind found by research to be associated with receipt of brief alcohol counselling and changes in drinking. Among these were multiple indicators of history and severity of unhealthy drinking.

Of 36,191 patients screened, 8759 screened positive. Of these, 4206 were re-screened from one month to three years later (averaging 14 to 15 months) and 4198 had a documented visit to the service during the study period; data from these 4198 patients was included in the study.

Main findings

Use of the alcohol counselling reminder was documented for 71% of positive-screen patients, most often for men, those who were single, relatively severely disabled due to military service, or had mental health diagnoses. No other measures of alcohol severity, other substance use, or physical comorbidity were associated with reminder use.

At follow-up screening, 31% of patients had resolved their unhealthy drinking; younger, female, non-white, single, service-disabled patients and those with more severe unhealthy alcohol use or other substance use and mental health diagnoses were more likely to have done so.

Before the figures had been adjusted for differences between patients, resolution of unhealthy alcohol use was found to have been significantly more likely (32% versus 28%) among those whose providers had used the reminder system, and these patients had also made significantly greater reductions in their drinking (averaging 1.65 versus 1.28 points). The resolution proportions were virtually unchanged (31% versus 28%) and remained significant after adjusting for patient differences.

Without the reminder, higher proportions of patients documented as having more severe Diagnosed with tobacco, alcohol, or drug use disorders or with medical illnesses due to excessive drinking. substance use problems resolved their drinking than did patients without such problems. This was not the case when providers had used the reminder.

The authors' conclusions

Clinical reminders succeeded in moving brief alcohol counselling up the busy clinical agenda for patients irrespective of the severity of their drinking, evidence that in routine practice such systems can extend brief alcohol counselling to more patients. Providers chose to use the reminders for nearly three quarters of patients who screened positive for unhealthy drinking, many more than the 28% of VA outpatients nationally with unhealthy drinking who said they had been advised about their drinking. This high performance was sustained for two years without any further support or incentives. Moreover, such patients were significantly if modestly more likely to report having resolved unhealthy alcohol use at follow-up than patients whose providers had not used the reminders. The fact that drinking severity was unrelated to documented advice/referral, and that reminder use 'evened out' differences in resolution rates associated with substance use severity, suggests that the reminder might counteract the inclination of providers to primarily counsel patients with the worst drinking problems.

It is unknown whether these findings would be replicated at VA or non-VA sites where clinical reminder use is not routine. Also, there is no consensus on what makes counselling effective, and it is unclear whether providers are prepared to offer effective counselling in the absence of education and coaching, even when prompted. The finding that use of the clinical reminder was associated with modest increases in resolution of unhealthy alcohol use may mean some providers have the necessary skills to offer effective brief alcohol counselling, or that the content of the counselling is less important than the fact that a provider raised the issue of drinking. Additional research is needed to evaluate the quality of counselling offered when reminders are used to prompt providers to counsel patients in real world settings, and to determine educational needs of providers and efficient approaches to meeting them.

Strengths of the study include no opportunity for bias due to selection of participants, and routine implementation with no special support and across several sites. However, the findings are vulnerable to patients who have been counselled being less willing to admit to continued drinking problems, to remaining differences between counselled and non-counselled patients which could not be adjusted for, and to differences between providers who did or did not tend to use the reminders. It could also be that patients were counselled but this was not documented in the reminder system.


Findings logo commentary With screening effectively incentivised at national level, the reminder system helped ensure that most patients who screened positive were documented as receiving some advice. In the context of other studies, it is a convincing demonstration that such reminders can set the stage (but not always below) for raising counselling rates to high levels. Much less convincing, however, is the conclusion that the result was to reduce drinking problems. As the authors acknowledged, and especially without comparison sites where the reminder system was not implemented, the small difference in the resolution of problems as defined by the study in patients who were or were not (according to records) counselled is indicative of at best a very modest impact, and possibly none at all given the limitations of the study. This in turn may be tied to the inability to assess or influence the quality of the counselling and even whether it actually happened.

The authors' caution that their findings may not be replicated at sites where clinical reminder use is not routine seems to have been borne out by another study of the same system in part of a VA clinic, which was able to compare results with another part of the same clinic in which the system was not implemented. No significant differences were found in the improvements made by risky drinking patients attending the two parts of the clinic, perhaps partly because (according to records) clinicians who were prompted by the reminders used the system for just 15% of patients, and then rarely to offer a brief intervention (just 6% of patients). The authors speculate that the difference was due to clinicians in the featured study being "expected to use clinical reminders," whilst presumably expectations were lower at the VA clinic where implementation was poor. This seems to decisively indicate that while the prompts provide a tool to improve performance, whether this tool is used depends on the culture and management of the organisation in which it is implemented, in line with emphases in reviews (1 2) of the implementation of screening and brief intervention.

Across an entire region, in the VA system drinking outcomes after brief intervention have also been disappointing. Using VA records, it was found that patients who screened positive for risky drinking and were re-screened around a year later were no more likely to have stopped risky drinking if their records indicated that had participated in a brief intervention than if they did not. The remission proportions were virtually identical – adjusted for other factors, 47% with advice, 48% without. Another finding was that just 28% of these repeat-screen patients had been advised about their drinking, and they tended to be the higher risk drinkers, despite the introduction of a national performance measure incentivising brief intervention aided by an electronic clinical reminder to positive-screen patients available to all VA facilities. Results from this early phase of the new national system offered no encouragement to its continuation, though results may change as the system beds in and is developed.

While the featured study was mainly about the counselling which should follow screening, another study has questioned the validity of screening results in the national VA system, finding that 61% of patients who screened positive when sent a postal survey did not do so when the same questions were asked as part of their routine care at their VA clinics.

Apart from the limitations noted by the authors, because of the requirement for a follow-up screen the study included only about half the patients who initially screened positive. Whether the other half were also counselled at the same high rate is not known. It means that potentially the counselling rate among all positive screen patients was as low as 34%. 2975 counselled of 8759 who screened positive. However, the report which documented the first eight months of the scheme also found high counselling rates (after bedding in, about two thirds of positive screen patients were recorded as having been counselled), and no mention is made of these results being limited to patients who were re-screened some time later.

Closely related studies and reviews

Also in the Effectiveness Bank is a review of performance measurement options for VA alcohol screening and brief intervention systems. This includes results from the first eight months of the system evaluated in the featured report. Further and less encouraging studies have evaluated the same system in another clinic and across a whole Veterans Affairs region. Another study has focused on the screening element at VA services nationally. Also available is an overview of issues and findings in respect of implementation of similar systems in the VA network nationally. In the Effectiveness Bank too are a review by the same research team and another conducted for Britain's National Institute for Health and Clinical Excellence of what impedes or promotes the implementation of brief alcohol interventions. The latter analysis includes extended commentary on the UK situation, partially replicated in a 'hot topic' entry discussing whether brief alcohol interventions really can deliver population-wide health gains.

Last revised 19 October 2015. First uploaded 01 February 2013

Comment/query to editor
Back to contents list at top of page
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates


Top 10 most closely related documents on this site. For more try a subject or free text search

STUDY 2010 Evaluation of an electronic clinical reminder to facilitate brief alcohol-counseling interventions in primary care

STUDY 2011 Quality concerns with routine alcohol screening in VA clinical settings

STUDY 2013 Screening and brief intervention for alcohol and other drug use in primary care: associations between organizational climate and practice

REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking

HOT TOPIC 2015 Can brief alcohol interventions improve health population-wide?

REVIEW 2011 Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review

REVIEW 2011 Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review

STUDY 2011 An evaluation to assess the implementation of NHS delivered alcohol brief interventions: final report

REVIEW 2011 Barriers and facilitators to implementing screening and brief intervention for alcohol misuse: a systematic review of qualitative evidence

STUDY 2013 Modelling the cost-effectiveness of alcohol screening and brief interventions in primary care in England





Evaluation of an electronic clinical reminder to facilitate brief alcohol-counseling interventions in primary care.

Williams E.C., Achtmeyer C.E., Kivlahan D.R. et al.
Journal of Studies on Alcohol and Drugs: 2010, 71(5), p. 720–725.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Williams at emily.williams3@va.gov. You could also try this alternative source.

When a patient has screened positive for risky drinking, up pops a computerised prompt to remind the clinician to consider counselling, yet at a service for US ex-military personnel the reminder was rarely used and made no difference to patients' drinking. Why were results so different from those at other clinics?

Summary Using electronic medical records to remind clinicians to consider or undertake preventive work can increase provision of recommended preventive care, including brief alcohol counselling to patients who screen positive for risky drinking.

The US 'VA' health care service for ex-military personnel commonly deploys clinical reminders in electronic records in conjunction with national performance measures linked to financial incentives. The service nationally implemented annual alcohol screening in 2003, resulting in over 90% of all outpatients being screened for unhealthy drinking. It remains however to maximise the numbers then offered advice.

With this aim in mind, an electronic clinical reminder was developed to prompt clinicians at VA services to offer brief alcohol counselling when patients were recorded as having screened positive for unhealthy drinking on the AUDIT-C screening questionnaire. As well as prompting the service provider, the system offered them information about what constitutes evidence-based brief alcohol counselling, supported assessment of alcohol use severity, provided an aid to deciding whether to implement brief counselling or referral, and documented these actions in the patient's record. Though prompted automatically, providers could choose whether to open and act on these and any other prompts in respect of other conditions.

Previously this system had been found to be associated with high counselling rates in a VA network and some indication of drinking reductions, but it was unclear whether similar results would be found where providers do not routinely use reminders.

This was addressed by the featured study over a three-year period at a single VA primary care clinic where reminder use was neither routine nor expected or incentivised. The reminder system was implemented in one randomly selected part of the clinic but not in the other part, offering a way to compare performance with and without the system in the same clinic. Implementation consisted solely of setting up the system and one e-mail to tell providers about this in the selected part of the clinic.

At issue was the degree to which the reminder system was used after clinicians had been alerted that their patient had screened as a risky drinker During the study period, the clinic had not implemented incentives to conduct alcohol screening, and the proportion of patients who were screened is unknown. on the AUDIT-C questionnaire, and whether use rates varied depending on the severity Men who scored 4–7 and women 3–7 were considered to screen positive for mild/moderate unhealthy alcohol use, and patients with scores of 8–12 were considered to screen positive for severe unhealthy alcohol use. of the patient's drinking. Use of the reminder was defined as an entry in the patient's medical record indicating that the clinician had used it to help assess, advise, or refer the patient to specialty care, or an entry indicating that drinking had not been addressed.

Also at issue was whether implementing the reminder system led to more patients overcoming their unhealthy alcohol use (defined as now screening negative plus at least a two point out of 12 reduction in score) at a follow-up screening conducted at least 18 months after the initial screen, relying only on VA records. Findings were adjusted for differences between patients, including those of the kind found by research to be associated with receipt of brief alcohol counselling and changes in drinking. Among these were multiple indicators of history and severity of unhealthy drinking.

Main findings

Essentially, no significant differences were found in the improvements made by risky drinking patients attending the two parts of the clinic, perhaps partly because clinicians who were prompted by the reminders used the system for just 15% of patients; further details below.

The first issue was whether the reminders were used. Of 22,863 patients who visited both parts of the clinic, 18% screened as drinking in an unhealthy manner, including 2640 patients assigned to the part of the clinic using the reminder system. Of these 2640, for just 398 (15%) was there any documented use of the clinical reminder. Use was more likely for patients with severe (20%) versus mild/moderate (14%) unhealthy drinking. Just 6% of positive screen patients were offered brief counselling, again more commonly (17% v. 4%) when the alert to the clinician had indicated a severe problem. Of the brief counselling interventions, a quarter consisted of advice to abstain and 15% to drink less. Most (59%) severe drinkers were advised to abstain, but about the same proportions of less severe drinkers (14% and 13% respectively) were advised to abstain or drink less. Nurse practitioners were the largest group of clinicians and also the ones most likely (in 23% of cases) to use the reminder system.

The remaining issue was whether having the reminder system made any difference to the patients' drinking. This could be assessed for 1358 of the 4202 positive-screen patients who were re-screened 18 months or more later. Of these, 40% where the reminder system had been implemented had resolved their unhealthy drinking compared to 43% where it had not, statistically not a significant difference. Given greater use of the reminders by the nurses, the researchers tested whether at least among their patients the system had made a difference; it had not. They also tested whether the minority of patients for whom the reminders had actually been used, or those then offered a brief intervention, had more often resolved their drinking problems than other patients. In neither case was there any substantial or statistically significant difference in resolution rates.

The authors' conclusions

With no active implementation efforts, little encouragement by local leaders for providers to use clinical reminders, and no incentives for their use or for brief alcohol interventions, few providers used a clinical reminder designed to facilitate brief interventions with patients who screened positive for unhealthy drinking. Providers who did use the reminder were mostly nurse practitioners; advice to abstain was the care most frequently documented. Offering providers access to the reminder did not lead to more patients who initially screened positive later resolving their unhealthy drinking.

Documented rates of use were higher in patients with severely unhealthy drinking, who were generally advised to abstain, probably reflecting biases of providers toward traditional case-finding approaches involving referring patients with alcohol-use disorders.

These findings suggest that a relatively passive clinical reminder alone is insufficient to get brief interventions onto the agenda of busy primary care providers. In contrast, at another VA centre where providers were expected to use clinical reminders, they did succeed in moving brief alcohol counselling up the agenda irrespective of the severity of the patient's drinking, and patients whose providers had used the reminders were (modestly) more likely to report having resolved unhealthy alcohol use at follow-up than other patients.

Studies have found that use of clinical reminders seems to depend on local clinical culture, and that clinicians adopt these more readily when aligned with performance measures and supported by leadership. Although there was a national performance measure for annual alcohol screening at the time of this study, there was none for brief intervention, and reminder use by providers was not routine at the featured site.

Together these findings suggest that clinical reminders might help implement brief interventions when accompanied by expectations that they will be used or incentives to promote use. As health care systems seek to integrate brief interventions into routine care, it will be important to consider other components of effective implementation, including incentives or clear expectations for providers to use decision-support systems, as well as to identify and address barriers to effective use of clinical reminders.

Previous trials of the effectiveness of brief interventions and/or methods of implementation have included select groups of patients and providers who consented to participate in research and were potentially more motivated to address drinking than typical patients and providers. These trials also focused specifically on alcohol-related counselling, and may have created unrealistic clinical situations which ignored other needs of primary care patients. Instead the results of the featured study derived solely from routine clinical and quality assurance procedures, evaluating a method of implementing brief interventions in a real-world clinical setting not limited by selection bias. Despite these strengths, the findings are vulnerable to remaining differences between counselled and non-counselled patients which could not be adjusted for, and to differences between providers. It could also be that patients were counselled but this was not documented in the reminder system.


Findings logo commentary There was indeed, as the authors point out, a startling difference between the 15% use rate of the reminder in the featured study and the 71% rate recorded in an recorded at different VA clinics. As well as the organisational factors mentioned by the authors, in that earlier study the reminders were implemented across entire clinics, not just in one part – a situation which might make it difficult for leaders to insist the reminders should be used.

In the context of other studies, the earlier study provided a convincing demonstration that such reminders can set the stage (but as the featured study shows, not always) for raising counselling rates to high levels. But it was much less convincing about any beneficial impact on drinking. Without comparison sites where the reminder system was not implemented, the small difference in the resolution of problems in patients who were or were not (according to records) counselled was indicative of at best a very modest impact, and possibly none at all given the limitations of the study. This in turn may have been linked to the inability to assess or influence the quality of the counselling and even whether it actually happened.

The conclusion that whether a tool like the reminder system is used depends on the culture and management of the organisation is in line with emphases in reviews (1 2) of the implementation of screening and brief intervention.

While the featured study was mainly about the counselling which should follow screening, another study has questioned the validity of screening results in the national VA system, finding that 61% of patients who screened positive when sent a postal survey did not do so when the same questions were asked as part of their routine care at their VA clinics.

Closely related studies and reviews

Also in the Effectiveness Bank is a review of performance measurement options for VA alcohol screening and brief intervention systems. This includes initial results from a more successful implementation of the system implemented in the featured study, later more fully evaluated in a report which found high counselling rates and some indication of drinking reductions. Another report has focused on the screening element at VA services nationally. Also available is an overview of issues and findings in respect of implementation of similar systems in the VA network nationally. In the Effectiveness Bank too are a review by the same research team and another conducted for Britain's National Institute for Health and Clinical Excellence of what impedes or promotes the implementation of brief alcohol interventions. The latter analysis includes extended commentary on the UK situation, partially replicated in a 'hot topic' entry discussing whether brief alcohol interventions really can deliver population-wide health gains.

Last revised 02 February 2013. First uploaded 02 February 2013

Comment/query to editor
Back to contents list at top of page
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates


Top 10 most closely related documents on this site. For more try a subject or free text search

STUDY 2010 Use of an electronic clinical reminder for brief alcohol counseling is associated with resolution of unhealthy alcohol use at follow-up screening

HOT TOPIC 2015 Can brief alcohol interventions improve health population-wide?

REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking

STUDY 2011 An evaluation to assess the implementation of NHS delivered alcohol brief interventions: final report

STUDY 2011 Quality concerns with routine alcohol screening in VA clinical settings

STUDY 2008 Universal screening for alcohol problems in primary care fails in Denmark and no longer on UK agenda

STUDY 2013 Screening and brief intervention for alcohol and other drug use in primary care: associations between organizational climate and practice

STUDY 2012 Alcohol screening and brief intervention in primary health care

STUDY 2012 Alcohol screening and brief intervention in emergency departments

STUDY 2013 Effectiveness of screening and brief alcohol intervention in primary care (SIPS trial): pragmatic cluster randomised controlled trial





Quality concerns with routine alcohol screening in VA clinical settings.

Bradley K.A., Lapham G.T., Hawkins E.J. et al.
Journal of General Internal Medicine: 2011, 26(3), p. 299–306.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Bradley at Katharine.Bradley@va.gov. You could also try this alternative source.

In the US health care service for ex-military personnel, 61% of patients who screened positive when sent a postal survey did not do so when the same questions were asked by their clinics, casting doubt on the validity of the test in routine practice in a service where the emphasis was more on the quantity than the quality of screening.

Summary The US 'VA' health care service for ex-military personnel implemented routine screening for risky drinking in 2004, and since 2006 has required that the Alcohol Use Disorders Identification Test-Consumption Questions In the UK the questions are:
How often do you have a drink containing alcohol?
How many units of alcohol do you drink on a typical day when you are drinking?
How often have you had [6 or more units if female; 8 or more if male] on a single occasion in the last year?
(AUDIT-C) be used for screening. How this is done is left to individual facilities or networks (eg, at triage, by primary care providers, paper questionnaires, etc). Over 90% of VA outpatients nationwide are screened with the AUDIT-C.

AUDIT-C has been validated when administered by interviewers and when completed on mailed questionnaires and the results shared with primary care providers, but little is known about its performance when implemented in routine clinical care. The featured study aimed to evaluate the quality of screening in the VA from 2006–2008 by taking advantage of the overlap between data from randomly sampled outpatients including their AUDIT-C screening results from their clinics, and that provided by patients who after an outpatient visit responded (55% did) to a confidential patient satisfaction survey which included the same questions.

Altogether 6861 patients (17% women) had AUDIT-C scores from both sources not more than 90 days apart, making it possible to see if they corresponded or were 'discordant', meaning that one screen indicated alcohol 'misuse' Defined for the study as scores of five or more out of 12, the threshold for offering counselling in the VA system from October 2007. but the other did not. Incidentally, the study was also able (among other things) to assess whether a new version of the computerised prompt which most services used to promote screening made any difference to its accuracy; from January 2008, this prompted providers to ask the screening questions exactly as written, non-judgmentally, and in a private setting.

Main findings

Key finding was that 61% of patients who screened positive when sent a postal survey did not do so when the same questions were asked as part of their routine care at their VA clinics, part of a pattern of results which indicated that patients were more likely to acknowledge heavy drinking when not in front of their doctors or nurses and when assured of confidentiality.

Discordance rates among patients screening positive

Though just 8% of patients had discordant scores in the two AUDIT-C sources, this was because the majority (87%) scored below the alcohol misuse threshold in both. Altogether 765 patients gave answers indicative of alcohol misuse in the confidential survey but just 390 when questioned at their VA clinics. Of the 765 confidential-screen alcohol misusers, 468 gave sufficiently different answers at their clinics to score below the misuse threshold. Assuming their confidential scores were accurate, this means clinic screening would have failed to identify 61% as possibly in need of counselling. In contrast, over three quarters (297 of 390) of the patients who answered as misusers at their clinics also did so on the confidential surveys charts.

Another indication that the survey recorded more drinking than was acknowledged at the clinics was that over a fifth (22%) of the patients who told their clinics that they had not drunk at all in the past year said they had drunk in the confidential survey. Again, at 9% the reverse (survey non-drinkers telling their clinics that they had drunk) was much less common.

The pattern of discordance from the clinic screen results looked like the result of random or normal fluctuations, but discordance from survey results was much more common among the heavier drinkers, taking the form of their not scoring as misusers at their clinics. Even among the heaviest drinkers according to their survey answers, 47% did not score as misusers at all at their clinics, meaning that each must have given answers there which resulted in AUDIT-C scores at least four and as many as eight (out of 12) points lower.

In some local VA networks all the patients who scored as misusers in the confidential survey did not do so at their clinics, ranging down to 43%. The revised prompt from January 2008 for clinicians to ask screening questions verbatim, non-judgmentally, and in private did not result in any statistically significant improvements in concordance with survey results. When all the factors available to the study had been taken in to account, clinic results differing from survey results was significantly more likely Not influential were age, gender, socioeconomic variables mental health diagnoses. among patients who scored as misusers on the survey, were black, or were patients at some local VA networks rather than others. Not significant was whether the survey had come before or after clinic screening, suggesting that 'natural' resolution of heavy drinking could not account for the findings.

The authors' conclusions

Results indicate that validated questionnaires like AUDIT-C do not of themselves ensure the quality of clinical screening; quality should still be monitored. The VA's targets for rates of alcohol screening create incentives for documenting screening results, but do not provide incentives for high quality screening that identifies patients in need of advice. Setting very high target rates could contribute to lower quality by encouraging providers to document screening when they did not have the time to ask screening questions verbatim and in private. Future research must address the need for performance measures which not only incentivise providers to screen, but also to identify alcohol misuse.

In particular, 61% of patients who screened positive for alcohol misuse on a mailed survey screened negative when screened clinically, despite use of the same validated screening questionnaire. Some discordance is to be expected due to differences in settings and methods and normal fluctuations. However, the observed discordance cannot be accounted for by these factors.

Desire to present oneself in a good light socially, avoid perceived stigma, and/or to avoid discussing drinking with one's health carers, probably accounted for some of the discordance in the form of minimising one's drinking. Over twice as many patients who screened positive in a confidential mailed survey had discordant results compared to patients with positive clinical screens (61% v. 24%). If the latter is an indication of the 'expected' discordance rate, then the extra 37% discordance is an indication of how many more patients present themselves as moderate or non-drinkers when facing their clinicians or when they know clinicians will see their answers.

That black patients were more likely than other groups to present 'better' at the clinic than in the survey might reflect greater social desirability bias among these patients, although it could also reflect bias due to differences in the way AUDIT-C was interpreted and/or administered across racial/ethnic subgroups. Variation in discordance across VA networks suggests that institutional factors also played their part. Anecdotally, there is considerable variability across sites in the privacy of screening. Differences in training and/or decisions about who conducts screening might also contribute to variability. Patients who returned the VA's outpatient satisfaction survey and could be included in the featured study were older and drank less than non-respondents. Since discordance was greatest among alcohol misusers, this means that across all patients the proportion of misusers not identified by clinic screening may be even greater than the 61% recorded in this study.


Findings logo commentary As reviewers have observed, research has largely evaluated screening in highly controlled studies when research staff administer and score the evaluated screening test and the test against which it is being benchmarked. In contrast, we know comparatively little about screening in real-world clinical settings. This study takes us further down that road, and implies that previous studies may have been falsely reassuring about the performance of screening tests in routine practice, in particular how well they identify risky drinkers when their answers could (or are thought to) have consequences for the respondent which they wish to avoid. Regardless of which was the more accurate set of answers to AUDIT-C, the results remind us that the context in which responses are made can have a great influence and should be borne in mind when devising screening strategies.

The social desirability of minimising one's drinking in a medical context seems a plausible explanation for the findings, along with the desire to avoid further treatment or referral to substance use services. This is likely to have been potentiated or aggravated by the lack of privacy in some clinics to which the authors refer, which may have contributed to the substantial inter-clinic variation; clearly questionnaires completed on paper and in private may predispose to responses biased differently to those asked out loud in a public reception area. How much difference consistent assurance of privacy would have made is unclear; there may still have been a residual desirability bias.

The study was conducted in a health care system which in US terms comes close to the principles of the British national health service, with the notable exception that it is for ex-military, meaning also that it sees few women, though for this study they were over-sampled to increase the numbers.

What do we really know about screening?

Studies in which the screening test being evaluated and the benchmark (and usually more detailed) test are both completed in similar circumstances (eg, both in a surgery and seen by clinicians, administered anonymously, or in confidence by researchers) cannot tell how many risky drinkers would be missed by the evaluated test in routine practice. For instance, a US study using a confidential survey found single questions about either how often or how much someone had drunk correctly placed over 80% of drinkers in problem and non-problem categories – seemingly reassuring, but only when the questions are being asked in a research context where the answers carry no or minimal consequences for the respondents. The high chance that otherwise answers will be at least partly self-serving is why researchers have have developed screening instruments which are not obviously about drinking or drug use.

A review of the AUDIT test published in 2007 which included mainly primary care studies also raised the issue of the validity of a test whose results are "entirely a function of the respondent's ability and willingness to provide accurate information on his or her use of alcohol and its effects". The reviewers' doubts had been raised by a study of transport sector employees attending a company health service for a routine health examination, in the course of which they were offered an alcohol screening test. Together with blood tests, 22% of patients screened as risky drinkers on one or other test, but AUDIT alone would have identified only half these.

The English SIPS trial of screening and brief intervention in primary care took in only patients who had already screened positive in the short tests being evaluated and compared these with results from the full AUDIT questionnaire. That there was good agreement between the measures cannot in these circumstances tell us how many patients were missed because they did not initially screen positive.

Another study conducted in Welsh primary care practices did not suffer from the same limitation, yet still found the AUDIT questionnaire an accurate way to identify hazardous drinkers. This study did sample patients who initially scored negative on an AUDIT administered by research nurses but scored by practice nurses, and tested whether these results matched those from a more extended assessment conducted by researchers. In all 69% of patients found by the extended assessment to be drinking in a hazardous manner had also scored this way on the AUDIT, suggesting that even when practice nurses see the results, the test is moderately good at identifying drinkers who might benefit from advice. However, in this study under a fifth (18%) of AUDIT-negative patients invited to join the study did so, leaving the AUDIT unvalidated on over four fifths of patients who answered its questions in a way which indicated they had no appreciable drinking problems.

VA brief intervention studies

Also in the Effectiveness Bank is a test of using computerised prompts to remind VA clinicians to counsel patients who screen positive for risky drinking. Although this found low counselling rates and no improvement in drinking where the reminders were implemented, another implementation of the same system in VA clinics did find high counselling rates and some indication of drinking reductions. Early results from this study were reported in a review of performance measurement options for VA alcohol screening and brief intervention systems. Also available is an overview of issues and findings in respect of implementation of similar systems in the VA network nationally. In the Effectiveness Bank too are a review of what impedes or promotes the implementation of brief alcohol interventions by the VA research team, and another conducted for Britain's National Institute for Health and Clinical Excellence. The latter analysis includes extended commentary on the UK situation, partially replicated in a 'hot topic' entry discussing whether brief alcohol interventions really can deliver population-wide health gains.

Thanks for their comments on this entry in draft to Robert Patton of the National Addiction Centre in London, England. Commentators bear no responsibility for the text including the interpretations and any remaining errors.

Last revised 07 February 2013. First uploaded 03 February 2013

Comment/query to editor
Back to contents list at top of page
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates


Top 10 most closely related documents on this site. For more try a subject or free text search

STUDY 2010 Use of an electronic clinical reminder for brief alcohol counseling is associated with resolution of unhealthy alcohol use at follow-up screening

STUDY 2010 Evaluation of an electronic clinical reminder to facilitate brief alcohol-counseling interventions in primary care

STUDY 2011 An evaluation to assess the implementation of NHS delivered alcohol brief interventions: final report

STUDY 2013 Screening and brief intervention for alcohol and other drug use in primary care: associations between organizational climate and practice

REVIEW 2011 Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review

REVIEW 2011 Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review

STUDY 2012 Alcohol screening and brief intervention in primary health care

HOT TOPIC 2015 Can brief alcohol interventions improve health population-wide?

STUDY 2012 Alcohol screening and brief intervention in emergency departments

REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking





Measuring performance of brief alcohol counseling in medical settings: a review of the options and lessons from the Veterans Affairs (VA) health care system.

Bradley K.A., Williams E.C., Achtmeyer C.E., et al.
Substance Abuse: 2007, 28(4), p. 133-149.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Bradley at willi@u.washington.edu. You could also try this alternative source.

Having mandated universal screening for alcohol problems, the US health system for ex military personnel here thoughtfully addresses how to measure the degree to which this led to appropriate implementation of brief interventions.

Summary Brief alcohol counselling is a top US prevention priority but has not been widely implemented. The lack of an easy performance measure for brief alcohol counselling is one important barrier to implementation. The purpose of this report is to outline important issues related to measuring performance of brief alcohol counselling in health care settings. We review the strengths and limitations of several options for measuring performance of brief alcohol counselling and describe three measures of brief alcohol counselling tested in the Veterans Affairs (VA) Health Care System. We conclude that administrative data are not well-suited to measuring performance of brief alcohol counselling. Patient surveys appear to offer the optimal approach currently available for comparing performance of brief alcohol counselling across health care systems, while more options are available for measuring performance within health care systems. Further research is needed in this important area of quality improvement.


Findings logo commentary Having mandated virtually universal screening for alcohol problems, the US health system for ex military personnel faced the problem of how to measure its success in a way which could drive up the implementation of brief interventions in appropriate cases. It concluded that just four or five extra questions in patient satisfaction surveys could be used to assess how many should have been counselled and then how many actually were by being given feedback on their health risk and explicit advice to cut back. The study also demonstrates how an automated clinical reminder system for positive-screen patients can raise counselling rates to nearly 70%.

Last revised 29 November 2008

Comment/query to editor
Back to contents list at top of page
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates


Top 10 most closely related documents on this site. For more try a subject or free text search

STUDY 2011 Supporting partnerships to reduce alcohol harm: key findings, recommendations and case studies from the Alcohol Harm Reduction National Support Team

STUDY 2012 Text-message-based drinking assessments and brief interventions for young adults discharged from the emergency department

STUDY 2011 An evaluation to assess the implementation of NHS delivered alcohol brief interventions: final report

DOCUMENT 2011 Alcohol dependence and harmful alcohol use quality standard

DOCUMENT 2011 Services for the identification and treatment of hazardous drinking, harmful drinking and alcohol dependence in children, young people and adults

STUDY 2008 Reducing alcohol harm: health services in England for alcohol misuse

STUDY 2010 Use of an electronic clinical reminder for brief alcohol counseling is associated with resolution of unhealthy alcohol use at follow-up screening

STUDY 2013 Screening and brief intervention for alcohol and other drug use in primary care: associations between organizational climate and practice

REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking

REVIEW 2011 Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review





Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review.

Williams E.C., Johnson M.L., Lapham G.T. et al.
Psychology of Addictive Behaviors: 2011, 25(2), p. 206–214.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Williams at emily.williams3@va.gov.

Applying a systematic and comprehensive framework to map the strategies trialled in attempts to implement screening and counselling for risky drinking primary care patients gives some clues to what it has taken to achieve a high screening rate, the essential first step in the process.

Summary This review's starting points were the observations that screening for risky drinking in primary care patients followed by brief counselling as needed has been shown to reduce drinking, and is in some countries considered a public health priority, yet sustained implementation in to routine clinical practice has not occurred, and what might facilitate implementation remains unclear. To inform implementation efforts, the review drew on the international literature to map evaluated efforts to implement screening and brief counselling, and attempted to relate the degree of success of these implementations to the strategies used to encourage implementation.

To map implementation strategies, the reviewers used the Consolidated Framework for Implementation Research. In respect of health care innovations in general, this model identifies five implementation domains, each divided in to several sub-domains. The five main domains with relevant examples are:
Characteristics of the intervention (in this case, alcohol screening and brief intervention in primary care) such as the strength of the evidence for its effectiveness and how far it was adapted to fit the particular circumstances in which it was being implemented.
Outer setting, which refers to the economic, political, and social environment surrounding and influencing the organisation undertaking the implementation – in this case, typically primary practices and organisations offering primary care services; included here might be national political drivers, the demand from patients and their identified need for the intervention, and the degree to which the implementing organisation is networked with others (such as accreditation bodies) in ways which might promote or hinder implementation of the intervention.
Inner setting is pertinent features of the implementing organisation including the degree to which its structures, internal communication mechanisms, resources, leadership, and culture facilitate the adoption of innovations, and the degree to which the particular innovation 'fits' the organisation's needs and circumstances.
Characteristics of the individuals conducting the intervention – in this case, doctors and other primary care staff – such as what they believe about the intervention and how enthusiastic and ready they are to implement it.
Process of implementation – the extent and quality of the implementation effort, including the degree to which relevant staff are actively engaged, the efficiency with which the implementation is carried out, the extent to which progress is appropriately monitored against specific goals and progress news fed back to the participants, and the extent to which this feedback is used to adapt and promote implementation.

Methodology

English language studies available up to March 2010 were included in the review if they evaluated the implementation of alcohol screening and brief intervention into routine primary care practice when screening and intervention were primarily conducted by usual primary care staff rather than research staff. For each implementation the analysts calculated the screening rate (the proportion of patients who should have been screened actually were) and the brief intervention rate (the proportion of patients who screened positive for risky drinking were actually counselled). These outcomes were related to the extent to which each implementation adopted the implementation strategies mapped in the Consolidated Framework for Implementation Research.

Main findings

Although the analysts found 17 relevant reports, these derived from just eight implementation programmes. These efforts spanned nine countries and involved 533,903 patients (127,304 of whom were screened), 2001 providers, and 1805 medical clinics. Across the programmes the screening rate varied hugely from 2% to 93% and so did the brief intervention rate, from 1% to 73%. The programmes adopted between 7 and 25 of the 38 detailed strategies identified in the Consolidated Framework, generally adopting at least one from each of the five major domains.

At 93%, the US health service for former military personnel screened the highest proportion of the patients intended to be screened. In this study the implementation effort was distinguished by extensive use (12 of 14 sub-domains) of Inner Setting domain strategies, of Process of Implementation strategies (7 of 8 sub-domains), and of Outer Setting strategies (3 of 4 sub-domains). Two other US programmes achieved the next highest screening rates of 65% and 60%. They too used several Inner Setting (5 of 14) and Process of Implementation (4 of 8) strategies, but not to an obviously greater degree than the remaining programmes with much lower screening rates ranging from 2% to 26%.

Of patients who screened positive for risky drinking, again it was the programme mounted by the US health service for ex-military which (at one of the implementation sites) achieved one of the highest proportions counselled. At 71%, their record was exceeded only by the 73% recorded in another US study. As noted above, the programme for former military personnel was implemented using a uniquely broad range of strategies but the same could not be said of the top-ranking programme, and no clearly successful configuration of strategies emerged from the remaining studies, whose rates ranged widely from 1% to 66%.

The authors' conclusions

The programme mounted by the US health service for ex-military personnel reported a substantially higher rate of alcohol screening than others and could be distinguished from other programmes by its focus on multiple elements of the Inner Setting, Outer Setting, and Process of Implementation domains of the framework. Strategies focused on the Inner Setting and Process of Implementation domains also characterised the two programmes next in the screening rate ranking. This suggests that focusing implementation strategies on Inner Setting, Outer Setting, and Process of Implementation domains is associated with high rates of screening. However, the picture was nether detailed nor entirely consistent: implementation programmes with the highest rates of screening did not consistently share a focus on the same sub-domains within these broad categories and, when they did, were not easily discernable from programmes which did not report high rates of screening.

It may be relevant that each of these three very successful screening implementation efforts deployed electronic medical records and some form of performance accountability via measurement and feedback. They also all took place in large, geographically diverse networks of clinical practices with centralised administrations that included a research infrastructure. Possibly their screening successes were partly due to being conducted within infrastructures aligned to the implementation and evaluation of programmes. This is, however, not to say that smaller networks or single practices cannot (perhaps with different methods) achieve good results.

Though for screening rates there was some indication of what distinguishes a successful implementation, this was not the case for the next phase of the procedure, engaging positive screen patients in counselling about their drinking. It seems likely that the strategies necessary to implement screening differ from those necessary for brief intervention. Screening involves the application of a validated screening survey or other method, which can be done either by the patient or by clinical staff at all levels. In contrast, counselling risky drinkers is more complex, typically requiring individualised assessment and judgement regarding the specific feedback and advice to be offered.

Though this review was able to offer limited guidance on what makes for a successful implementation, the framework on which it was based (or other similar frameworks) can be useful in other ways. Firstly, as a roadmap for planning an implementation programme, and secondly, as a structure for documenting the strategies tested in an implementation effort.

However, this particular framework and others too perhaps have their limitations. The framework assumes that a single intervention is being implemented, when, in fact, implementing screening and brief intervention involves multiple steps, each of which may be responsive to different implementation strategies. Also it is often unclear whether a strategy belongs in the Inner or Outer Setting domain. For example, when the clinic where the work is taking place is part of a larger organisation, is that organisation an Outer or Inner domain influence? It also seems likely that there is no single answer to what is needed to successfully implement alcohol screening and brief intervention in primary care practices. In different circumstances, different strategies will be needed and be effective. When for example the existing environment, organisation and staff are already highly conducive to implementation, domains identified by the framework may be less closely related to success than in less conducive circumstances.


Findings logo commentary In recent years Britain has certainly made progress in extending alcohol screening and brief intervention to more primary care patients, but it is unclear whether this has been to the degree needed to make noticeable public health gains, and provision remains patchy. The framework adopted by the featured study offers one way to audit which implementation levers have been activated and which have yet to be adequately activated, revealing the gaps in implementation efforts.

In both England and Scotland, the prime objective for primary care is to screen new patients and/or those thought in advance to possibly be at risk from their drinking. Screening newly registered patients was the reimbursement indicator for the enhanced alcohol service. Initially for two years from 2008 but then extended to March 2013, this requires all primary care trusts in England to offer GP practices in their areas the chance to contract to provide alcohol screening and brief intervention to their new patients. If they wish, local commissioners can go further to contract for more extended services. Also in England, directors of public health are expected to include such activity among attempts to address the population-wide determinants of ill health.

In line with Scotland's own practice recommendations, national policy in Scotland prioritises screening and brief intervention, backed by a health service target for 2008/09–2010/11 to deliver 149,449 brief interventions supported by dedicated funding. The target was exceeded; over the three-year period 174,205 alcohol brief interventions were recorded across the three priority settings – primary care, accident and emergency departments, and antenatal services. In 2008, the Welsh Assembly Government announced its intention to instigate a programme to promote alcohol brief interventions in both primary and secondary health care settings.

These policy initiatives implement guidelines from Britain's National Institute for Health and Clinical Excellence (NICE), which encourage screening for new patients and in circumstances where both patient and doctor might feel it was 'natural' and justified to ask about a patient's drinking. Touching on a key barrier to widespread implementation beyond these circumstances, the guidelines cautioned that, "Clinical consultations for non-alcohol-related medical problems can be an inappropriate time to discuss alcohol use, given that users are focused on the condition for which they are seeking advice", and recognised the greater acceptability of discussing drinking "in a context that is related to the purpose of the visit (such as lifestyle assessment or chronic condition monitoring)".

It is unclear how far things have moved on since 2008 when an national audit found that systematic screening by GPs in England was the exception and few patients were screened or offered brief advice. The requirement to offer screening and intervention contracts to GPs has generated more activity, but far from consistently, and the quality and even the reality of the services supposed to have been provided has been questioned. In London in 2010 a survey of staff responsible for local alcohol policy indicated low levels of investment in developing the role of GPs in screening and treating alcohol use disorders. Nearly two thirds of areas had yet to invest in or develop screening systems beyond those nationally required. In one large London borough not known for the rarity of its drinking problems, over half the practices which had contracted to provide screening failed to identify any risky drinkers using the stipulated screening survey, and in a year screening resulted in just ten people being referred for treatment. Whilst reluctance to address drinking 'out of the blue' is understandable, there is even reluctance to raise the topic in general health and well-being assessments.

As expressed in the featured report, brief interventions have tremendous public health potential. A remaining major challenge is how to consistently realise that potential. This hot topic search retrieves relevant documents on the Findings site, but as yet these do not include the unpublished results from a government-funded national implementation trial in England, whose findings are expected to be highly influential.

Last revised 09 November 2011

Comment/query to editor
Back to contents list at top of page
Give us your feedback on the site (two-minute survey)
Open Effectiveness Bank home page
Add your name to the mailing list to be alerted to new studies and other site updates


Top 10 most closely related documents on this site. For more try a subject or free text search

REVIEW 2011 Strategies to implement alcohol screening and brief intervention in primary care settings: a structured literature review

REVIEW 2011 Barriers and facilitators to implementing screening and brief intervention for alcohol misuse: a systematic review of qualitative evidence

STUDY 2010 Routine alcohol screening and brief interventions in general hospital in-patient wards: acceptability and barriers

STUDY 2013 Screening and brief intervention for alcohol and other drug use in primary care: associations between organizational climate and practice

STUDY 2014 The effectiveness of alcohol screening and brief intervention in emergency departments: a multicentre pragmatic cluster randomized controlled trial

REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking

STUDY 2013 Modelling the cost-effectiveness of alcohol screening and brief interventions in primary care in England

STUDY 2010 Use of an electronic clinical reminder for brief alcohol counseling is associated with resolution of unhealthy alcohol use at follow-up screening

STUDY 2012 Alcohol screening and brief intervention in primary health care

STUDY 2013 Effectiveness of screening and brief alcohol intervention in primary care (SIPS trial): pragmatic cluster randomised controlled trial





L10 Web Stats Reporter 3.15 LevelTen Hit Counter - Free PHP Web Analytics Script
LevelTen dallas web development firm - website design, flash, graphics & marketing