This entry is our analysis of a study added to the Effectiveness Bank. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.
Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text
Copy title and link
| Comment/query | Tweet
Williams E.C., Achtmeyer C.E., Kivlahan D.R. et al.
Journal of Studies on Alcohol and Drugs: 2010, 71(5), p. 720–725.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Williams at emily.williams3@va.gov. You could also try this alternative source.
When a patient has screened positive for risky drinking, up pops a computerised prompt to remind the clinician to consider counselling, yet at a service for US ex-military personnel the reminder was rarely used and made no difference to patients' drinking. Why were results so different from those at other clinics?
Summary Using electronic medical records to remind clinicians to consider or undertake preventive work can increase provision of recommended preventive care, including brief alcohol counselling to patients who screen positive for risky drinking.
The US 'VA' health care service for ex-military personnel commonly deploys clinical reminders in electronic records in conjunction with national performance measures linked to financial incentives. The service nationally implemented annual alcohol screening in 2003, resulting in over 90% of all outpatients being screened for unhealthy drinking. It remains however to maximise the numbers then offered advice.
With this aim in mind, an electronic clinical reminder was developed to prompt clinicians at VA services to offer brief alcohol counselling when patients were recorded as having screened positive for unhealthy drinking on the AUDIT-C screening questionnaire. As well as prompting the service provider, the system offered them information about what constitutes evidence-based brief alcohol counselling, supported assessment of alcohol use severity, provided an aid to deciding whether to implement brief counselling or referral, and documented these actions in the patient's record. Though prompted automatically, providers could choose whether to open and act on these and any other prompts in respect of other conditions.
Previously this system had been found to be associated with high counselling rates in a VA network and some indication of drinking reductions, but it was unclear whether similar results would be found where providers do not routinely use reminders.
This was addressed by the featured study over a three-year period at a single VA primary care clinic where reminder use was neither routine nor expected or incentivised. The reminder system was implemented in one randomly selected part of the clinic but not in the other part, offering a way to compare performance with and without the system in the same clinic. Implementation consisted solely of setting up the system and one e-mail to tell providers about this in the selected part of the clinic.
At issue was the degree to which the reminder system was used after clinicians had been alerted that their patient had screened as a risky drinker During the study period, the clinic had not implemented incentives to conduct alcohol screening, and the proportion of patients who were screened is unknown. on the AUDIT-C questionnaire, and whether use rates varied depending on the severity Men who scored 4–7 and women 3–7 were considered to screen positive for mild/moderate unhealthy alcohol use, and patients with scores of 8–12 were considered to screen positive for severe unhealthy alcohol use. of the patient's drinking. Use of the reminder was defined as an entry in the patient's medical record indicating that the clinician had used it to help assess, advise, or refer the patient to specialty care, or an entry indicating that drinking had not been addressed.
Also at issue was whether implementing the reminder system led to more patients overcoming their unhealthy alcohol use (defined as now screening negative plus at least a two point out of 12 reduction in score) at a follow-up screening conducted at least 18 months after the initial screen, relying only on VA records. Findings were adjusted for differences between patients, including those of the kind found by research to be associated with receipt of brief alcohol counselling and changes in drinking. Among these were multiple indicators of history and severity of unhealthy drinking.
Essentially, no significant differences were found in the improvements made by risky drinking patients attending the two parts of the clinic, perhaps partly because clinicians who were prompted by the reminders used the system for just 15% of patients; further details below.
The first issue was whether the reminders were used. Of 22,863 patients who visited both parts of the clinic, 18% screened as drinking in an unhealthy manner, including 2640 patients assigned to the part of the clinic using the reminder system. Of these 2640, for just 398 (15%) was there any documented use of the clinical reminder. Use was more likely for patients with severe (20%) versus mild/moderate (14%) unhealthy drinking. Just 6% of positive screen patients were offered brief counselling, again more commonly (17% v. 4%) when the alert to the clinician had indicated a severe problem. Of the brief counselling interventions, a quarter consisted of advice to abstain and 15% to drink less. Most (59%) severe drinkers were advised to abstain, but about the same proportions of less severe drinkers (14% and 13% respectively) were advised to abstain or drink less. Nurse practitioners were the largest group of clinicians and also the ones most likely (in 23% of cases) to use the reminder system.
The remaining issue was whether having the reminder system made any difference to the patients' drinking. This could be assessed for 1358 of the 4202 positive-screen patients who were re-screened 18 months or more later. Of these, 40% where the reminder system had been implemented had resolved their unhealthy drinking compared to 43% where it had not, statistically not a significant difference. Given greater use of the reminders by the nurses, the researchers tested whether at least among their patients the system had made a difference; it had not. They also tested whether the minority of patients for whom the reminders had actually been used, or those then offered a brief intervention, had more often resolved their drinking problems than other patients. In neither case was there any substantial or statistically significant difference in resolution rates.
With no active implementation efforts, little encouragement by local leaders for providers to use clinical reminders, and no incentives for their use or for brief alcohol interventions, few providers used a clinical reminder designed to facilitate brief interventions with patients who screened positive for unhealthy drinking. Providers who did use the reminder were mostly nurse practitioners; advice to abstain was the care most frequently documented. Offering providers access to the reminder did not lead to more patients who initially screened positive later resolving their unhealthy drinking.
Documented rates of use were higher in patients with severely unhealthy drinking, who were generally advised to abstain, probably reflecting biases of providers toward traditional case-finding approaches involving referring patients with alcohol-use disorders.
These findings suggest that a relatively passive clinical reminder alone is insufficient to get brief interventions onto the agenda of busy primary care providers. In contrast, at another VA centre where providers were expected to use clinical reminders, they did succeed in moving brief alcohol counselling up the agenda irrespective of the severity of the patient's drinking, and patients whose providers had used the reminders were (modestly) more likely to report having resolved unhealthy alcohol use at follow-up than other patients.
Studies have found that use of clinical reminders seems to depend on local clinical culture, and that clinicians adopt these more readily when aligned with performance measures and supported by leadership. Although there was a national performance measure for annual alcohol screening at the time of this study, there was none for brief intervention, and reminder use by providers was not routine at the featured site.
Together these findings suggest that clinical reminders might help implement brief interventions when accompanied by expectations that they will be used or incentives to promote use. As health care systems seek to integrate brief interventions into routine care, it will be important to consider other components of effective implementation, including incentives or clear expectations for providers to use decision-support systems, as well as to identify and address barriers to effective use of clinical reminders.
Previous trials of the effectiveness of brief interventions and/or methods of implementation have included select groups of patients and providers who consented to participate in research and were potentially more motivated to address drinking than typical patients and providers. These trials also focused specifically on alcohol-related counselling, and may have created unrealistic clinical situations which ignored other needs of primary care patients. Instead the results of the featured study derived solely from routine clinical and quality assurance procedures, evaluating a method of implementing brief interventions in a real-world clinical setting not limited by selection bias. Despite these strengths, the findings are vulnerable to remaining differences between counselled and non-counselled patients which could not be adjusted for, and to differences between providers. It could also be that patients were counselled but this was not documented in the reminder system.
commentary There was indeed, as the authors point out, a startling difference between the 15% use rate of the reminder in the featured study and the 71% rate recorded at different VA clinics. As well as the organisational factors mentioned by the authors, in that earlier study the reminders were implemented across entire clinics, not just in one part – a situation which might make it difficult for leaders to insist the reminders should be used.
In the context of other studies, the earlier study provided a convincing demonstration that such reminders can set the stage (but as the featured study shows, not always) for raising counselling rates to high levels. But it was much less convincing about any beneficial impact on drinking. Without comparison sites where the reminder system was not implemented, the small difference in the resolution of problems in patients who were or were not (according to records) counselled was indicative of at best a very modest impact, and possibly none at all given the limitations of the study. This in turn may have been linked to the inability to assess or influence the quality of the counselling and even whether it actually happened.
The conclusion that whether a tool like the reminder system is used depends on the culture and management of the organisation is in line with emphases in reviews (1 2) of the implementation of screening and brief intervention.
While the featured study was mainly about the counselling which should follow screening, another study has questioned the validity of screening results in the national VA system, finding that 61% of patients who screened positive when sent a postal survey did not do so when the same questions were asked as part of their routine care at their VA clinics.
Also in the Effectiveness Bank is a review of performance measurement options for VA alcohol screening and brief intervention systems. This includes initial results from a more successful implementation of the system implemented in the featured study, later more fully evaluated in a report which found high counselling rates and some indication of drinking reductions. Another report has focused on the screening element at VA services nationally. Also available is an overview of issues and findings in respect of implementation of similar systems in the VA network nationally. In the Effectiveness Bank too are a review by the same research team and another conducted for Britain's National Institute for Health and Clinical Excellence of what impedes or promotes the implementation of brief alcohol interventions. The latter analysis includes extended commentary on the UK situation, partially replicated in a 'hot topic' entry discussing whether brief alcohol interventions really can deliver population-wide health gains.
Last revised 02 February 2013. First uploaded 02 February 2013
Comment/query
Open Effectiveness Bank home page
Top 10 most closely related documents on this site. For more try a subject or free text search
HOT TOPIC 2017 ‘My GP says I drink too much’: screening and brief intervention
REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking
STUDY 2011 Quality concerns with routine alcohol screening in VA clinical settings
STUDY 2008 Universal screening for alcohol problems in primary care fails in Denmark and no longer on UK agenda
STUDY 2012 Alcohol screening and brief intervention in primary health care