Improving the delivery of brief interventions for heavy drinking in primary health care: outcome results of the Optimizing Delivery of Health Care Intervention (ODHIN) five-country cluster randomized factorial trial.
Anderson P., Bendtsen P., Spak F. et al.
Addiction: 2016, 111(11), p. 1935–1945.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Anderson at firstname.lastname@example.org. You could also try this alternative source.
The EU-funded ODHIN trial tested eight strategies to promote screening and brief interventions for risky drinking in primary health care units in five European countries. Results suggested that financial incentives were key but were reinforced by training and support.
Summary The EU-funded ODHIN trial tested strategies to promote screening and brief interventions for risky drinking in primary health care units in five European countries. The featured article reports impacts on screening and brief intervention while the strategies were in place. Our account also draws on the study’s plan and registration documents, the follow-up sub-study testing whether the effects of the strategies persisted six months after they ended, and cost-effectiveness estimates from three of the countries, also analysed separately for the Effectiveness Bank.
The EU-funded ODHIN trial tested eight strategies to promote screening and brief interventions for risky drinking in primary health care units in Catalonia in Spain, England, the Netherlands, Poland and Sweden – an important attempt to find how best to bridge the gap between the numbers who might benefit from these interventions and those who actually receive them.
While training/support allied with payments for were in place, the boost given these gave to implementation rates of screening and brief interventions was convincing and substantial.
The implications are that financial incentives for clinicians and/or their workplaces and giving clinicians the tools to do this work extend any benefits to more patients, though still a small minority.
Several features of the study lead to doubt about the size, persistence and statistical significance of the effects it registered and their benefits for patients and society, and yet more so about the resulting cost-effectiveness of the strategies.
Among these is the degree of impact brief interventions have on drinking and how long these last, and whether impacts seen in the study would be replicated in a routinely and broadly implemented programme.
Based on evidence of effectiveness, many national and international guidelines recommend routine screening in primary health care and the offer of advice to patients whose screening scores indicate risky drinking. However, commonly there is a large gap between the need for advice and its provision. For example, the featured study found that before attempts were made to increase this, only 5.3% of adult patients consulting their primary health-care practitioner during a four-week period were screened for their drinking.
Conducted in the Catalonia region of Spain, England, the Netherlands, Poland and Sweden, the featured study attempted to raise both screening and brief intervention rates among primary care patients aged at least 18 through a variety of incentive and support strategies offered to their surgeries and clinicians. The aim was not to test whether as a whole these narrowed the ‘implementation gap’, but to find which strategies worked best. The variety of primary care structures and drinking patterns across the countries means the results may be applicable across Europe and other similar Western countries.
Of 618 primary health care units invited to join the study, 120 did so. Of the practitioners in those units who volunteered to record their screening and brief intervention activity, 55% were doctors, 38% nurses and 7% other permanently appointed practice staff such as psychologists.
The packages offered the units were intended to promote screening for hazardous and harmful alcohol consumption using the three questions of the AUDIT-C screening questionnaire, which assesses the respondent’s typical current drinking pattern. When screening suggested a patient was at risk due to their drinking, practitioners were asked to deliver a brief alcohol intervention consisting of advice lasting five to 15 minutes in line with their national guidelines, or instead (among units allocated to this option) to refer the patients to an alcohol screening and advice website.
The main outcomes assessed were:
• the screening rate, defined as the percentage of consultations with practitioners in the trial during which screening was completed, and;
• the brief advice rate, defined as the percentage of patients screening positive who (orally or through a leaflet) were then advised about their drinking, or referred for advice to the websites arranged for the study or elsewhere.
Also reported was the proportion of all consultations with clinicians participating in the trial which resulted in a patient screening positive for risky drinking and being given brief advice. Termed here the population intervention rate, this is an amalgam of the screening rate, the proportion of screens which revealed risky drinking, and the brief advice rate.
To raise screening and advice rates, over a 12-week implementation period during 2012 and/or 2013 four foundation strategies were tried alone or in combination:
Minimal. All the units were given cards summarising their national guidelines on screening and advice for hazardous and harmful alcohol consumption, and clinicians were asked to screen all patients aged at least 18.
Training and support. Units were offered two initial face-to-face educational training sessions lasting one to two-hours adapted for each country, plus one 10–30-minute phone support call to the lead contact in the unit during the first four to six weeks of the implementation period.
Financial reimbursement. Levels and maximum earnings were in line with normal fees and rates for clinical preventive activities in each country, but the principle was that each primary care unit or clinician would be paid in proportion to the number of times they screened or advised patients. For example, in England fees were €6 per screening and €25 per advice session, up to a maximum of €2,200 per unit.
Brief intervention website. Practitioners were asked to refer patients identified as at-risk drinkers to a website developed for that country by handing them a leaflet with the website’s address and a personal log-in code, and briefly explaining why they should use the site and how to do so. As a minimum the sites offered patients screening for risky drinking, feedback on the results, information on sensible drinking guidelines and on alcohol’s impacts on health and wellbeing, and a drink-diary facility.
Within each country, units were allocated at random to one of the foundation strategies or to a combination. The result was eight sets of 15 units offered packages escalating from minimal only to all four foundation strategies:
• Minimal only. The control set of units were offered only the minimal strategy, the aim being to test whether adding further support or incentives improved implementation rates.
• Minimal plus training and support.
• Minimal plus financial reimbursement.
• Minimal plus brief intervention website.
• Minimal plus training and support and financial reimbursement.
• Minimal plus training and support and brief intervention website.
• Minimal plus financial reimbursement and brief intervention website.
• Minimal plus training and support, financial reimbursement and brief intervention website.
For each unit, screening and brief advice rates were assessed during a four-week ‘baseline’ period before the implementation strategies started. During this time clinicians were asked to respond to hazardous and harmful drinking in their usual manner, but to record each time they conducted screening or a brief intervention. Two to six weeks later the implementation strategies started and then ran for 12 weeks, during which screening and brief advice rates were re-assessed over each of the three four-week periods. After a break of about six months the rates were re-assessed over a four-week follow-up period. Effectively the main analyses tested whether rates changed from the baseline to a different degree depending on the implementation strategy offered the units.
In summary, of the foundation strategies added on their own to the minimal control strategy, financial reimbursement gave the greatest boost to screening and brief intervention rates, the biggest impact being on screening rates. These impacts were magnified when financial reimbursement was allied with training and support, a combination which consistently gave the maximal boost to implementation rates and was estimated to be most cost-effective. However, financial reimbursement was effective only when it was being applied; though its effects also waned, training and support had a longer-lasting impact. Details below.
Sophisticated analyses reported in the following sections assessed statistical significance and teased out active ingredients. This section provides a simple description of the raw figures which fed into those analyses.
During the baseline period, on average about 6% of consultations included screening for risky drinking, and when the patient screened positive, 74% resulted in a brief intervention. The upshot was that on average about 1% of consultations resulted in a patient who screened positive for risky drinking being given what the study defined as brief advice. In England in particular, just under 1 in 20 of the consultations intended to feature screening did so and about half resulted in a positive screen, 86% of which were followed by advice or referral for advice about the patient’s drinking.
Whenever financial reimbursement was part of the implementation package, screening rates increased from the pre-incentives baseline to the 12 weeks when the incentives were applied, maximally when combined with training and support but not website referral. But even this package resulted in just 18% of consultations involving patients practitioners were asked to screen actually including screening. In the absence of financial reimbursement, the screening rate fell when units were offered training and support, a website to refer patients to, or both.
In contrast to screening rates, except for the control units offered only minimal encouragement/support, brief advice rates universally increased, reaching 90% when financial reimbursement was combined with training and support (and not website referral).
Population intervention rates also increased whenever financial reimbursement was part of the implementation package, except when it was combined solely with website referral. The greatest increase – a near tripling – came when financial reimbursement was combined with training and support but not website referral.
All three rates tended to fall during the 12-week implementation period.
However, since screening and brief advice rates fell (and in the case of screening, fell steeply) among the control set of units offered only minimal encouragement/support, the main issue was not whether the supplementary strategies were accompanied by absolute increases or decreases in the rates, but whether these significantly differed from the benchmark set by the control units. Additionally an attempt was made to tease out active ingredients by testing whether packages featuring each strategy led to significantly higher implementation rates than packages lacking that strategy. These results are reported below.
In respect of the screening rate, financial reimbursement and training and support emerged as the most active ingredients, while website referral was not found effective and may have been counterproductive when combined with some other strategies; details below.
Of the solo implementations of the foundation strategies, financial reimbursement was associated with the greatest increase in screening. Changes in the control group were also significantly improved on by adding training and support with or without website referral. When financial reimbursement was combined only with website referral, there was no significant effect. By some way the largest increase in screening rates was seen among units offered both financial reimbursement and training and support but not website referral. On their own or together, training and support and website referral did not significantly increase the screening rate relative to control units.
To more specifically tease out active ingredients, the researchers tested whether packages featuring each strategy led to a significantly higher screening rate than packages lacking that strategy. This analysis reinforced the findings above. Of the foundation strategies, again financial reimbursement was the most prominent; packages including this doubled the rate compared to packages without it. Next was training and support, whose roughly 50% boost to screening rates just reached statistical significance. In contrast, when website referral was part of the package the screening rate was (non-significantly) slightly lower than when it was not. The biggest boost to screening rates came from packages featuring both the effective foundation strategies – financial reimbursement and training and support – especially when not accompanied by website referral.
In respect of the brief advice rate too – the proportion of identified risky drinkers advised about their drinking – financial reimbursement and training and support emerged as the most active ingredients, while website referral may have been counterproductive when combined with some other strategies; details below.
Of the solo implementations of the foundation strategies, financial reimbursement was associated with the greatest increase in the brief advice rate relative to control units. All packages incorporating financial reimbursement significantly increased the relative rate, maximally when combined with training and support but not the website option, and minimally when combined only with website referral. Packages including training and support were also generally significantly effective; the exception was when combined only with website referral. Unless packages featuring website referral also offered both training/support and financial reimbursement, there was no significant effect relative to control units.
As with screening, to tease out active ingredients the researchers tested whether a package featuring each evaluated strategy led to a significantly higher brief advice rate than packages lacking that strategy. In contrast to screening, the presence or absence of strategies led to little variation in the brief advice rate, and none of the findings were statistically significant. However, the non-significant and therefore possibly chance trends reinforced the findings reported in the previous paragraph, suggesting that financial reimbursement and training and support were effective ingredients, the more so when combined. In contrast, website referral was associated with no increase in the brief advice rate unless combined with both training/support and financial reimbursement – and then the increase was less than when website referral was left out of the pacakage.
Relative to control units, none of the implementation strategies significantly improved the population intervention rate (the proportion of all adult consultations with participating clinicians which resulted in a patient screening positive and being given brief advice) between the pre-strategies baseline and the 12 weeks when the strategies were in place. Though chance findings could not be ruled out, the rate did increase about twice as much as in control units, except when website referral was combined either with training and support or financial reimbursement, but not both; in these cases, the rate barely increased.
Again, to tease out active ingredients the researchers tested whether a package featuring each strategy led to a significantly higher rate than packages lacking that strategy. Results mirrored those for the screening rate. Of the foundation strategies, financial reimbursement was the most prominent; packages including this doubled the rate compared to packages without it. Next at 1.6 times greater was training and support. The biggest boost to population intervention rates (2.4 times greater) came from packages featuring both these foundation strategies – financial reimbursement and training and support – especially when not accompanied by website referral. In contrast, when website referral was part of the package, the rate was only significantly higher when referral was reinforced by both financial reimbursement and training/support, and then to a lesser degree than when referral was omitted from the package.
While it was applied financial reimbursement was the most effective of the strategies, but when reassessed about six to seven months after the strategies had been withdrawn, only training and support had significantly contributed to a persisting positive difference in the population intervention rate. This rate – the proportion of all adult consultations with participating clinicians which resulted in a patient screening positive and being given brief advice – was the sole outcome reported in the follow-up sub-study.
To reach this finding the researchers tested whether a package featuring each strategy led to a persistently higher population intervention rate than packages lacking that strategy. The only statistically significant finding was a greater rate when training and support had been part of the package than when it had not, but the difference was small and took the form not of an increased rate relative to baseline, but a lesser fall. During the pre-strategies four-week baseline, 1.1 per 100 consultations resulted in a screen-positive patient being given advice. In units not offered training and support, by the follow-up this had fallen to just under 0.8 per 100; with training and support it had also fallen, but by a lesser amount to just over 0.9 per 100. When present the combination of training and support and financial reimbursement also led to a greater persisting rate than when absent, but the difference was not statistically significant.
Notwithstanding the results of this sophisticated analysis, the raw figures showed that only strategies which had included financial reimbursement ended at the follow-up with a higher population intervention rate than before the strategies were applied, but the differences were marginal. From the longer-term perspective of the follow-up it became clear that financial reimbursement (especially when reinforced with training and support) had substantially boosted the population intervention rate while the incentives were in place, but that this regressed to near baseline levels within six or seven months of the incentives ending, possibly sooner. At its maximum, financial reimbursement with training and support (but not website referral) had boosted the rate from about 1 per 100 patients to just over 3, but it fell back to just over 1 per 100 in the follow-up period chart.
A cost-effectiveness study based largely on the follow-up results from three of the countries in the study has been separately analysed for the Effectiveness Bank. The conclusions were that for all three (including England), training and support plus financial reimbursement yielded the greatest health gains, and did so at a cost below each country’s ceiling for a cost-effective use of resources to gain of an extra year of life adjusted for the health-related quality of that year (a ‘QALY’). Costs were estimated as the cost of the interventions minus any savings in healthcare costs associated with alcohol-related conditions.
In England in particular, a QALY was gained at a cost to health services of €3,257, well below the €22,918 cost-effectiveness ceiling adopted at the time by the National Institute for Health and Care Excellence (NICE). This finding assumed that the effect of brief interventions on drinking faded over seven years. At €21,668 per QALY gained, the [arguably more defensible below] assumption that effects decayed over three years left the cost-effectiveness estimate barely under the €22,918 ceiling. Extra costs, but also greater effectiveness, would come from continuing financial reimbursement, and extra costs from refreshing training every few years on the assumption that this was needed to sustain greater activity.
On the basis of the 12 weeks the implementation strategies were running, the researchers recommended that to maximise brief intervention in primary care, jurisdictions “could consider” providing training and support and financial reimbursement plus guidance and strategic leadership. With the follow-up results suggesting financial reimbursement was effective only when it was being applied, recommendations changed to focus instead on training and support. As concluded on the basis of the 12-week implementation period, for two of the three countries which could be analysed (including England) the cost-effectiveness analysis came out in support of a combination of training and support and financial reimbursement.
Overall the authors of these three sub-studies seem to favour training and support plus persisting financial incentives in the context of strong government support for brief alcohol interventions and performance management arrangements, guidance, and strategic leadership. This combination promises to maximally raise what the study found was a very low baseline intervention rate; just 3% of primary care patients whose heavy drinking meant they might have benefited from brief advice actually received it. Impacts on the extent of brief advice came mainly via raising the screening rate. Even before the implementation strategies, patients identified as risky drinkers were usually advised. Proved unfounded were expectations that providing a website to refer patients to would increase the advice rate by relieving clinicians of the need to do the advising themselves.
However, these conclusions were based on findings from a sample of clinicians who volunteered for the study and had a greater than average confidence in and commitment to working with drinkers, meaning they were more motivated to undertake this work than primary health care clinicians in general in the same countries.
commentary The featured study was an important attempt to find the best way to promote universal screening of primary care patients for risky drinking and of advising those who screened positive. Largely due to just a few per cent of patients being screened, poor implementation rates in the UK and elsewhere have been recognised as a – if not the – major impediment to achieving the hoped-for health gains across an entire population. Though these findings can be questioned, there is some evidence that if this implementation gap could be substantially closed, screening and brief interventions in primary healthcare may offer “a cost-effective policy option for tackling alcohol-related harms, at least in high-income countries”.
While these were being applied, the boost given by payments and training/support to the screening rate (feeding through to the proportion of patients advised about their risky drinking) was substantial and ‘made sense’, making it a convincing finding. The implications are that financial incentives for clinicians and/or their workplaces (in which clinicians may have a financial stake) and giving clinicians the tools to do this work extend any benefits to more patients – though still a small minority. However, whether there were any benefits for the patients could not be established by the study, and there are reasons (expanded on below) to doubt assumptions made about the size of these benefits. Despite training and support, screening and brief intervention activity receded once payments were withdrawn, offering further evidence of their impact while active. If anything, the attempt to mount a ‘resource-light’ programme by referring patients to a website backfired.
The study adds to a body of literature evaluating screening and brief intervention strategies in primary care whose findings were synthesised in a meta-analysis published in 2015 (free source at time of writing). It found that across all studies, strategies to increase implementation had boosted both screening and the brief intervention rates but not significantly affected drinking. Greatest impacts on screening and brief intervention were seen from multi-strand strategies, and screening benefited from involving staff such as nurses as well as doctors. These findings are consistent with the argument that “To foster development of positive attitudes and effective responses … a focus that extends beyond the individual worker is required. Education and training are a necessary, but not sufficient, condition to ensure health professionals’ capacity and willingness to respond to [substance use] issues. Research on organizational culture provides valuable insight into the types of organizational and systems factors likely to influence … attitudes and work practice.”
As far as England is concerned, the featured study tested a strategy no longer being prioritised as the country has largely abandoned what seemed the unrealistic universal screening strategy, instead narrowing the ambition to ‘targeted’ screening during health checks, of new patients, or of those already thought to be at risk. In practice this seems to have been what was happening during the baseline period, when in England fewer than 1 in 20 of the intended patients were screened, of whom about half screened positive, indicative of selection for screening of patients thought by clinicians most likely to have been heavy drinkers. The figures were similar for the other two countries for which this data has been published. Across all five countries, boosted by training and support and financial reimbursement, the screening rate peaked across the 12-week implementation period at 18% of consultations, still well short of the intended universality. Interviews with clinicians (of which more below) revealed that “Despite their intrinsic motivation to prevent patients from [suffering] alcohol-related disabilities, GPs and nurses feel more rationale for selective [ie, targeted] screening rather than opportunistic screening.”
Without questioning the boost to activity given by payments allied with training/support, several features of the study lead to doubt about the size, persistence and statistical significance of the effects it registered and their benefits for patients and society, and yet more so about the resulting cost-effectiveness of the strategies, calculating which entailed further assumptions and uncertainties.
Considerations expanded on below include the assumptions made on the basis of other studies about the effect of brief interventions on drinking and how long they last. The featured study was unable to check if these materialised for its clinicians’ patients because no data was gathered on impacts on the patients’ drinking or alcohol-related harm. Contrary to the assumption that increased intervention activity meant reduced drinking, boosted intervention rates promoted by strategies such as those tested in the study have not been shown to have significantly affected drinking. An update of the review used by the study to set the assumed drinking reductions due to its strategies suggests that these reductions were overestimated, and doubts remain over whether any such estimates derived from pooling research findings apply to routine practice. In respect of the featured study in particular, these doubts are magnified by the fact that participating clinicians were particularly motivated to help reduce alcohol-related harm. Findings of increased implementation rates associated with some of the strategies would have been aided by the unexplained steep fall in these rates in control units which provided the benchmark against which the strategies were compared. There are reasons to suspect these falls were generated by the study itself, creating an artificially low benchmark. Clinicians directly or perhaps indirectly rewarded financially were responsible for collecting the data which justified those payments, with no checks on volume or quality of the activity being paid for. Of concern too is that follow-up results were based on an outcome (the population intervention rate) which was not specified in advance, opening the door to selecting an outcome which cast the implementation strategies in the best light. Considerations specific to the cost-effectiveness calculations include the study’s inability to account for the value of what might have been done with the time clinicians devoted to alcohol-related activities. These considerations are expanded in the supplementary text: click to unfold .
The puzzle of the suppressant effect of website referral option when combined with other strategies – consistently reducing screening and brief intervention rates compared to strategies which omitted it – may be explained by clinicians’ reactions to this option. These were probed through interviews with 40 GPs and 28 nurses who participated in the trial in Catalonia, the Netherlands, Poland, or Sweden. Just over half were among the 25% most frequent screeners for their country during the 12-week implementation phase of the trial, and the remainder among the 25% least frequent.
In general, essential ingredients for implementation seemed to be gaining knowledge and skills, team-based training, and learning to prioritise screening and brief interventions even during high-workload periods. Most of the professionals allocated to training and support saw their nation’s or the EU’s drinking guidelines to be feasible and compatible with their daily practice. Among this set of clinicians aware of the importance of alcohol-related harm and motivated to help prevent it, commonly those allocated to training and support thought yet more provision would help further, calling for “continuous training provision, more time to learn intervention techniques and more tailoring to experienced barriers, such as a perceived lack of time”. Already generally convinced of the need for this work, what training and support primarily offered was practical ways to do it. Training also helped them prioritise alcohol-related work, but this could be just a “temporary stimulus … Embedding [screening and brief intervention] in the long term requires a continuous trigger, such as booster sessions.”
Payments for screening and brief intervention were seen as most important when they went directly into the hands of the relatively poorly paid clinicians in Catalonia and Poland rather than (as in other countries) to the primary care service as a whole. Elsewhere payments might be considered irrelevant to patient-centred practice.
Only in respect of website referral as a brief intervention option (termed ‘e-BI’) was opinion negative across all four countries. In all four, “patients’ lack of interest inhibited both nurses and GPs from being active in referring patients to e-BI”. This option neither offered clinicians guidance in providing brief interventions nor did it engage the patients, “Therefore, face-to-face interventions were the preferred method in such cases.” Frequent screeners did not refer to this option as facilitating their performance, while infrequent screeners saw it as counterproductive. But it was not just experience in the trial that affected clinicians’ willingness to screen and intervene if the end product was referral to a website. Not one mentioned having access to this option as a reason for participating in the trial, and most were ambivalent about ‘e-health’. Even those positive about ‘e-BI’ saw it primarily as a way to provide information for patients, presumably rather than as an intervention in itself.
Studies reviewed in the Effectiveness Bank indicate that without material or reputational and possibly career-affecting sanctions/incentives, implementation drives based on educating, persuading and supporting practitioners have reached just a minority of the intended patients. Strong sanctions and incentives can generate the desired activity, but may be costly and tempt services and practitioners to short-change quality and ‘game’ the system.
In the cost-effectiveness sub-study ODHIN researchers referred to ethical questions surrounding financial incentives to healthcare practitioners. Writing in the British Journal of General Practice, Dr Graham Kramer expressed the following about the implications of rewarding practices for the delivery of high quality care (in this case under the UK’s Quality and Outcomes Framework):
“I had worried that, by being paid to implement evidence-based guidelines, my work would become a restricted, target-driven exercise that shifted the balance of my consultations to a doctor and disease-centred agenda. I had been concerned that this created conflicts of interest and how that might undermine, not only trust by my patients in me as a doctor, but also the trustworthiness of the profession. I worried that in some domains I was taking money to engage in work that I felt had limited value for my patients, money that could possibly be spent in more useful areas. Was I colluding in a wholesale folly of medical practice and worse still, why wasn’t I doing anything about it? Had my mouth been effectively ‘stuffed with gold’?”
The Alcohol Treatment Matrix in the Effectiveness Bank (see cell E1) discusses incentives, including whether financial incentives divert clinical practice in the intended direction at the risk of distorting record-keeping and practice overall, and confirming to staff that alcohol screening and brief interventions are not their core business.
Last revised 31 August 2019. First uploaded 18 August 2019