This entry is our analysis of a study considered particularly relevant to improving outcomes from drug or alcohol interventions in the UK. The original study was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the study. Below is a commentary from Drug and Alcohol Findings.
Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text
Copy title and link
| Comment/query | Tweet
Angus C., Li J., Romero-Rodriguez E. et al.
European Journal of Public Health: 2019, 29(2), p. 219–225.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Angus at c.r.angus@sheffield.ac.uk. You could also try this alternative source.
Could combinations of three strategies – training and support, financial reimbursement, and the opportunity to refer patients to a website – cost-effectively boost delivery of brief interventions in European primary care? The important aim was to find the best way to narrow the ‘implementation gap’ between the number of patients who could benefit from these interventions and those who receive them.
Summary Screening and brief interventions for heavy drinking are effective (1 2) and cost-effective (3) approaches to reducing alcohol-related harm. Yet delivery rates (eg, only 1 in 20 eligible patients being screened in Europe) remain low.
Current delivery of screening and brief interventions in primary care is low across Europe. To address this ‘implementation gap’, the featured study invertigated the cost-effectiveness of strategies intended to boost delivery.
In England, the Netherlands and Poland, training and support, financial reimbursement, and the opportunity to refer patients to an online tool were found likely to be cost-effective strategies for increasing rates of delivery.
Several features of the study lead to doubt about the size, persistence and statistical significance of the effects it registered and their benefits for patients and society, and yet more so about the resulting cost-effectiveness of the strategies.
Using data collected in the Optimizing Delivery of Health Care Interventions (ODHIN) trial, in combination with the Sheffield Alcohol Policy Model, the featured study aimed to assess the cost-effectiveness of levers to improve the delivery of screening and brief interventions, enabling policymakers to make more informed decisions when allocating potentially scarce resources.
Described more fully elsewhere in the Effectiveness Bank, the ODHIN trial took place in 120 primary care settings, equally distributed across five European countries: England, the Netherlands, Poland, Catalonia in Spain, and Sweden.
In 2012/2013 the trial tested the effects of three different strategies on the delivery of screening and brief interventions by primary care services:
There were eight arms to the trial in total. Primary care practices were randomly allocated to either a control group given basic information on national safer drinking guidelines and asked to screen all adult patients, or additionally to one of the three strategies above or combinations of these strategies. The control group of practices was used to estimate the results of not mounting any appreciable programme to increase implementation of screening and brief interventions, providing a benchmark against which to assess the more active implementation options.
Delivery rates were measured during a baseline period before the strategies, during a 12-week implementation period when the strategies were running, and during a four-week follow-up period six months later. For the cost-effectiveness calculations, the measure of effectiveness used was the proportion of all adult consultations with participating clinicians which resulted in a patient screening positive for risky drinking and being given brief advice – what we have called the “population intervention rate”.
One of the options tried in the ODHIN trial was per-patient payments for alcohol screening and advice. In England these payments were €6 per screening and €25 per patient advised, up to a ceiling of €2,200 per practice. Relative to basic information, across the five nationsEngland, the Netherlands, Poland, Catalonia and Sweden. payments significantly doubled the screening rate, feeding through to a non-significant doubling in the proportion of patients seen at the practices who were offered advice on their drinking. But still just 1 in 8 attendees were screened leading to about 2% being advised.
When the analysis assessed whether among all the combinations of intervention-promoters, certain elements seemed to make a difference when present versus when not, payments emerged as the most important component, one which allied with training and support more than doubled the screening rate leading to a similar increase in the proportion of patients being advised. However, even this combination would across the entire sample have raised the proportion of risky drinkers who received advice from an estimated 3% before to only about 9%Before the programmes 11 out of 1000 patients were advised while 330 scored as risky drinkers on a screening test = 3.33% of risky drinkers advised. With training/support and financial incentives, the advice rate among those practices rose from 12.5 per 1000 patients to 34.6. Extrapolated to the whole sample, this would have had the effect of raising the advice rate to about 34.6/12.5 x 3.33, equating to 9.2%. afterwards. Without knowing if there were any impacts on the patients, the most the researchers could say was that jurisdictions “could consider” promoting brief alcohol advice in primary care through training and guidance, financial and performance management arrangements, and strategic leadership.
The analyses described below estimated the long-term costs and effects of the trialled strategies for three out of the five countries – England, the Netherlands and Poland – over a 10-year period of investment and a 30-year window of health outcomes, to account for time lags between changes in alcohol consumption and the development of alcohol-related harm.
To assess cost-effectiveness the researchers first estimated how many years of life adjusted for the quality of those years (quality-adjusted life yearsOne quality-adjusted life year is equal to one year of life in perfect health. or ‘QALYs’) each implementation strategy would save relative to the minimal information and encouragement given to the control group of practices, a measure of the increased health gained by each strategy. Then the costs of each strategy were calculated, again relative to no strategy at all. Costs included the cost of screening and brief intervention and resultant healthcare costs associated with the treatment of alcohol-related health conditions; effective programmes should reduce healthcare costs, potentially more than offsetting the cost of the programme. Dividing net extra costs by QALYs gained yielded an estimate of how much extra each strategy would cost the health service of each country per extra QALY gained. A strategy was considered cost-effective if this figure was lower than what each country (according to official policy) considered a cost they were prepared to pay to gain a quality-adjusted year of life.
A key challenge in estimating the impact of the strategies on delivery was that, while online brief interventions and training and support are essentially ‘one-off’ policies – ie, practitioners are trained or introduced to the online tool at the outset and not subsequently re-trained – financial reimbursement requires continuous investment. To overcome this issue, two different types of analyses were conducted: the first, providing a measure of exactly what was implemented in the trial (ie, financial reimbursement was withdrawn after 12 weeks); and the second, an estimate of what would have happened had payments continued for the full 10 years. When payments were assumed to continue through the 10 years the analysis applied to the entire period the effects on the population intervention rate seen during the 12-week implementation phase; otherwise the effects were those seen at the follow-up six months after the strategies had been withdrawn.
The most effective strategy at increasing the delivery of brief interventions was training and support combined with financial reimbursement, although this had one of the highest costs for each additional brief intervention delivered across all three countries. In contrast, online brief interventions appeared to offer the cheapest way to achieve delivery of additional brief interventions, but was among the least effective strategies at increasing delivery.
In all three countries, training and support plus financial incentives were estimated to yield the greatest health gains, and to do so at a cost below each country’s ceiling for cost-effectiveness per extra year of life in perfect health:
• England, €3,257 costs v. €22,918 ceiling;
• The Netherlands, €3,953 v. €20,000;
• Poland, €8,319 v. €14,666.
In England, relative to the minimal control strategy, both financial incentives alone and financial incentives plus training and support generated health gains while actually reducing overall costs. In the Netherlands this was also the case for financial incentives, although the estimated scale of savings was very different between the two countries (€150 million and €7.8 million respectively). Both training and support, and training and support plus financial incentives, incurred a net cost to the healthcare system in Poland, with the most effective strategy (training and support plus financial incentives) costing €6.8 million extra over 30 years compared to the control group. Estimated health gains under training and support plus financial incentives were largest in England at 15,400 QALYs over 30 years compared to 2,400 in the Netherlands and 2,600 in Poland.
When additional analyses were performed based on the assumptions that (1) financial incentives continued to be paid for a full 10-year period, and (2) that, as observed in previous studies, the effect of these on provider behaviour was maintained in the long term, the results were broadly similar. Training and support plus financial incentives was still the optimal strategy in England and Poland, but it was no longer cost-effective in the Netherlands compared to training and support alone. As in the previous analyses, the optimal strategies were estimated to be cost-saving in England and the Netherlands, but not Poland, and the health gains in England were significantly larger than in the other countries.
Based on assumptions that training had to be re-delivered every five or every two years in order to achieve any persistence of effect of other strategies, there were significant increases in the costs associated with any strategy involving training and support. For example, the cost over 10 years of delivering training and support in the Netherlands increased from €8.6 million to €15.9 million with re-training every five years, and €36.9 million with re-training every two years. However, these increased costs made little difference to the overall cost-effectiveness results and the overall conclusions of the analysis. The only significant change was that training and support ceased to be cost-effective in the Netherlands if re-training was required every two years, with referral to online brief interventions becoming the most cost-effective option under this scenario.
When the researchers assumed that the effect of brief interventions on alcohol consumption persisted for less time (three years), cost-effectiveness was reduced for all strategies. While training and support plus financial incentives remained the most cost-effective option for both England and the Netherlands (€21,668 and €13,413 respectively per extra QALY gained), it was no longer cost-effective in Poland, with training and support alone offering the most cost-effective strategy at €2,609.
While delivery rates of screening and brief interventions in European primary care are currently low, several cost-effective strategies exist to increase these rates.
Training and support combined with financial incentives may offer the most cost-effective strategy for increasing delivery, and subsequently reducing alcohol-related harm and associated costs to society. However, this finding is sensitive to both the characteristics of the country and assumptions around the long-term effects of brief interventions. Furthermore, policymakers may need to be mindful of the potential ethical issues (1 2) associated with offering financial incentives to healthcare practitioners.
commentary The level of alcohol consumption in Europe is high compared with the rest of the world, and although there is evidence that screening and brief interventions in primary healthcare may offer “a cost-effective policy option for tackling alcohol-related harms, at least in high-income countries”, their level of implementation in routine primary healthcare remains low. The featured study was part of a package of work seeking to address this problem – assessing the cost-effectiveness of levers to improve delivery. However, before accepting that the study has found ways to cost-effectively narrow the ‘implementation gap’ and move towards brief interventions being delivered “widely and well enough” to improve health across the entire population, it is necessary to examine the study’s underlying assumptions, including the assumption that brief interventions remain effective in real-world settings – arguably the ‘Achilles heel’ of the estimates.
Cost per QALY (years of life adjusted for the health-related quality of that year) calculations depend on the assumptions and data fed into them and the influences on both cost and quality/length of life taken into account. In the case of the featured study these limitations are substantial enough to cast doubt on the whether the calculations are a reliable guide to policy and practice.
Considerations include an over-estimate of the size of the effect of brief interventions on drinking and in the primary analysis too, how long they last, doubts over whether such research-derived estimates apply to routine practice, and uncertainty over whether continued payments and training or support would sustain screening and brief intervention activity, all magnified by the fact that the practitioners in the study were particularly motivated to respond to drinking. Omitted from the calculations were the costs of checking screening and intervention volume and quality to substantiate eligibility for payments, and the value of what else might have been done with the time devoted to alcohol-related training, screening and intervention. The projected healthcare savings which largely generated the cost-effectiveness findings were not based on reports from the patients themselves but assumed drinking reductions and consequent savings, yet across all relevant studies, increased intervention rates promoted by strategies such as those tested in the study have not been shown to have significantly affected drinking. Findings of raised screening, brief intervention and population intervention rates for some of the strategies would have been aided by the unexplained steep fall in these rates in the control units which provided the benchmark against which the strategies were compared, findings which fed into the cost-effectiveness estimates. Of concern too is that these estimates were based on an outcome (the proportion of all adult consultations with participating clinicians which resulted in a patient screening positive and being given brief advice) which was not specified in advance, opening the door to selecting an outcome which cast the implementation strategies in the best light. These considerations are expanded in the supplementary text: click to unfold .
Studies reviewed in the Effectiveness Bank indicate that without material or reputational and possibly career-affecting sanctions/incentives, implementation drives based on educating, persuading and supporting practitioners have reached just a minority of the intended patients. Strong sanctions and incentives can generate the desired activity, but may be costly and tempt services and practitioners to short-change quality and ‘game’ the system.
In the cost-effectiveness sub-study ODHIN researchers referred to ethical questions surrounding financial incentives to healthcare practitioners. Writing in the British Journal of General Practice, Dr Graham Kramer expressed the following about the implications of rewarding practices for the delivery of high quality care (in this case under the UK’s Quality and Outcomes Framework):
“I had worried that, by being paid to implement evidence-based guidelines, my work would become a restricted, target-driven exercise that shifted the balance of my consultations to a doctor and disease-centred agenda. I had been concerned that this created conflicts of interest and how that might undermine, not only trust by my patients in me as a doctor, but also the trustworthiness of the profession. I worried that in some domains I was taking money to engage in work that I felt had limited value for my patients, money that could possibly be spent in more useful areas. Was I colluding in a wholesale folly of medical practice and worse still, why wasn’t I doing anything about it? Had my mouth been effectively ‘stuffed with gold’?”
The Alcohol Treatment Matrix in the Effectiveness Bank (see cell E1) discusses incentives, including whether financial incentives divert clinical practice in the intended direction at the risk of distorting record-keeping and practice overall, and confirming to staff that alcohol screening and brief interventions are not their core business.
Last revised 30 August 2019. First uploaded 09 January 2019
Comment/query
Open Effectiveness Bank home page
Top 10 most closely related documents on this site. For more try a subject or free text search
STUDY 2012 Alcohol screening and brief intervention in primary health care
STUDY 2012 Alcohol screening and brief intervention in emergency departments
REVIEW 2010 Alcohol-use disorders: Preventing the development of hazardous and harmful drinking
STUDY 2012 Alcohol screening and brief intervention in probation