Evidence-based psychotherapy relationships: Collecting client feedback
Effectiveness bank home page. Opens new windowReview analysis

This entry is our analysis of a review or synthesis of research findings added to the Effectiveness Bank. The original review was not published by Findings; click Title to order a copy. Free reprints may be available from the authors – click prepared e-mail. The summary conveys the findings and views expressed in the review. Below is a commentary from Drug and Alcohol Findings.

Links to other documents. Hover over for notes. Click to highlight passage referred to. Unfold extra text Unfold supplementary text
Copy title and link | Comment/query |

Evidence-based psychotherapy relationships: Collecting client feedback.

Lambert M.J., Shimokawa K.
Psychotherapy: 2011, 48(1), p. 72–79.
Unable to obtain a copy by clicking title? Try asking the author for a reprint by adapting this prepared e-mail or by writing to Dr Kolden at michael_lambert@byu.edu. You could also try this alternative source.

This meta-analytic review commissioned by the American Psychological Association finds outcomes improve (and clients doing poorly can be 'rescued') when therapists get real-time feedback on patient progress and the client-therapist relationship. Providers may want to consider one of the evaluated systems or an alternative.

Summary Updated in 2018; see Effectiveness Bank analysis.

[Though not specific to patients with drug and alcohol problems, studies in the analyses described included such patients, and the principles are likely to be applicable to these disorders among others, not least because substance use problems generally form part of a complex of broader psychosocial problems.]

This review is one of several in a special issue of the journal Psychotherapy devoted to evidence-based, effective therapist-client relationships. It reports on a research synthesis of whether therapy outcomes are improved when therapists are provided regular, real-time feedback on how the client is responding using either of the two feedback systems evaluated to date. Unlike the other reviews, it had available to it and relied on trials which (usually randomly) allocated clients to feedback-informed therapy or therapy without feedback.

Feedback systems have been developed partly to identify patients who are not doing well or deteriorating in time for the therapist to do something about it, and partly simply to improve the therapist's performance by making them more aware (and correcting mistaken impressions) of how well the client is doing, what is working, what is not, and which areas need further attention.

Monitoring and systematic feedback are valuable to the degree that they provide information beyond that a clinician can observe and understand without these aids. The simplest way to do this is to assess client improvement on dimensions related to the desired outcomes. However, this does not in itself necessarily motivate or guide practice adjustments.

Two evaluated systems have gone beyond this. The first is the Partners for Change Outcome Management System, a psychotherapy assurance system based on two four-item scales. One measures psychological wellbeing and social functioning, the other the strength of the therapeutic alliance. Normally these are completed by the patient in the presence of the therapist, affording an opportunity to discuss the results. Three studies of the system were included in a new meta-analysis, A study which uses recognised procedures to combine quantitative results from several studies of the same or similar interventions to arrive at composite outcome scores. Usually undertaken to allow the intervention's effectiveness to be assessed with greater confidence than on the basis of the studies taken individually. the only English-language studies to have randomly allocated clients to no feedback or feedback-based therapy using this system. One was conducted at a university counselling centre, another US study was at a graduate training clinic, and the third at a Norwegian family counselling clinic providing couples therapy.

The other system is the OQ Psychotherapy Quality Management System, which also assesses psychological disturbance (particularly anxiety and depression) and social functioning, and, in a second scale, the therapist-client relationship. The first is applied to all patients and identifies those 'not on track' – whose progress is falling behind that expected According to the system's statistically generated expected recovery curve for differing levels of pre-treatment distress. for patients with equally severe problems. These patients then complete the therapist-client relationship scale, whose results are then used to decide a corrective course of action according to the system's flowchart of what to try in certain circumstances. The primary aim of the entire system is to prevent treatment failure, or more specifically, to enhance the outcome of clients the system predicts would otherwise experience treatment failure. A recent analysis had already amalgamated results from the six trials of the OQ system, all of which compared outcomes from feedback-based therapy to those of a no feedback comparison group. In four studies clients had been randomly allocated. All but one study had been conducted at a university counselling centre.

The featured review drew on this existing meta-analysis, and conducted a new one amalgamating results from the three Partners for Change system studies. Both synthesised results from relevant studies to provide estimates of the overall improvement in client progress conferred by allocating clients to a feedback-based system. In both too, only clients who completed a certain amount of the treatment programmes and of the research assessments were included in the analyses. The degree to which feedback improved psychosocial wellbeing outcomes was expressed as effect sizes A standard way of expressing the magnitude of a difference (eg, between outcomes in control and intervention groups) applicable to most quantitative data. Enables different measures taken in different studies to be compared or (in meta-analyses) combined. Based on expressing the difference in the average outcomes between control and experimental groups as a proportion of how much the outcome varies across both groups. By convention, in the behavioural sciences an effect size of the type used in the featured analyses is considered a small effect at 0.10, a medium effect at 0.30, and a large effect at 0.50. using the 'r' metric, which can be squared to calculate how much of the difference in outcomes can be attributed to differences in the therapy dimension being investigated. The assumption was made that there is no single, true strength of the link between outcomes and feedback which appears to vary only because of methodological differences, but that instead strength really might vary across the studies included in the analysis.

Main findings: Partners for Change

Across the three studies of the Partners for Change system, compared to treatment as usual, providing feedback led to significantly higher post-treatment psychological wellbeing and social functioning as measured by the system's own scale. The advantage equated to a small to medium effect size of 0.23, indicating that the average client in the feedback group was better off than about 68% (50% is the 'break-even' point) of those whose treatment was conducted without systematic feedback.

Conventional benchmarks were applied to these outcomes to calculate the numbers of clients whose improvements were great enough to be considered real and reliable rather than due to measurement error, and the numbers who had deteriorated to an equally reliable degree. For both feedback and non-feedback groups, the odds of being an improver versus a non-improver were calculated. These odds were 3.5 times higher when clients had feedback-informed therapy, a statistically significant gain equating to an effect size of 0.34. The corresponding ratio for deterioration versus non-deterioration was more than halved when clients had been in feedback-informed therapy, but this was not a statistically significant difference. Worth noting is that in the couples therapy study, six months later non-feedback couples who could be followed up were nearly twice as likely to have divorced or separated (34% v 18%) as those whose treatment had been informed by systematic feedback.

These results suggest that even very brief questionnaires feeding back to therapists data on client progress and therapeutic relationships substantially improve progress, when applied in a way which enables the therapist to use the results in therapy.

Main findings: OQ system

Given its different objectives (to prevent treatment failure), results for the OQ system are presented only for clients the system had predicted would otherwise have done badly. The issue addressed was whether across the six studies, fewer of these clients actually did poorly in feedback-informed therapy than without systematic feedback. As measured by the system's own scale, across the six studies, for these clients feedback significantly improved post-treatment psychological wellbeing and social functioning. The improvement equated to a small to medium effect size of 0.25 when feedback on these dimensions was provided only to the therapist. This effect was no stronger when feedback was also provided to the client.

However, it was slightly strengthened when instead of feeding back to the client, the therapist was given further assistance in the form of feedback from the therapeutic relationship scale plus corresponding guidance on a corrective course of action. With this information – not just on the fact that things are going badly, but why they might be and what could be done about it – clients ended treatment feeling better psychologically and also feeling that their relationships were better than when no feedback was provided at all, equating to a small to medium effect size of 0.33. [Editor's note: in the original paper the gain due to providing extra information and guidance to the therapist versus just feedback on client progress was small but statistically significant when all the clients were included in the analysis.]

These results meant that average clients in the feedback groups ended better off than 70% to 76% of those treated without systematic feedback (50% is the 'break-even' point). When these results were converted to improvement and deterioration indices ( above for what this means), the 20% of clients who 'reliably' deteriorated without feedback was cut to 9% with therapist feedback, 15% with this plus client feedback, and 6% with therapist feedback plus feedback on the therapeutic relationship and guidance on corrective action. Corresponding figures for the proportions of these unpromising clients who nevertheless reliably improved were 22% without feedback, and 38%, 45% and 53% respectively in the three feedback-informed treatments.

[Editor's note: in the original paper there was no statistically significant difference between the three different types of feedback when all the clients were included in the analysis. Also non-significant results suggested that providing the client with feedback as well as the therapist led more to reliably improve, but also more to reliably deteriorate, as if some clients could use and benefit from feedback on their poor progress, while others became yet more demoralised. This paper also reports results for clients who were not identified as doing poorly. Because all seemed well, their therapists did not receive extra feedback and guidance, so the only issues are whether outcomes were improved by feeding back data on the client's psychosocial progress to the therapist, or to the therapist and the client. On average this was the case in both feedback conditions, and in both more clients reliably improved. When only the therapist received feedback, fewer clients reliably deteriorated, an effect not apparent when the client was also given feedback.]

These results suggest that 'real-time' feedback to therapists on their clients' progress which enables the therapist to use the results in therapy does help rescue clients from failure and improve their outcomes and outcomes overall. For clients doing poorly, extra gains from offering feedback on the client-therapist relationship and 'what to do' guidance to the therapist are modest but apparent, while offering progress feedback to the client as well as the therapist is of equivocal extra value.

Practice recommendations

Reliance on these recommendations must be tempered by the fact that they derive from few studies, few researchers, just two self-report measures, and just two feedback systems, a corpus of work which offers a very limited view of the benefits of feedback.

Use real-time client feedback to monitor patients' responses to psychotherapy and satisfaction with the therapy relationship. Such feedback probably improves psychotherapy outcomes overall, and certainly for clients at risk of deterioration or drop-out.

Employ real-time client feedback to compensate for therapists' limited abilities to accurately detect client deterioration. Therapists' often mistaken confidence in their clinical judgment is a barrier to implementation of monitoring and feedback systems.

Beware of when clients feel it may be in their interests to under- or over-state their problems, supplying inaccurate feedback ratings which render feedback systems less effective.

As suggested by the general literature on feedback and the evidence presented here, in addition to feedback, problem-solving and decision-enhancement tools help clinicians and, most importantly, clients at risk of treatment failure.

It is unclear whether always and directly sharing progress feedback with clients (as opposed to this being at the therapist's discretion and reframed as they see fit) improves outcomes over and above feeding back to the therapist alone.

Consider using electronic versions of feedback systems that expedite and ease practical difficulties, providing real-time or pre-session feedback at the cost of just a few minutes of the client's time.

Findings logo commentary This article was in a special issue of the journal Psychotherapy devoted to effective therapist-client relationships. For other Findings entries from this issue see:
Evidence-based psychotherapy relationships: Psychotherapy relationships that work II
Evidence-based psychotherapy relationships: Alliance in individual psychotherapy
Evidence-based psychotherapy relationships: The alliance in child and adolescent psychotherapy
Evidence-based psychotherapy relationships: Alliance in couple and family therapy
Evidence-based psychotherapy relationships: Cohesion in group therapy
Evidence-based psychotherapy relationships: Empathy
Evidence-based psychotherapy relationships: Goal consensus and collaboration
Evidence-based psychotherapy relationships: Positive regard
Evidence-based psychotherapy relationships: Congruence/genuineness
Evidence-based psychotherapy relationships: Repairing alliance ruptures
Evidence-based psychotherapy relationships: Managing countertransference
Evidence-based psychotherapy relationships: Research conclusions and clinical practices

The special issue which contained the article featured above was the second from the task force. The first was a special issue of the Journal of Clinical Psychology. While the second aimed to identify elements of effective therapist-client relationships ('What works in general'), the first aimed to identify effective ways of adapting or tailoring psychotherapy to the individual patient ('What works in particular'). For Findings entries from this first special issue see this bulletin. Both bodies of work have also been summarised in this freely available document from the US government's registry of evidence-based mental health and substance abuse interventions.

Last revised 31 May 2011. First uploaded

Open Effectiveness Bank home page

Top 10 most closely related documents on this site. For more try a subject or free text search

REVIEW 2018 Collecting and delivering progress feedback: a meta-analysis of routine outcome monitoring

ABSTRACT 2011 Evidence-based therapy relationships: research conclusions and clinical practices

DOCUMENT 2017 Drug misuse and dependence: UK guidelines on clinical management

REVIEW 2018 Psychotherapy relationships that work III

STUDY 2011 Therapist effectiveness: implications for accountability and patient care

REVIEW 2011 Evidence-based psychotherapy relationships: Repairing alliance ruptures

REVIEW 2011 Evidence-based psychotherapy relationships: Goal consensus and collaboration

REVIEW 2018 Meta-analyses of the relation of goal consensus and collaboration to psychotherapy outcome

REVIEW 2018 Alliance rupture repair: a meta-analysis

REVIEW 2018 A meta-analysis of the association between patients’ early treatment outcome expectation and their posttreatment outcomes