Effective clinical feedback provision to ambulance clinicians: a literature review

Evaluating the effectiveness of feedback provision

All of the research papers were observational studies without concurrent control so the majority highlighted that establishing the causality of results was a study limitation. Few studies were able to demonstrate any effects of feedback on patient outcomes.

However, many demonstrated improvements in performance measures with previous evidence supporting their beneficial value, such as call-to-hospital treatment times for acute myocardial infarction (AMI) and hands-on-chest times during cardiopulmonary resuscitation (CPR) (Scholz et al, 2008; Daudelin et al, 2010; Lyon et al, 2012; Bleijenberg et al, 2017; Hermans et al, 2017; Hubner et al, 2017).

Daudelin et al (2010) specifically acknowledged that their results might have been influenced by the Hawthorne effect—the unknown consequences of being aware of the research on behaviour (Hughes, 2013). However, they argued that any quality improvement feedback loop would increase the focus on the specific clinical practice to which it was applied, in much the same way that knowledge of a research study might. Scholz et al (2008) argued that the benefits seen from their feedback system were achieved by increasing the ambulance clinician's understanding and their acceptance of the accountability of their role in the complete AMI care pathway. This practitioner engagement, facilitated by feedback, may be difficult to distinguish from the Hawthorne effect; however, if it persists, perhaps distinction is unnecessary. The time frames involved in the majority of these studies being in excess of a year would seem to support this argument.

Scott et al (2017) note that their interactive feedback meetings included identifying barriers to optimal performance that had been experienced (such as whether the equipment required was on the ambulance) and that efforts to remedy these issues may have accounted for some of the improvements recorded. Other studies included the delivery of educational sessions on the medical condition and treatment pathway involved and cannot, therefore, distinguish to what extent results are down to these interventions (Scholz et al, 2008; Daudelin et al, 2010; Lukas et al, 2012; Lyon et al, 2012; Siriwardena et al, 2014; Scott et al, 2017).

While the direct causality of results may be unclear, these studies demonstrate that an effective feedback mechanism includes meaningful interaction with clinicians. This includes maximising engagement, identifying barriers to implementation and facilitating clinical development based upon the gaps in knowledge exposed.

Barriers to effective feedback delivery

Difficulty in matching patient records across emergency medical service (EMS) and hospital systems was an emergent theme in this review.

In Germany and Rwanda, studies highlighted that research teams were unable to access hospital data to learn patient outcomes (Lukas et al, 2012; Scott et al, 2017). In the United States (US), Daudelin et al (2010) said a lack of infrastructure to link prehospital and hospital data was the primary reason that their feedback mechanism did not remain in place after the study was completed, despite impressive results. They suggested that continued development of electronic patient records, with interoperability between EMS and hospital systems, would facilitate future feedback mechanisms. However, certainly here in the UK, the development of compatible electronic record systems remains an aim rather than a widespread reality (Porter et al, 2018).

In addition to not having the resources to match patient data, many studies suggested that the demands of delivering feedback might be prohibitive for ambulance services. In both the Netherlands and the US, evaluation of resuscitation records taken from defibrillators and the subsequent delivery of individual debriefing and educational sessions in a timely manner were resource-draining activities in services already under significant operational pressures (Bleijenberg et al, 2017; Weston et al, 2017). In the UK, Siriwardena et al (2014) highlighted that many local quality improvement facilitators volunteered their time because ambulance trusts were unable to fund them.

Finally, the delivery of feedback information is not directly related to an individual patient's care, which raises legal and ethical issues (Schuman 2016; Eburn, 2017). Patient consent and maintaining patient confidentiality during delivery are important considerations in the initiation of any feedback system. Notably, in the UK, a policy of assumed patient consent to outcome information being shared has recently been sanctioned for a prehospital emergency medicine feedback project (PAH, 2018).

Facilitators of effective feedback

A feedback intervention is more effective when existing performance differs significantly from that desired (Ivers et al, 2012), and this seems intuitive. Papers with comparatively low baseline performance measures demonstrated significant improvements (Kramer-Johansen et al, 2006; Daudelin et al, 2010; Siriwardena et al, 2014), while two studies suggested that a high baseline performance explained the limited nature of improvements in their CPR performance measures (Hostler et al, 2011; Weston et al, 2017).

Ivers et al (2012) also concluded that feedback was most effective when delivered by colleagues or supervisors on more than one occasion, in more than one medium (e.g. verbally and written down) and when developmental targets were set. Siriwardena et al (2014) support these conclusions, remarking that the trusts that were most successful in implementing quality improvement measures for AMI and cerebral vascular accident care, provided printed aide-memoires, performance targets, reminder posters and pens, and educational sessions, on top of repeated individual and team feedback meetings facilitated by peer clinicians. These measures maximised engagement with the patient pathways involved. In Olasveengen et al's (2007) analysis of their failure to improve CPR performance, they cited a single method of feedback delivery, an absence of direct, interactive feedback to clinicians and a failure to engage practitioners as contributing factors.

As mentioned, some studies supplemented feedback with education to facilitate clinical development and some stressed the importance of interactive feedback to enable two-way discussion of barriers to optimal performance (Scholz et al, 2008; Daudelin et al, 2010; Lukas et al, 2012; Lyon et al, 2012; Siriwardena et al, 2014; Bleijenberg et al, 2017; Scott et al, 2017). An additional benefit of this direct contact with clinicians is a greater understanding of their knowledge and beliefs (Olasveengen et al, 2007; Bleijenberg et al, 2017). Olasveengen et al (2007) reported that, although clinicians were informed that chest compressions recorded were predominantly shallower than recommended, no improvement in compression depth was found during the study. Subsequent qualitative interviews revealed that clinicians believed that guideline compression depths were too deep and likely to cause injury. This highlights the value of understanding the clinicians' baseline attitudes when any feedback mechanism is being implemented.

Timeliness of feedback should be considered for effective delivery. Weston et al (2017) concluded that the benefits of real-time automated feedback from defibrillators could be augmented by the addition of feedback after an event, facilitating clinician reflection, once the immediacy and stress of the resuscitation had ended. Other studies discuss an optimal time window in which to provide feedback, where clinicians have had time to disengage from the demands of an incident but before they have lost the ability to recollect its details (Bleijenberg et al, 2017; Hubner et al, 2017).

Hubner et al (2017) also stressed the importance of providing both positive and negative feedback to maintain engagement with the process. It is important that a feedback system is not viewed as punitive and simply highlighting substandard performance (Hodell et al, 2016; Morrison et al, 2017).

Desire for feedback

A small amount of evidence from these studies suggests that feedback interventions are well received by ambulance clinicians.

Brinkrolf et al (2018) compared the attitudes of clinicians using real-time automated feedback defibrillators with those who did not have this equipment. They reported favourable attitudes towards the devices, with concerns about such systems appearing to resolve once they were actually employed. Pollard and Black (2015) reported that 100% of respondents to their questionnaire, following an 8-week period of receiving outcome feedback, wished to continue to receive this information. In addition, 90% confirmed that receiving feedback had prompted them to engage in clinical reflection, involving further reading or discussion with colleagues. The sample size (n=19) and potential for bias in this study, however, limit generalisability of these results.

Sommers et al (2017) presented an analysis of which patients for whom feedback was requested and the information sought. Most requests were for clarification of diagnosis and information about the patient's outcome. Interestingly, Pollard and Black (2015) recorded that 63% of their respondents recognised hospital diagnoses differed from their prehospital clinical impressions, which calls into question whether practitioners are best placed to know when and to whom feedback should be provided (Brichko et al, 2015). Providing outcome feedback on all incidents (Pollard and Black, 2015) or on patients where there was a diagnostic difference (Brichko et al, 2015) would potentially overcome this hurdle. However, the former may result in an information overload and the latter may result in feedback being perceived as predominantly negative.

The criteria that must be met for prehospital emergency medicine feedback reflect the perceived benefits of feedback for clinician wellbeing. In addition to cases of diagnostic uncertainty, requests can be submitted if the incident involved a critically unwell patient or if the incident had had a significant emotional impact on the ambulance clinician (PAH, 2018). The addition of patients with whom ambulance clinicians have become emotionally involved may, therefore, be a further consideration for effective outcome feedback mechanisms.