How accurate is the prehospital diagnosis of hyperventilation syndrome?

Study population

Records of patients were included if they were aged ≥18 years and transported by the regional emergency ambulance service to the A&E departments of two neighbouring UK hospitals between 1 January 2012 and 31 December 2013. Exclusion criteria were private transport, direct admission to a hospital ward, transfer between facilities and paper prehospital records.

Index test and reference standard

The chosen index test was the diagnosis given to patients by prehospital clinicians documented in the ePRF by tick box or free text. The reference standard was the diagnosis given to patients by A&E doctors as documented within patients' medical notes. This reference standard was selected because it was deemed to be the closest option to the hypothetical gold standard answer, providing full certainty of whether a patient was presenting with HVS (Knottnerus et al, 2002).

Blinding

Local protocols required prehospital clinicians to have finalised their ePRFs before patients were assessed in hospital. This means that when making the initial diagnosis, they should not have been aware of patients' in-hospital diagnosis.

Although this sequence of events suggests that prehospital clinicians were blinded to the reference standard, the retrospective nature of this study did not allow a confidence procedure to be included to ensure compliance with the requirement to finalise ePRFs in a timely fashion, so the authors cannot report that blinding occurred. It is likely that hospital physicians would have consulted prehospital ePRFs, making them aware of the proposed prehospital diagnosis and so were not blinded.

Data were collected using separate data collection forms in two distinct phases to ensure that the researcher was blind to the patients' index test and the reference standard results while collecting data. This would have reduced the risk of test review bias and diagnostic review bias (Ransohoff and Feinstein, 1978; Philbrick et al, 1980).

Data analysis

Descriptive data were reported as numbers (per cent) for categorical variables and mean (standard deviation) for metric variables with normal distribution or median (minimum-maximum) with skewed distribution (Thabane and Akhtar-Danesh, 2008; Tuuli and Odibo et al, 2011).

To estimate diagnostic accuracy, sensitivity and specificity were measured, as well as positive/negative predictive values (PPV/NPV) and positive/negative likelihood ratios (LR+/LR-) calculated using the Stata command ‘diagt’ (Seed, 2010). Likelihood ratios were depicted visually using a Fagan nomogram (Fagan, 1975). Corresponding 95% confidence intervals (CI) were calculated to analyse the variability associated with each estimate (Lang and Secic, 2006).

The percentage and index test results of missing data were reported and a sensitivity analysis planned to explore whether diagnostic accuracy estimates would be altered if patients with missing data were assumed to be false negatives or false positives (Naaktgeboren et al, 2016).

Several subgroup analyses were planned in advance to prevent bias being introduced through post-hoc analyses. Statistically, these subgroup analyses were conducted using c² tests or Fisher's exact test. Results were deemed statistically significant at the 0.05 level, if P values were below P=0.05/8=0.00625. This modified P value was obtained using a Bonferroni correction to address the increased possibility of false-positive results associated with multiple hypothesis testing (Bland and Altman, 1995).

Sample size

Because the authors could not anticipate the sensitivity and specificity values, it was not possible to conduct a formal sample size calculation to ensure the study was sufficiently powered. For pragmatic reasons, a sample of 2 years (2012 and 2013) was chosen as ePRFs were later discontinued by the ambulance service in this locality.

Ethical considerations

To identify eligible patient records, two gatekeepers were appointed to identify and anonymise the records of patients diagnosed with HVS by the hospital and/or ambulance trust, which they matched using a pseudonymisation code in a process known as data linkage (Bohensky et al, 2010).

Summarised information on the patients not diagnosed with HVS in either setting was obtained in an anonymised form through the ambulance service's clinical governance department, as less in-depth data were required for these patients.

Using the above principles of data collection, the study was exempt from needing informed consent because the researcher only had access to data after they had been processed by the gatekeepers. The researcher therefore did not view or collect patient-identifiable information.

Furthermore, the principle of confidentiality was observed because re-identification of patients was not possible and the data were considered ‘de-identified information’ (House of Commons Health Committee, 2007: 88).

This resulted in a favourable ethical opinion from the University of Leeds School of Healthcare ethics committee (HREC15-021).

Study population

The median age of the 19 386 study patients was 58 years. The median age was 48 years for patients diagnosed with HVS prehospitally and 32 years for patients diagnosed with HVS in hospital (Table 1). There was unequal representation of men and women in the study sample, with women being more prevalent in all four categories at a statistically significant level (P=0.014).

Referral to the onsite primary care centre was most frequent for patients diagnosed in hospital with HVS. In contrast, being admitted to a hospital ward was most common in false-positive patients. Length of median stay was longer for false-positive patients than for true-positive patients.

Rates of reattendance at the A&E department within 28 days were similarly high across the groups.

In-hospital mortality was noted for two false-positive patients with in-hospital diagnoses of chronic obstructive pulmonary disease and pulmonary embolism, following hospital admissions for 4 and 5 days, respectively.

Patient characteristics

Overall, patients diagnosed with HVS prehospitally and in hospital presented most often with tachypnoea, feelings of anxiety, chest pain and shortness of breath; they also reported past medical problems of anxiety, mental health issues and respiratory disorders (Table 2). Vital signs were similar across the groups and at the high end of normal adult prehospital limits.

In the 224 false-positive patients, the most frequent hospital diagnoses were non-cardiac chest pain, acute coronary syndrome, chronic obstructive pulmonary disease and chest infection; three false-positive patients were diagnosed with a pulmonary embolism (Table 3).

For the 23 false-negative patients, common alternative prehospital diagnoses were other medical problem, other mental health problem and unknown problem, which indicated the difficulty and uncertainty surrounding patients' assessment and diagnosis.

Clinical assessments

Clinical assessments consisted of prehospital and in-hospital examinations and diagnostic tests that were conducted as part of routine practice but none were exclusive to diagnosing HVS (Figure 2).

Prehospitally, respiratory examinations were most commonly conducted. There was evidence in patients' records that almost one in four true-positive patients were coached in breathing, which was less prevalent among misdiagnosed patients. In hospital, patients diagnosed with HVS received fewer assessments than those with a false-positive diagnosis. Overall, obtaining venous blood samples and performing chest x-rays were the most frequent tests carried out.

Estimation of diagnostic accuracy

Overall, prehospital clinicians' diagnosis of HVS had a sensitivity of 88% (95% CI [82–92%]) and specificity of 99% (95% CI [99–99% (figures after decimal point within range have been rounded]) (Figure 3). This shows that prehospital clinicians correctly identified 88% of individuals who had HVS and 99% of patients who did not.

The calculated predictive values were PPV 0.42 (0.37, 0.47) and NPV 1.00 (1.00, 1.00). The NPV shows that if patients were diagnosed as not having HVS by prehospital clinicians, they were 100% likely not to be diagnosed with HVS in hospital. Although this value suggests absolute certainty, it should be noted that this measurement was subject to rounding and a small number of patients in this study were diagnosed with HVS in hospital, despite being diagnosed as not having HVS prehospitally. The PPV demonstrated that patients diagnosed with HVS prehospitally had a 42% chance of being diagnosed with HVS in hospital.

Likelihood ratios were 75.2 (65.3, 86.5) for a positive diagnosis and 0.12 (0.08, 0.18) for a negative diagnosis. The likelihood ratios were combined visually with prevalence of HVS in the study population (pr=0.96%) in a Fagan nomogram (Figure 4). The graph illustrates that, before they were assessed prehospitally, patients had a pre-test probability of 0.96% of being diagnosed with HVS in hospital. However, if a prehospital clinician diagnosed a patient with HVS, this probability increased to 39%. If a prehospital clinician deemed a patient to not have HVS, this probability decreased to close to 0%.

Subgroup analyses

The study sample size was too small to draw statistically significant conclusions regarding whether sensitivity varied according to the number of prehospital diagnoses, prehospital clinician qualification, patient age or patient sex (Table 4). Furthermore, it was not possible to determine specificity or NPVs across subgroups because diagnosis information was unavailable for true-negative patients. However, a statistically significant difference (P<0.001) was evident for PPV depending on the number of prehospital diagnoses and patient age. If a prehospital clinician solely selected a diagnosis of HVS, the patient had a probability of 80% of being diagnosed with HVS in hospital; however, if two or more diagnoses were documented, the patient's probability of being diagnosed with HVS in hospital was 20%. Overall, the probability that patients diagnosed with HVS prehospitally were diagnosed with HVS in hospital decreased with increasing patient age, with the exception of patients aged ≥70 years.

Missing data

As previously stated, data were missing for seven patients, which was 0.00036% of the total sample size. Conducting the pre-planned sensitivity analysis meant a decrease in sensitivity of 0.005 and a decrease in specificity of 0.0003, which were considered to be very small and not clinically relevant.

Patient demographics and outcomes

The study prevalence of HVS (pr=0.96%) was similar to the 0.3% found by Pfortmueller et al (2015) but considerably lower than the 6–11% cited in other literature (McKell and Sullivan, 1947; Rice, 1950; Singer, 1958; Yu et al, 1959; Thomas et al, 2005). However, the higher prevalence was found in patients with respiratory, cardiac and gastrointestinal problems and has been criticised as not accurately representing the wider population (Hornsveld and Garssen, 1996).

Patients who had an in-hospital diagnosis of HVS were on average 26 years younger than study patients overall, which reflects findings by Brashear (1983) and Pfortmueller et al (2015) that HVS is most common in people aged 20–40 years. Prehospitally, HVS was diagnosed 1.6 times as often in women than in men, which is slightly less than the 2–4 times reported in the literature (Perkin and Joseph, 1986; Thomas et al, 2005); however, equal sex incidence was reported by Lum (1975) and Saisch et al (1996).

Rates of hospital admission, length of stay and mortality for patients diagnosed in hospital with HVS were comparable to those found by Pfortmueller et al (2015), and support that HVS is not a life-threatening condition (Smith, 1985; Jones et al, 2013). The two cases of mortality were false positives, where HVS had been recorded as one of numerous prehospital diagnoses and treated prehospitally with oxygen-driven nebulisers. This suggests that paramedics were not convinced of the proposed HVS diagnoses as oxygen or nebuliser therapy are not indicated for HVS (Association of Ambulance Chief Executives (AACE), 2019).

Referrals to the primary care centre were made for almost one in five patients diagnosed with HVS, which is a new finding and not previously examined in the literature. It is supported by suggestions in the wider literature that invasive procedures and emergency treatments are not routinely required for patients with HVS (Smith, 1985; Jones et al, 2013).

Readmission rates (19–26%) were considerably higher than the 0.5% reported by Pfortmueller et al (2015), which suggests that A&E diagnoses were potentially incorrect or that patients with HVS were frequent A&E attenders, in line with suggestions by Harvison et al (2004).

Clinical characteristics, past medical history and vital signs

Tachypnoea was the most common sign observed in study patients, which is appropriate given that HVS is defined as ‘breathing in excess of metabolic requirements’ (Gardner, 2003: 7).

Chest pain was the most commonly reported symptom by false-positive patients, which mirrors reports in the wider literature (Lewis, 1953; Singer, 1958; Yu et al, 1959). Chest pain was reported far less in the true-positive and false-negative groups, which supports Gardner's (2004) hypothesis that chest tightness is not a symptom of HVS.

Breathlessness, dizziness and paraesthesia were reported in the study to a lesser extent than in the wider literature (McKell and Sullivan, 1947; Singer, 1958; Yu et al, 1959; Perkin and Joseph, 1986; Pfortmueller et al, 2015).

The variation in HVS symptoms between the study and the wider literature illustrated the highly variable frequency of HVS symptoms and the difficulty of diagnosing HVS (Rice, 1950; Singer, 1958; Yu et al, 1959).

On average, patients' vital signs were within normal adult prehospital limits but 13–16% of patients had observations that were outside acceptable parameters and triggered early warning scores. Although triggers were spread equally across the groups, they indicated different things: critical underlying pathology for false positives; and unresolved idiopathic hyperventilation episodes for true positives and false negatives. However, for prehospital clinicians, these indications were unclear when making a diagnosis. This underlines that paramedics should consider non-conveyance only if symptoms resolve (AACE, 2019).

Differential diagnoses

Of the most common diagnoses proposed in hospital, only acute coronary syndrome required emergency treatment; however; it could be argued that some non-cardiac chest pain requires invasive diagnostic tests.

Other diagnoses, such as chronic obstructive pulmonary disease and chest infections, can often be treated by primary care services (National Institute for Health and Care Excellence (NICE), 2010; 2014). In fact, referrals to specialist community teams are encouraged to reduce hospital admissions for patients with long-term conditions (Department of Health and Social Care (DHSC), 2005). Therefore, even for false positives, transport to A&E may not necessarily be in a patient's best interest.

Clinical assessments

The clinical assessments performed illustrated clinicians' attempts to exclude differential diagnoses using physical examinations and diagnostic tests; these are also found in the wider literature (Kerr et al, 1938; Smith, 1985; Harvison et al, 2004). An example was 12-lead ECGs, which were performed almost twice as often in false positives than in true positives. This could indicate that paramedics were unsure about their proposed HVS diagnosis and were conducting further tests to investigate cardiac differential diagnoses.

Limitations

A major limitation of this study was that not all eligible ePRFs were identified because of a fault with the automatic search string. A further limitation was the study's pioneering status, which meant that sample size and subgroup analyses were based on clinical experience rather than existing literature.

In addition, only patients transported to A&E departments with ePRFs were included. This would have excluded an unknown number of patients who may have refused transport, been discharged or referred by paramedics to community services, or for whom paper records were completed.

Finally, the choice of reference standard could be questioned in light of the unexpectedly high readmission rates which imply potentially incorrect diagnoses were made in A&E. This could be explored in further prospective research studies measuring hospital doctors' diagnostic accuracy of HVS, or simulated prehospital studies using an alternative reference standard.