Concerns have been raised about the UK child mortality rate, specifically in comparison to that in other European countries (Wolfe et al, 2014). Although infant mortality rates in England and Wales have generally declined since the 1980s, the most recent 2021 data show rates have increased since 2020 (Office for National Statistics, 2023). Furthermore, in recent literature, concerns have been expressed about poor confidence and knowledge regarding paediatric assessment, often by staff themselves, and attributed to the low incidence of these patient contacts (Butler and Adefuye, 2019; Jeruzal et al, 2019; Hetherington and Jones, 2021).
Paediatric early warning scores (PEWS) are commonly implemented as a patient safety measure to improve the quality of care for children (Sefton et al, 2015; Corfield et al, 2018; de Groot et al, 2018; Sambeeck et al, 2018). Many variations exist in practice and, despite the increasing use of PEWS, they are frequently not validated (Chapman et al, 2017; de Groot et al, 2018; Sambeeck et al, 2018), particularly in the prehospital setting (Roland et al, 2017). However, work is under way in England to develop a standardised system (Royal College of Paediatrics and Child Health (RCPCH), 2023).
Authors have repeatedly cautioned that PEWS are assessment tools to be used in conjunction with and not instead of clinical judgement (Roland et al, 2016; Sandell and Maconochie, 2016; Jensen et al, 2018). Therefore, definitive evidence of excellent specificity and sensitivity is unlikely, as the tools are not designed to be the only factor considered. Consequently, outcome measures of PEWS should be examined. Despite previously declaring that PEWS was a good predictor of adverse paediatric outcomes (Corfield et al, 2018), Corfield et al (2020) found prehospital PEWS unhelpful in predicting hospital admission. They attributed this to the fact that more than physical observations needed to be considered in such decisions.
Notably, the evidence supporting PEWS is often limited because vital signs are missing and scores are incomplete (Corfield et al, 2018; Chapman et al, 2019; Jensen et al, 2019). This issue is most clearly demonstrated by Corfield et al (2018), where a sample of 102 993 was reduced to 21 202 because values were missing. If PEWS compliance is poor or records are inaccurate, it is difficult to fully assess the effectiveness of PEWSs and their benefits.
The retrospective nature of many studies is a major drawback as it is impossible to explore whether these are errors and omissions within clinical assessment or poor documentation. Cotterill et al (2016) predict issues with staff apathy because poorly sensitive PEWS identify more children than just those seriously ill, which suggest a more specific tool is needed for emergency department (ED) use.
In a prehospital setting, children account for a smaller proportion of emergency calls than other age groups (Drayna et al, 2015). This suggests that inexperience could negatively affect the quality of assessment and management of children in a prehospital setting. Partially because of motivation to address concerns about health professionals failing to identify the most unwell children (Wolfe et al, 2014), the paediatric observation priority score (POPS) was designed to reduce unnecessary admissions and assist in identifying the most unwell children (Roland et al, 2014). The RCPCH (2018) advocate the use of PEWS and recognise POPS as the most validated and therefore appropriate for use in emergency care settings.
POPS was designed specifically for use in emergency care and reliability in this environment and suitability for wider use have been demonstrated (Roland et al 2017; Langton et al, 2018; Bonfield and Roland, 2020). POPS has already been implemented in the prehospital setting without validation in this environment (Studnek et al, 2019).
POPS includes measurement of a child's level of consciousness, heart rate, respiratory rate, oxygen saturations, temperature (scored in the context of their age) alongside consideration of any adverse breathing signs (e.g. wheeze), the clinician's level of concern and patient risk factors (e.g. premature birth). Significantly, POPS includes the consideration of ‘gut feeling’ and patient risk factors, incorporating elements of clinical judgement (Roland et al, 2016). An example of a POPS chart is shown in Figure 1 (Roland et al, 2014).
The system provides a score of between 0 and 16, with higher scores associated with more seriously unwell patients; some patients may always score at least 1 because of pre-existing risk factors. Roland et al (2014) advise that scores of ≥8 require immediate review and should be considered for transfer to the resuscitation area. POPS is available as a mobile application for automated calculation.
Cotterill et al (2016), in their cross-sectional study (using a sample of 2068 children in an ED), identified that POPS performed better (receiver operating characteristic (ROC) 0.72 versus 0.67) than a similar scoring tool for pat ients aged under 16 years, although they emphasised that multicentre validation was required. Teheux et al (2019) also supported the consideration of risk, demonstrating it significantly improved the sensitivity of a score (0.70 versus 0.30; P<0.001). While this was also limited as a single-centre study, the growing evidence is encouraging.
Many authors recognise that there is a significant gap in prehospital PEWS research and agree that further, in-depth research is required (Cotterill et al, 2016; Lillitos et al, 2016; Jensen et al, 2019). This study intends to address this gap and examine the use and documentation of POPS by staff in one region served by an NHS ambulance trust.
Research question
How is POPS currently used in a prehospital setting within one Welsh health board?
Methods
The study had a cross-sectional design. Data were collected directly from patient clinical records (PCRs). The ambulance service had begun to implement POPS in 2018. The study sample consisted of children attended by staff working at one Welsh health board between 1 December 2020 and 28 February 2021. The inclusion criteria were children aged between birth and 16 years. Children were excluded where POPS was not required or was inappropriate, i.e. where assessment was not documented on a digital PCR, inter-hospital transfers, children transported via urgent care services only, cardiac arrest or other time-critical scenarios, newborns (Apgar score more appropriate) and acute mental health presentations.
A convenience sample was used. Data were collected retrospectively from all PCRs where a child was attended by an emergency medical technician (EMT), a newly qualified paramedic (NQP; one within two years of qualification), a paramedic or an advanced paramedic practitioner. Data were recorded directly from PCRs into a database by the researcher and verified by a second person.
Data were analysed using SPSS statistical software (v. 26). The chi-square test was used to analyse categorical data, with Fisher's exact test used for any cases where the assumption for the chi-square test was not met (minimum expected cell frequency ≥5).
Neither of the continuous variables (age and documentation of POPS) were normally distributed (Kolmogorov-Smirnov and Shapiro-Wilk tests, sig. 0.000), so non-parametric tests were used: the Mann-Whitney U test, when comparing two independent groups, and the Kruskal-Wallis test when three or more independent groups were compared. Relationships were considered statistically significant where P≤0.05.
Results
The sample included 121 children of whom 53% (n=64) were male and 47% (n=57) female. Their age was in a range of 0–15 years (Figure 2). POPS was used most commonly in children aged <1 year (41.3%; n=12/29). However, this difference was not statistically significant (Mann-Whitney U=1532.5; P=0.709). Almost 97% (n=32/33) of 7–15 year-olds had all the required observations documented, while younger children were more likely to have observations missing (Mann-Whitney U=796; P=0.003). Generally, blood pressure (BP) was more frequently taken for older children; those aged ≥11 years had BP recorded 86.3% (n=19/22) of the time, while children aged ≤6 years were had it recorded in only 11.4% (n=10/88) of cases (Mann-Whitney U=469.5; P=0.000).
To calculate the POPS, the following observations are required: respiratory rate; oxygen saturations; heart rate; and temperature. POPS was documented in the PCR in 32.2% (n=39/121) of cases. Scores were in a range of 0–11 (on a scale of 0–16) and 74.4% (n=29/39) of POPS were in the range of 0–3, with 12.8% (n=5/39) ≥7. The mean score was 2.7 (SD 2.7). This should be recorded alongside the medical history and assessment information.
The frequency of calls throughout the day was spread fairly evenly (Table 1), but the majority of calls were during the evening (35.5%; n=43/121). Compliance with recording POPS varied between 27.3% (n=6/22) and 47.6% (n=10/21), with POPS most frequently documented between 00:00 and 06:00 hours; however, this difference was not statistically significant (Kruskal-Wallis H=1.666; P=0.644). Time of day did not affect the accuracy of the POPS recordings (Fisher's exact test=2.431; two-sided significance=0.539).
| Morning (06:00–12:00) | Afternoon (12:00–18:00) | Evening (18:00–00:00) | Night (00:00–06:00) | Total | ||
|---|---|---|---|---|---|---|
| POPS used | Yes | 6 (27.3%) | 9 (25.7%) | 14 (32.6%) | 10 (47.6%) | 39 (32.2%) |
| No | 16 (72.7%) | 26 (74.3%) | 29 (67.4%) | 11 (52.4%) | 82 (67.8%) | |
| Total | 22 (100%) | 35 (100%) | 43 (100%) | 21 (100%) | 121 (100%) | |
Kruskal-Wallis H=1.666; P=0.644. Note: Although 39 POPS were recorded, 38 was used as the accuracy total owing to missing observation information on the paperwork, making it impossible to calculate the accuracy of one score.
POPS was documented in only 32.2% (n=39/121) of cases, but the required observations were recorded for 77.7% of children (n=94/121). BP is not required to calculate POPS but was documented in 30.6% (n=37/121) of cases. When documented, POPS was accurately calculated 84.2% (n=32/38) of the time.
Of the 121 children, the highest proportion were attended by paramedics (50.4%; n=61/121). NQPs used POPS most frequently (52.6%; n=10/19) and were the only staff grade to use POPS more often than not. NQPs were most likely to have completed the necessary observations (89.5%; n=17/19), closely followed by paramedics (83.6%; n=51/61). EMTs had the poorest compliance with completing observations (60.6%; n=20/33) (Fisher's exact test=7.756; two-sided significance=0.043). Advanced paramedic practitioners had the poorest compliance with POPS use (12.5%; n=1/8) but also attended the lowest number of children (6.6%; n=8/121). However, no statistically significant difference was found (Fisher's exact test=5.562; two-sided significance=0.107), suggesting that staff qualification does not influence the use of POPS (Table 2).
| Staff grade/qualification | |||||||
|---|---|---|---|---|---|---|---|
| Emergency medical technician | Newly qualified paramedic | Paramedic | Advanced paramedic practitioner | Test | Sig. | ||
| POPS used | Yes | 8 (24.2%) | 10 (52.6%) | 20 (32.8%) | 1 (12.5%) | Fisher's exact | 107 |
| No | 25 (75.8%) | 9 (47.4%) | 41 (67.2%) | 7 (87.5%) | |||
| Outcome | Hospital | 27 (81.8%) | 17 (94.4%) | 47 (79.7%) | 6 (75%) | Chi-square | 495 |
| Discharged | 6 (18.2%) | 1 (5.6%) | 12 (20.3%) | 2 (25%) | |||
| POPS accuracy | Yes | 6 (75%) | 8 (80%) | 17 (89.5%) | 1 (100%) | Fisher's exact | 663 |
| No | 2 (25%) | 2 (20%) | 2 (10.5%) | 0 | |||
| Observations required for POPS | Yes | 20 (60.6%) | 17 (89.5%) | 51 (83.6%) | 6 (75%) | Fisher's exact test | 043 |
| No | 13 (39.4%) | 2 (10.5%) | 10 (16.4%) | 2 (25%) | |||
Note: Although 39 POPS were recorded, 38 was used as the accuracy total owing to missing observation information on the paperwork, making it impossible to calculate the accuracy of one score.
Regarding outcomes, 70.2% (n=85/121) of children were transported by ambulance to hospital, with a further 9.9% (n=12/121) attending hospital via alternative transport. Of the remaining children, 12.4% (n=15/121) were discharged with no further action and 5% (n=6/121) were referred to their GP, out-of-hours service or another healthcare service (Figure 3). Patient outcomes were not affected by staff qualification (Mann-Whitney U=2.391; P=0.495), nor was accuracy in calculating POPS (Fisher's exact test=1.951; two-sided significance=0.663) (Table 2). All children where POPS >3 were transported to hospital, while 50% (n=3/6) of children with a POPS of 0 were discharged and left at home in the care of their parents (Kruskal-Wallis H=6.771; P=0.009).
Discussion
POPS was documented in 32.3% (n=39/121) of children in this study. This is similar to the 36% compliance achieved in a previous audit in another Welsh health board (Welsh Ambulance Service Trust, unpublished POPS audit, 2020). While other authors (Almblad et al, 2018; de Groot et al, 2018) have observed compliance rates >65% with PEWS, direct comparison with this study is difficult as this work was conducted outside the UK.
However, despite the relatively low POPS compliance, 77.7% (n=94/121) of children had a complete set of observations excluding BP. Drayna et al (2015) suggest that missing observations, whether this is because an assessment was not carried out or not documented, is a key issue in prehospital paediatric care. Roland et al (2021) specifically investigated POPS, albeit within an ED setting, and found that 22% of children had at least one missing observation. They identified that more investigation was required into whether missing values were justified or reflected substandard care. Nonetheless, POPS appears to have better observation compliance than other PEWS (Corfield et al, 2018; Chapman et al, 2019; Jensen et al, 2019), which is a clear positive.
It was noticeable that the likelihood of recording all observations increased with age. This could indicate potential difficulties with confidence and competence regarding the assessment of younger children as they were less likely to have all observations completed. A qualitative study of nurses found they had concerns regarding a lack of suitable equipment and that children would not cooperate, particularly with reference to BP measurement (Jensen et al, 2018). Corfield et al (2020) found the poorest observation compliance was with BP (36%), particularly in younger children, with concerns that such low compliance impaired the performance of the PEWS and therefore raised questions over its inclusion. It may therefore be a strength of POPS that BP is not required, although there is a lack of prehospital specific research concerning the necessity of BP measurement in children.
The accuracy in calculating POPS in this study was excellent (84.2%; n=32/38). While this compares well with the findings of other papers, for example, 35.9% (Chapman et al, 2019) and 71% (Dean et al, 2020) accuracy in calculating PEWS, neither are direct comparisons with the accuracy using POPS. No correlation was found between the level of staff qualification and completion of POPS.
In this study, 74.4% (n=29/39) of POPS scores were in a range of 0–3, with a mean of 2.7 (SD 2.7). Previous studies specific to POPS reported a mean score of 2.33 and 75%–88% of scores in the 0-4 range (Roland et al, 2016; 2017). This corresponds with existing literature that demonstrates the majority of children are low scoring and the number of seriously ill children is minimal (Roland et al, 2016; 2017; Vredebregt et al, 2018; Heal et al, 2021). This reflects that clinicians have minimal exposure to children, especially those who are critically unwell (Cushman et al, 2010; Brown et al, 2016); if confidence and competence are negatively affected as a result, this could be linked to missing observations or poor use of PEWS.
Over 80% (n=97/121) of children attended hospital (Figure 3). Data analysis found that POPS value was significantly linked to patient outcome (Kruskal-Wallis H=6.771; P=0.009). All (100%; n=16) children with POPS >3 were transported to hospital and 50% (n=3) of those scoring 0 were discharged. It is difficult to draw comparisons between this and the existing literature because of the gaps in this surrounding prehospital care. Nevertheless, this supports the proposal of POPS as a tool to detect seriously ill children and also assist in the appropriate discharge of children who are well (Roland et al, 2016; 2017).
Strengths and limitations
The sample was limited by several practical aspects. At the time of this study, staff used both digital pens and carbon-copy PCRs, which can be affected by technical and practical issues. Moreover, before being made available for this study, all PCRs had to be validated and anonymised by the trust's clinical audit department. Electronic PCRs have since been introduced, potentially reducing limitations for future studies. Furthermore, electronic PCRs can automatically calculate POPS if the required observations are recorded.
The documentation of POPS was determined via screening of the PCR, as there is no specific box to record POPS. It is possible that POPS was calculated more often than was identified through this. Handwriting legibility complicated data collection and assessment of POPS accuracy, as scoring factors such as breathing and risk factors were documented in the narrative. Accuracy was also reliant on subjective ‘gut feeling’ scores which the retrospective method made difficult to ascertain.
A key strength of this study is that it addresses a prehospital literature gap. This study will be beneficial in informing the ambulance trust of staff practice and providing recommendations on how improvements can be made.
Conclusion
In summary, although the use of POPS was relatively low in this study, predominantly children did have relevant observations recorded and, when POPS was used, it was accurate. Missing observations correlated directly with younger age, and further research should identify what issues exist and investigate them.
While these results contradict much of the literature regarding reporting limitations, this may primarily be because BP was absent from the chosen PEWS. Future research could investigate the relevance of BP to children's assessment, why it is less commonly measured and whether it should be included in a standardised PEWS.
Based on these initial results, compliance with POPS could be improved. The ambulance trust could facilitate this with targeted learning regarding POPS (University of Leicester, n.d.), and also focus on specific paediatric education. A POPS e-learning module has already been adapted in conjunction with the local paediatric ED consultant and is available to staff.
This ambulance trust has since introduced electronic patient records with automatic calculation of POPS if the information and observations are provided.
Results from this study indicate that compliance in recording the required observations was high, which means that POPS will be calculated for the majority of sick children.