Modification of a children's hospital pediatric early warning score (EWS): An evaluation of inter-rater reliability, nurses' critical thinking and perceptions of the tool

Published:October 23, 2021DOI:


      • Newly modified Pediatric EWS tool revealed moderate agreement between nurses' scores.
      • Qualitative data showed greater consensus between nurses than the objective tool's scores.
      • Illustrates nurses' concerns of the ability of an objective tool to reflect a subjective assessment.


      Pediatric early warning scores (EWS) have been utilized to assist the identification of children at risk for clinically decompensating, experiencing a cardiac or respiratory arrest, or requiring a transfer to a higher level of care. Although their use is widespread, little consistency exists between tools and research evaluating the effectiveness of these tools is lacking. This quasi-experimental project evaluated twenty-five medical-surgical staff nurses' use and perceptions as well as the inter-rater reliability of a newly modified pediatric EWS tool at a free standing, academic Midwestern pediatric hospital. The tool was modified utilizing existing literature and an interdisciplinary team's expertise. Five fictionalized patients, presented in case studies, were developed and nurses were asked to score these patients using the newly modified tool with rationale. Inter-rater reliability was assessed utilizing Fleiss' Kappa and qualitative questionnaire data was analyzed for emerging themes. Overall, Fleiss' Kappa showed that there was moderate agreement between the nurses' judgments and scoring, with scores primarily differing due to the difficulty level of each case study. Nurses' responses to a questionnaire indicated differing levels of comfort identifying and managing children that present with mid-range total scores as opposed to those who scored in the lower or higher ranges. This project's findings highlight nurses' concerns that an objective tool may not accurately describe a subjective assessment. The results of this project indicated that use of this tool, with some modifications to address nursing concerns, may help to identify clinically decompensating pediatric patients being treated on medical-surgical units.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


        • Agulnik A.
        • Forbes P.W.
        • Stenquist N.
        • Rodriquez-Galindo C.
        • Kleinman M.
        Validation of a pediatric early warming score in hospitalized pediatric oncology and hematopoietic stem cell transplant patients.
        Pediatric Critical Care Medicine. 2016; 17: e146-e153
        • Bonafide C.P.
        • Localio A.R.
        • Roberts K.E.
        • Nadkarni V.M.
        • Weirich C.M.
        • Keren R.
        Impact of rapid response system implementation on critical deterioration events in children.
        JAMA Pediatrics. 2014; 168: 25-33
        • Bonafide C.P.
        • Localio A.R.
        • Song L.
        • Roberts K.E.
        • Nadkarni V.M.
        • Priestley M.
        • Keren R.
        Cost-benefit analysis of a medical emergency team in a children’s hospital.
        Pediatrics. 2014; 134: 235-241
        • Brady P.W.
        • Muething S.
        • Kotagal U.
        • Ashby M.
        • Gallagher R.
        • Hall D.
        • Wheeler D.S.
        Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events.
        Pediatrics. 2013; 131 ( 298-308
        • Bucholz E.M.
        • Toomey S.L.
        • Schuster M.A.
        Trends in pediatric hospitalizations and readmissions: 2010-2016.
        Pediatrics. 2019; 142: 1-10
        • Cassidy C.E.
        • MacEachern L.
        • Best S.
        • Foley L.
        • Rowe M.E.
        • Dugas K.
        • Mills J.L.A.
        Barriers and enablers to implementing the children’s hospital early warning score: A pre- and post-implementation qualitative descriptive study.
        Journal of Pediatric Nursing. 2019; 46 ( 39-47
        • Chapman S.M.
        • Wray J.
        • Oulton K.
        • Pagel C.
        • Ray S.
        • Peters M.J.
        ‘The score matters’: Wide variations in predictive performance of 18 paediatric track and trigger systems.
        Archives of Disease in Childhood. 2017; 102: 487-495
        • Children'’s Hospitals'’ Solutions for Patient Safety
        Network design and process.
        • Children'’s Hospitals'’ Solutions for Patient Safety
        Our goals.
        • Girotra S.
        • Spertus J.A.
        • Li Y.
        • Berg R.A.
        • Nadkarni V.M.
        • Chan P.S.
        Survival trends in pediatric in-hospital cardiac arrests: An analysis from get with the guidelines – Resuscitation.
        Circulation. Cardiovascular Quality and Outcomes. 2013; 6: 42-49
        • Hayes L.W.
        • Dobyns E.L.
        • DiGiovine B.
        • Brown A.M.
        • Jacobson S.
        • Randall K.H.
        • Markovitz B.
        A multicenter collaborative approach to reducing pediatric codes outside the ICU.
        Pediatrics. 2012; 129: e785-e791
        • Holmberg M.J.
        • Ross C.R.
        • Fitzmaurice G.M.
        • Chan P.S.
        • Duval-Arnould J.
        • Grossestreuer A.V.
        • Andersen L.W.
        Annual incidence of adult and pediatric in-hospital cardiac arrest in the United States.
        Circulation. Cardiovascular Quality and Outcomes. 2019; 12: 1-8
        • Hoppes M.
        • Mitchell J.L.
        • Venditti E.G.
        • Bunting R.F.
        White paper series: Serious safety events, getting to zero.
        American Society for Healthcare Risk Management, 2012
        • Humphreys S.
        • Totapally B.R.
        Rapid response team calls and unplanned transfers to the pediatric intensive care unit in a pediatric hospital.
        American Journal of Critical Care. 2016; 25: e9-e14
        • Hussain F.S.
        • Sosa T.
        • Ambroggio L.
        • Gallagher R.
        • Brady P.W.
        Emergency transfers: An important predictor of adverse outcomes in hospitalized children.
        Journal of Hospital Medicine. 2019; 14: 482-485
        • Institute for Healthcare Improvement
        Early warning systems: Scorecards that save lives.
        • Jensen C.S.
        • Nielsen P.B.
        • Olesen H.V.
        • Kirkegaard H.
        • Aagaard H.
        Pediatric early warning score systems, nurses perspective – A focus group study.
        Journal of Pediatric Nursing. 2018; 41 ( e16-e22
        • Krmpotic K.
        • Lobos A.T.
        Clinical profile of children requiring early unplanned admission to the PICU.
        Hospital Pediatrics. 2013; 3: 212-218
        • Laerd Statistics
        Fleiss' kappa using SPSS Statistics.
        (Statistical tutorials and software guides from)
        • Lambert V.
        • Matthews A.
        • MacDonell R.
        • Fitzsimons J.
        Paediatric early warning systems for detecting and responding to clinical deterioration in children: A systematic review.
        British Medical Journal Open. 2017; 7: 1-13
        • Maharaj R.
        • Raffaele I.
        • Wendon J.
        Rapid response systems: A systematic review and meta-analysis.
        Critical Care. 2015; 19: 254-269
        • Mansel K.O.
        • Chen S.C.
        • Mathews A.A.
        • Gothard M.D.
        • Bighan M.T.
        Here and gone: Rapid transfer from the general care floor to the PICU.
        Hospital Pediatrics. 2018; 8: 1-6
        • McLellan M.C.
        • Gauvreau K.
        • Connor J.A.
        Validation of the cardiac Children’s hospital early warning score: An early warning scoring tool to prevent cardiopulmonary arrests in children with heart disease.
        Congenital Heart Disease. 2014; 9: 194-202
        • McLellan M.C.
        • Gauvreau K.
        • Connor J.A.
        Validation of the children’s hospital early warning system for critical deterioration recognition.
        Journal of Pediatric Nursing. 2017; 32 ( 52-58
        • Miles A.H.
        • Spaeder M.C.
        • Stockwell D.C.
        Unplanned ICU transfers from inpatient units: Examining the prevalence and preventability of adverse events associated with ICU transfer in pediatrics.
        Journal of Pediatric Intensive Care. 2016; 5: 21-27
        • Murray J.S.
        • Williams L.A.
        • Pignataro S.
        • Volpe D.
        An integrative review of pediatric early warning systems scores.
        Pediatric Nursing. 2015; 41 (Retrieved April 1, 2020 from): 165-174
        • Nadeau N.
        • Monuteaux M.C.
        • Tripathi J.
        • Stack A.M.
        • Perron C.
        • Neuman M.I.
        Pediatric ICU transfers within 24 hours of admission from the emergency department: Rate of transfer, outcomes, and clinical characteristics.
        Hospital Pediatrics. 2019; 9: 393-397
        • Nadkarni V.M.
        • Larkin G.L.
        • Peberdy M.A.
        • Carey S.M.
        • Kaye W.
        • Mancini M.E.
        • A.
        First documented rhythm and clinical outcome from in-hospital cardiac arrest among children and adults.
        JAMA. 2006; 295 (Retrieved April 26, 2020 from:): 50-57
        • Parshuram C.S.
        • Dryden-Palmer K.
        • Farrell C.
        • Gottesman R.
        • Gray M.
        • Hutchison J.S.
        • Tomlinson G.A.
        Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: The EPOCH randomized clinical trial.
        JAMA. 2018; 319: 1002-1012
        • Parshuram C.S.
        • Hutchison J.
        • Middaugh K.
        Development and initial validation of the bedside Paediatric early warning system score.
        Critical Care. 2009; 13: R135
        • Reese J.
        • Deakyne S.J.
        • Blanchard A.
        • Bajaj L.
        Rate of preventable early unplanned intensive care unit transfer for direct admission and emergency department admissions.
        Hospital Pediatrics. 2015; 5: 27-34
        • Stevens K.R.
        The impact of evidence-based practice in nursing and the next big ideas.
        Online Journal of Issues in Nursing. 2013; 18 (Retrieved April 30, 2020 from)
        • Trubey R.
        • Huang C.
        • Lugg-Widger F.V.
        • Hood K.
        • Allen D.
        • Edwards D.
        • Powell C.
        Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalized children: A systemic review.
        British Medical Journal Open. 2019; 9: 1-22
        • Witt W.P.
        • Weis A.J.
        • Elixhauser A.
        Overview of hospital stays for children in the United States, 2012.
        Healthcare Cost and Utilization Project, 2014, December