Evaluating automated rules for rapid response system alarm triggers in medical and surgical patients

Patients typically show signs and symptoms of deterioration hours to days prior to cardiorespiratory arrest.^1,2 The rate of inhospital cardiorespiratory arrest (CRA) requiring cardiopulmonary resuscitation is estimated to be 0.174 per bed per year in the United States.³ After CRA, survival to discharge is estimated to be as low as 18%.^3,4 Efforts to predict and prevent arrest could prove beneficial.^1,2

Rapid response systems (RRS) have been proposed as a means of identifying clinical deterioration and facilitating a timely response. These systems were designed to bring clinicians with critical care expertise to the bedside to prevent unnecessary deaths. They typically include an afferent limb (detects deteriorating patients), an efferent limb (responds to calls and acts to avoid further deterioration), and administrative and data analysis limbs.^5,6 Automatic provision of recommendations and computer-based systems are desirable components of the afferent limb of the detection system.⁶ Both are independent predictors of improved clinical practices for clinical decision support systems.⁷ However, the existing early warning scores (EWS) may not be ready for automation due to low positive predictive values (PPV) and sensitivities.⁸

It is possible that the low discriminatory accuracy of the published EWS may be secondary to the use of aggregate patient populations for derivation of scores. We hypothesized that these EWS perform differently in medical and in surgical subpopulations. Also, the EWS need to be tested in a time-dependent manner to serve as a realistic clinical support tool for hospitalized patients.

STUDY AIM

The aim of this study was to evaluate the differential performance of widely used EWS in medical vs surgical patients.

Site

The study was conducted in an academic center with 2 hospitals in Southeastern Minnesota totaling approximately 1500 general care nonintensive care unit (ICU) beds. The Mayo Clinic Institutional Review Board approved the research proposal.

Subjects

Our retrospective cohort was comprised of all adult inpatients discharged from 2 academic hospitals between January 1, 2011 and December 31, 2011 who spent any time in a general care (non-ICU) unit. We excluded patients younger than 18 years, psychiatric or rehabilitation inpatients, those without research authorization, and patients admitted for research purposes.

Study patients were divided into medical and surgical cohorts. Hospitalizations were considered surgical if patients had surgery at any time during their hospital stay according to billing data. A trigger was an instance in which a patient met the conditions of a specific rule (score/vital sign exceeded the published/defined threshold).

A resuscitation call was defined as a call for cardiopulmonary resuscitation when a patient has a CRA.

An event was an occurrence of 1 of the following in a general care setting: unplanned transfer to the ICU, resuscitation call, or RRS activation.

The RRS activation criteria consisted of an “acute and persistent change” in any 1 or more of the following: oxygen saturations less than 90%, heart rate less than 40 or greater than 130 beats/minute, systolic blood pressure less than 90 mm Hg, or respiratory rate less than 10 or greater than 28 breaths/minute. The RRS activation requires health provider action; they are not electronically generated. Nurses and physicians may also activate the RRS if they are concerned about a patient, even if calling criteria are not met. This is in contrast to the EWS analyzed, which are aggregate composites of multiple parameters. However, whether or not a derangement in vital signs is considered an “acute and persistent change” still involves clinical judgment. Any movement from a general care bed to an ICU bed, or from a general care bed to a procedure area, and from there to an ICU, was considered unplanned. Transfers to the ICU directly from the emergency department or operating room (OR) were not considered as an unplanned transfer and were not included in the analyses.

Coverage time was the period observed for events after a rule was triggered. In this analysis, a coverage time of 24 hours was considered, with a 1-hour look-back. A trigger was counted as a true positive if an event occurred during the following 24 hours. The 1-hour look-back was included to take into account the nursing clinical process of prioritizing a call to the RRS followed by documentation of the altered vital signs that prompted the call.

An episode was the continuous time on the general care floor within a hospitalization, excluding times when a patient was in the OR or ICU. For example, if a patient was admitted to a general bed on a surgery floor, subsequently went to the OR, and then returned to the surgery floor, the 2 episodes were considered separate: the time on the floor before surgery, and the time on the floor after surgery.

Assessment of implementation of RRS in our hospitals showed a significant drop in the failure-to-rescue rate (issues considered related to delay or failure to identify or intervene appropriately when a patient was deteriorating, as identified through mortality review) and a decrease in non-ICU mortality.^9,10 This suggests that our current process captures many of the relevant episodes of acute deterioration when a rapid response team is needed and supports using RRS activation as outcomes.