Improved Safety Event Reporting in Outpatient, Nonacademic Practices with an Anonymous, Nonpunitive Approach

Journal of Clinical Outcomes Management. 2015 February;February 2015, VOL. 22, NO. 2:

February 9, 2015|Journal of Clinical Outcomes Management

Herbert W. Clegg II, MD;Terri Cardwell, RPh, PharmD, MHA;Aaron M. West, BS;Felicia Ferrell, BA

From Novant Health and Novant Health Medical Group, Winston-Salem, NC.

Abstract

Objective: To evaluate the effect of an educational intervention with regular audit and feedback on reporting of patient safety events in a nonacademic, community practice setting with an established reporting system.
Methods: A quasi-experimental with comparator design was used to compare a 6-practice collaborative group with a 27-practice comparator group with regard to safety event reporting rates. Baseline data were collected for a 12-month period followed by recruitment of 6 practices (3 family medicine, 2 pediatric, and 1 general surgery). An educational intervention was carried out with each, and this was followed by monthly audit and regular written and in-person feedback. Practice-level comparisons were made with specialty- and size-matched practices for the 6 practices in the collaborative group.
Results: In the 12-month period following the intervention in March 2013, the 6 practices reported 175 patient safety events compared with only 19 events in the previous 12-month period. Each practice at least doubled reporting rates, and 5 of the 6 significantly increased rates. In contrast, rates for comparator practices were unchanged, with 84 events reported for the pre-intervention period and 81 for the post-intervention period. Event classification and types of events reported were different in the collaborative practices compared with the comparators for the post-intervention period. For the collaborative group, near miss events predominated as did diagnostic testing and communication event types.
Conclusion: An initial educational session stressing anonymous, voluntary safety event reporting together with monthly audit and feedback and a focus on developing a nonpunitive environment can significantly enhance reporting of safety events.

Multiple challenges in the outpatient setting make establishing a culture of safety and improving care delivery more difficult than for inpatient settings. In the outpatient setting, care is often inaccessible, not well coordinated between providers and between facilities and providers, and delivered in many locations. It may also involve multiple sites and providers for a single patient, may require multiple visits in a single location, and can be provided by phone, email, mail, video, or in person [1]. Errors and adverse events may take long periods of time to become apparent and are more often errors of omission compared with those in the inpatient setting [2].

Incident reporting systems are considered important in improving patient safety [3], and their limitations and value have recently been reviewed [4]. However, limited research has been conducted on medical errors in ambulatory care, and even less is available on optimal monitoring and reporting strategies [5–12].Reporting in our system is time-consuming (about 15 minutes for entry of a single report), is not tailored for outpatient practices, may be considered potentially punitive (staff may believe that reporting may place themselves at risk for performance downgrade or other actions), and marked under-reporting of safety events was suspected. Most but not all of the suggested characteristics considered important for hospital-based reporting systems are fulfilled in our ambulatory reporting system [13].

Several academic groups have reported much improved reporting and a much better understanding of the types of errors occurring in their respective outpatient settings [14–16]. The most compelling model includes a voluntary, nonpunitive, anonymous reporting approach and a multidisciplinary practice-specific team to analyze reported errors and to enact change through a continuous quality improvement process [14,15].

We implemented a project to significantly improve reporting of safety events in an outpatient, nonacademic 6-practice collaborative by using education, monthly audit, and regular feedback.

Methods

Setting

Novant Health Medical Group is a consortium of over 380 clinic sites, nearly 1300 physicians, and over 500 advanced practice clinicians. Clinic locations are found in Virginia, North Carolina, and South Carolina. Medical group members partner with physicians and staff in 15 hospitals in these geographic locations. Novant Health utilizes Epic (Epic Systems, Verona, WI) as an electronic health record. Safety event reporting is accomplished electronically in a single software program (VIncident, Verge Solutions, Mt. Pleasant, SC), used for all patients in our integrated care system (inpatient and outpatient facilities).

Intervention

We designed a quasi-experimental study enrolling a 6-practice collaborative of 3 family medicine practices, 2 pediatric practices, and 1 general surgery practice. These practices was selected because each had a proven physician leader and an experienced practice manager willing to participate in this initiative. We developed a compendium of patient safety events (see Appendix) that had been reported over time in our safety event reporting program. Historically, reports were made electronically in the program by a single reporter in clinics, and these reports were initially verbally communicated by a staff member or provider to the reporter.

Two of the authors (HWC and TC) met in March 2013 with the lead physician, practice manager, and patient safety coach at each clinic for approximately 1 hour. We discussed current reporting practice, delivered education for the safety event compendium, and detailed an anonymous, voluntary, and nonpunitive approach (stressing the use of the term “safety event” and not “error”) to reporting using a single page, 8-question paper report about the event. The report was not to be signed by the person completing the event data with placement in a drop box for later collection and electronic reporting as per usual practice in the clinic. We agreed that clinic leaders would stress to staff and providers that the initiative was nonpunitive and anonymous and that the goal was to report all known safety events, as an improvement project.

Patient safety coaches were selected for each of the 6 practices by the manager. Patient safety coaches are volunteer clinical or nonclinical staff members whose role is to observe, model, and reinforce pre-established patient safety behaviors and use of error prevention tools among peers and providers. Training requirements include an initial 2-hour training session in which they learn fundamentals of patient safety science, high reliability principles, coaching techniques for team accountability, and concepts for continuous quality improvement. Additionally, they attend monthly meetings where patient safety concepts are discussed in greater detail and best practices are shared. Following this training, each clinic’s staff was educated on the project, a process improvement team (lead physician, manager, and patient safety coach) was constituted, and the project was begun in April 2013. In quarter 3 of 2013, each practice team selected a quality improvement project based upon reported safety events in their clinic. We asked our medical group risk managers to continue event discussion with practice managers as usual, as each event is discussed briefly after a report is made.

We audited reports monthly and provided feedback to the practice team with a written report at the end of each 3-month period starting in June 2013 and ending in June 2014 (5 reports). Individual on-site visits to meet and discuss progress were completed in September 2013 and March 2014, in addition to the initial visit in March 2013.

Evaluation

We compared reported monthly safety events for each of the 6 practices and for the 6-practice collaborative in the aggregate for the 12-month pre-intervention period (April 2012 through March 2013) and post-intervention period (April 2013 through March 2014). Each practice was compared with 3 specialty- and size (number of providers)-matched practices, none of whom received education or feedback on reporting or had patient safety coaches in the practice. In addition, for each of the 3 family medicine practices in the collaborative, we matched 1:3 other family medicine practices for specialty, size, and presence of a designated and trained patient safety coach. For the duration of the project, only 50 of 380 practices in the medical group had a trained patient safety coach.

The rate of safety events reported (ie, number of safety events reported/number of patient encounters) was compared for the 2 time periods using Poisson regression or zero-inflated Poisson regression. SASenterprise guide5.1 was used for all analyses. A P value of < 0.05 was considered statistically significant. The protocol was reviewed by the institutional review board of Novant Health Presbyterian Medical Center and a waiver for informed written consent was granted.

Results

For the year preceding the recruitment and education of the collaborative practices (pre-intervention year), reporting rates for the 6 collaborative practices (1.2 or 19 events reported/154,148 patient encounters) and for the aggregate of 27 comparators (1.5 or 84/568,417) were very similar (P = 0.47). For the post-intervention year, the collaborative practices’ rate increased to 11.5 (175/152,610, P < 0.001), while the rate for the comparator practices remained stable at 1.5 (81/554,608). Rates remained unchanged as well for all other Novant Health Medical Group practices (Figure 1).

Each of the 6 collaborative clinics experienced at least a doubling of reporting rates, and 5 of the 6 clinics significantly increased reporting rates (Figure 2). Practices 1 through 5 had pre-intervention rates of 0 to 2.4 and post-intervention rates of 6.0 to 8.2. Practice 6 increased from 5.9 to 164.6, an increase largely due to reporting of communication issues for this practice. In practice 1, a general surgery practice, reporting increased from 0 events (in 5093 encounters) to 4 events (in 5071 encounters, a rate of 7.9) for the 2 time periods. However, this increase did not reach the level of significance (P = 0.09). The 3 general surgery clinic comparators together reported 0 safety events for the post-intervention year among 13,793 encounters in clinic.

To control for the presence of patient safety coaches in practices, the 3 family medicine clinics (clinics 4 through 6, Figure 2) were each matched 1:3 for size (number of providers) and specialty (other family medicine clinics), also with a patient safety coach. While the rates were significantly increased for the 3 collaborative family medicine clinics (P < 0.001), only 1 of the comparators clinic’s rate changed significantly (0.2 or 1/44,580 to 1.3 or 6/45,157), and this change was marginally significant (P = 0.048). This practice was the only one of the 27 comparator clinics to demonstrate any increased rate.

We also compared the classification and types of safety events reported for each of 3 groups for the year April 2013 through March 2014 (Figures 3 and 4; see Appendix for definitions of safety event types and classifi-cation). In the collaborative group, 88% of events reported were near miss events and precursor safety events. Only 12% were non-safety events, as compared with 65% for comparator practices and 63% for all others. Two serious safety events were reported in the “all other practices” group. Too few events occurred in the pre-intervention collaborative group to make meaningful pre- and post-intervention comparisons.

A different pattern of event type was seen for the collaborative group. Falls accounted for only 6% and diagnostic testing (16%) and communications issues (34%) totaled 50%. In contrast, for comparator practices, falls accounted for 51% of reported safety events and only 3% of events were reported as diagnostic testing (2%) and communication issues (1%). This pattern accounts for the marked differences in event classification, as falls are usually classified as non-safety events.

Discussion

In our nonacademic community practices, patient safety reporting rates improved following an initial educational session stressing anonymous, voluntary safety event reporting together with monthly audit and feedback. Our findings corroborate those of others in academic ambulatory settings, who found that an emphasis on patient safety reporting, particularly if an anonymous approach is taken in a nonpunitive atmosphere, can significantly increase the reporting of patient safety events [14–16]. We demonstrated marked under-reporting and stability of patient safety event reporting throughout our ambulatory practice group and a 10-fold increase in reporting among the 6-practice collaborative.

An unexpected finding was that with the exception of 1 practice, we found no increased reporting in comparator practices that had a patient safety coach. Additionally, we noted that general surgery practices report (or experience) very few ambulatory safety events, as a total of 4 events were reported for all 4 general surgery practices in 18 months.

We chose a quasi-experimental with a comparison group and pre-test/post-test design since randomization of practices was not feasible [17]. We used a 2-year period to control for any seasonal trends and to allow time after the intervention to see if meaningful improvement in reporting over time would continue. We attempted to address the potential for nonequivalence in the comparison group by matching for specialty and size of practice.

There are several limitations to this study. Bias in the selection of collaborative practices may have occurred since each had a proven leader, and this may have led to more rapid adoption and utilization of this reporting approach. Also, our findings may not be generalizable to other integrated health systems given the unique approaches to patient safety culture development and the disparate nature of reporting systems. In addition, with our study design we could not be certain that anonymous reporting was a key factor in the increase in reporting rates, but de-briefing interviews indicated that both anonymous reporting and declaring a nonpunitive, supportive approach in each practice was important to enhanced reporting.

We expect increased reporting to decline over time without consistent feedback, as has been demonstrated in other studies [18], and we will continue to monitor rates over time.

As our current reporting system requires considerable reporter time for data input and discussion with risk managers, is not specifically configured for ambulatory reporting, is considered by staff and providers potentially punitive, and marked under-reporting is clear, we have proposed moving to a new system that is more user-friendly, ambulatory-focused, and has a provision for anonymous reporting.

Presented in part at the Institute for Healthcare Improvement 15th Annual International Summit on Improving Patient Care in the Office Practice and the Community, Washington DC, March 2014.

Acknowledgements: We gratefully acknowledge the work of collaborative practice team members, including Christopher Isenhour MD, Janet White, Shelby Carlyle, Mark Tillotson MD, Maria Migliaccio, Melanie Trapp, Jennifer Ochs, Gary DeRosa MD, Margarete Hinkle, Scott Wagner, Kelly Schetselaar, Timothy Eichenbrenner MD, Sandy Hite, Jamie Shelton, Raymond Swetenburg MD, James Lye MD, Kelly Morrison, Jan Rapisardo, Jane Moss, Rhett Brown MD, Dorothy Hedrick, Camille Farmer, and William Anderson, MS, for assistance with analysis.

Corresponding author: Herbert Clegg, MD, 108 Providence Road, Charlotte NC, 28207, hwclegg@novanthealth.org.

Financial disclosures: None.

References