The Need for Standardized Metrics to Drive Decision-making During the COVID-19 Pandemic

The rapid onset of the novel coronavirus disease 2019 (COVID-19) pandemic forced the US healthcare system to scramble to prepare for a health crisis with many unknowns. Early on, it was unclear exactly how the virus was transmitted, how many people would fall ill or how ill they would get, what treatments would be most efficacious, and what resources were needed to care for patients.¹ Given the short window the healthcare system had to prepare, many initial and important decisions were made quickly and often at a local level, with limited coordination and standardization across localities and organizations. These decisions included what services could be offered, how best to allocate potentially scarce resources (such as personal protective equipment and ventilators), and how much surge capacity to build.^2,3 In short, many of the early decisions about the pandemic were understandably varied, and the lack of standardized metrics to help guide decision-making did not help the situation.

Unfortunately, as the COVID-19 pandemic continues, there has been insufficient movement toward standardizing definitions for many key measures needed to manage the public health response. Even small differences in definitions can have important implications for decision-making.⁴ For example, public health officials have recommended communities achieve a positivity rate of 5% or lower for 14 straight days before easing virus-related restrictions.⁵ In Maryland, two different entities are calculating positivity rates for the state using different methodologies and producing different results, which can have significant public health and economic implications for the state. Johns Hopkins University’s Resource Center calculates the positivity rate by comparing the number of people who tested positive for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) to all people who were tested. This method consistently produces a positivity rate for Maryland above the 5% threshold. In contrast, the state of Maryland calculates the positivity rate by comparing the number of positive tests for SARS-CoV-2 to the number of tests conducted, even if the same person had multiple tests (unless the tests are performed the same day at the same location). This method consistently produces a positivity rate for Maryland below the 5% threshold.⁶

The lack of standardized definitions leads not only to debate and confusion over what steps to take next, but also opens the door to politicization of pandemic data. This is readily apparent when considering mortality due to COVID-19. For example, different states use different definitions for COVID-19 mortality. Alabama defines COVID-19 mortality by only including patients who tested positive for the SARS-CoV-2 virus and the cause of death was attributed to COVID-19. In contrast, Colorado’s COVID-19 mortality definition includes those patients who are believed to have died of COVID-19, but does not require confirmation of SARS-CoV-2 infection by a positive test.⁷ Further compounding the challenge, some politicians reference the COVID-19 mortality rate as a comparison of those who died from COVID-19 with those who were sick with COVID-19, reflecting the success rate of treating patients with COVID-19, an area in which the United States has done relatively well compared with other countries. This definition of the mortality rate suits a narrative of successful pandemic management.⁸ However, many public health officials suggest the COVID-19 mortality rate should be defined by comparing the number of deaths from COVID-19 as a percentage of the population, which reflects the percentage of the population dying from the disease. In this regard, the United States has not done as well relative to other countries.⁹ These different definitions highlight how the United States lacks a standardized way to compare its performance across states and with other countries, even on a straightforward measure like mortality.