Trends in COVID-19 Risk-Adjusted Mortality Rates

Early reports showed high mortality from coronavirus disease 2019 (COVID-19), while current United States data mortality rates are lower, raising hope that new treatments and management strategies have improved outcomes. For instance, Centers for Disease Control and Prevention data show that 6.7% of cases resulted in death in April, compared with 1.9% in September.¹ However, the demographics of those infected have also changed, and more available testing may mean more comprehensive identification and earlier treatment. Nationally, for instance, the median age of confirmed cases was 38 years at the end of August, down from 46 years at the start of May.² Therefore, whether decreasing COVID-19 mortality rates simply reflect changing demographics or represent actual improvements in clinical care is unknown. The objective of this analysis was to assess outcomes over time in a single health system, accounting for changes in demographics, clinical factors, and severity of disease at presentation.

We analyzed monthly mortality rates for admissions between March 1 and August 31, 2020, in a single health system in New York City. Outcomes were obtained as of October 8, 2020. We included all hospitalizations of people 18 years and older with laboratory-confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection identified during the hospitalization or in the prior 2 weeks, excluding those admitted to hospice care. Patients with multiple hospitalizations (N=208 patients, 229 hospitalizations, 4.4%) were included repeatedly if they continued to have laboratory-confirmed disease. Patients without admission vital signs (N=28) were excluded. Mortality was defined as in-hospital death or discharge to hospice care. In-house laboratory testing began March 16 and all inpatients were tested for SARS-CoV-2 by April 1; elective surgeries resumed May 4-11 and were only conducted on confirmed SARS-CoV-2–negative patients.

All data were obtained from the electronic health record (Epic Systems, Verona, Wisconsin). Diagnosis codes were obtained from the problem list, past medical history, and billing codes. In addition, we used objective data such as hemoglobin A1c, ejection fraction, outpatient creatinine, and outpatient blood pressure to augment problem list diagnoses where relevant.

Based on prior literature, we constructed multivariable logistic regression models for mortality adjusting for age; sex; self-reported race and ethnicity; body mass index; smoking history; presence of hypertension, heart failure, hyperlipidemia, coronary artery disease, diabetes, cancer, chronic kidney disease, dementia, or pulmonary disease individually as dummy variables; and admission oxygen saturation, D-dimer, ferritin, and C-reactive protein.^3-6 In the first model (C statistic 0.82), we did not include month of admission as a covariate and calculated the ratio of the sum of observed and expected deaths (obtained from the model) in each month to obtain the standardized mortality ratio (SMR) for each month. We then multiplied each period’s SMR by the overall average crude mortality to generate monthly adjusted mortality rates. We calculated Poisson control limits and indicated points outside the control limits as significantly different.

In a second model (C statistic 0.84), we included month as a covariate and calculated average marginal effects (AME) for each time period by using the margins library in R,⁷ which uses a discrete first-difference in predicted outcomes to obtain the AME. The average marginal effect represents the percentage point difference between the reference period (March) and a subsequent time period in probability of death or discharge to hospice, for equivalent patients. We obtained lower and upper confidence intervals for the AME using a bootstrapping approach described in Green.⁸ Finally, we conducted two sensitivity analyses: one, restricting the analysis to only those patients with principal diagnosis of COVID-19, sepsis, or respiratory disease (see Appendix A for complete list of codes) and one restricting the analysis to only those with length of stay of at least 3 days.

All statistical analyses were conducted with R, version 4.0.2. All analyses used 2-sided statistical tests, and we considered a P value < .05 to be statistically significant without adjustment for multiple testing. The NYU institutional review board approved the study and granted a waiver of consent and a waiver of the Health Information Portability and Accountability Act.

We included 5,121 hospitalizations, of which 5,118 (99.94%) had known outcomes (death or hospital discharge). Peak hospitalizations occurred in late March to mid-April, which accounted for 53% of the hospitalizations. Median length of stay for patients who died or were discharged to hospice was 8 days (interquartile range, 4-15; max 140 days). The median age and the proportion male or with any comorbidity decreased over time (Table). For instance, the proportion with any chronic condition decreased from 81% in March to 72% in August.

horwitz11661023e_t1.jpg

Adjusted mortality dropped each month, from 25.6% in March to 7.6% in August (Table and Figure). The SMR declined progressively over time, from 1.26 (95% CI, 1.15-1.39) in March to 0.38 (95% CI, 0.12-0.88) in August (Table). The adjusted average marginal effect was also significantly lower than in March in every subsequent month, reaching a maximum of an average 18.2 (95% CI, 12.0-24.4) percentage point decrease in probability of death in August, accounting for changes in demographics and clinical severity (Table and Appendix B). The decrease in unadjusted mortality over time was observed across age groups (Appendix C).

horwitz11661023e_f1.jpg

Results of the two sensitivity analyses were similar (Appendices D and E), though attenuated in the case of the sepsis/respiratory cohort, with adjusted mortality falling from 31.4% to 14.4%, SMR decreasing from 1.28 (95% CI, 1.16-1.41) to 0.59 (95% CI, 0.16-1.50), and AME in August 17.0 percentage points (95% CI, 6.0-28.1).

In this study of COVID-19 mortality over 6 months at a single health system, we found that changes in demographics and severity of illness at presentation did not fully explain decreases in mortality seen over time. Even after risk adjustment for a variety of clinical and demographic factors, including severity of illness at presentation, mortality was significantly and progressively lower over the course of the study period.

Similar risk-adjusted results have been preliminarily reported among intensive care unit patients in a preprint from the United Kingdom.⁹ Incremental improvements in outcomes are likely a combination of increasing clinical experience, decreasing hospital volume, growing use of new pharmacologic treatments (such as systemic corticosteroids,¹⁰ remdesivir,¹¹ and anticytokine treatments), nonpharmacologic treatments (such as placing the patient in the prone position, or proning, rather than on their back), earlier intervention, community awareness, and, potentially, lower viral load exposure from increased mask wearing and social distancing.¹²

Strengths of this study include highly detailed electronic health record data on hospitalizations at three different hospitals, a diverse patient population,⁶ near-complete study outcomes, and a lengthy period of investigation of 6 months. However, this study does have limitations. All patients were from a single geographic region and treated within a single health system, though restricting data to one system reduces institution-level variability and allows us to assess how care may have evolved with growing experience. Aggregating data from numerous health systems that might be at different stages of local outbreaks, provide different quality of care, and contribute different numbers of patients in each period introduces its own biases. We were also unable to disentangle different potential explanatory factors given the observational nature of the study. Residual confounding, such as a higher proportion of particularly frail patients admitted in earlier periods, is also a possibility, though the fact that we observed declines across all age groups mitigates this concern. Thresholds for hospital admission may also have changed over time with less severely ill patients being admitted in the later time periods. While changing admission thresholds could have contributed to higher survival rates in the latter portions of the study, our inclusion of several highly predictive clinical and laboratory results likely captured many aspects of disease severity.

In summary, data from one health system suggest that COVID-19 remains a serious disease for high-risk patients, but that mortality rates are improving over time.

Early reports showed high mortality from coronavirus disease 2019 (COVID-19), while current United States data mortality rates are lower, raising hope that new treatments and management strategies have improved outcomes. For instance, Centers for Disease Control and Prevention data show that 6.7% of cases resulted in death in April, compared with 1.9% in September.¹ However, the demographics of those infected have also changed, and more available testing may mean more comprehensive identification and earlier treatment. Nationally, for instance, the median age of confirmed cases was 38 years at the end of August, down from 46 years at the start of May.² Therefore, whether decreasing COVID-19 mortality rates simply reflect changing demographics or represent actual improvements in clinical care is unknown. The objective of this analysis was to assess outcomes over time in a single health system, accounting for changes in demographics, clinical factors, and severity of disease at presentation.

We analyzed monthly mortality rates for admissions between March 1 and August 31, 2020, in a single health system in New York City. Outcomes were obtained as of October 8, 2020. We included all hospitalizations of people 18 years and older with laboratory-confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection identified during the hospitalization or in the prior 2 weeks, excluding those admitted to hospice care. Patients with multiple hospitalizations (N=208 patients, 229 hospitalizations, 4.4%) were included repeatedly if they continued to have laboratory-confirmed disease. Patients without admission vital signs (N=28) were excluded. Mortality was defined as in-hospital death or discharge to hospice care. In-house laboratory testing began March 16 and all inpatients were tested for SARS-CoV-2 by April 1; elective surgeries resumed May 4-11 and were only conducted on confirmed SARS-CoV-2–negative patients.

All data were obtained from the electronic health record (Epic Systems, Verona, Wisconsin). Diagnosis codes were obtained from the problem list, past medical history, and billing codes. In addition, we used objective data such as hemoglobin A1c, ejection fraction, outpatient creatinine, and outpatient blood pressure to augment problem list diagnoses where relevant.

Based on prior literature, we constructed multivariable logistic regression models for mortality adjusting for age; sex; self-reported race and ethnicity; body mass index; smoking history; presence of hypertension, heart failure, hyperlipidemia, coronary artery disease, diabetes, cancer, chronic kidney disease, dementia, or pulmonary disease individually as dummy variables; and admission oxygen saturation, D-dimer, ferritin, and C-reactive protein.^3-6 In the first model (C statistic 0.82), we did not include month of admission as a covariate and calculated the ratio of the sum of observed and expected deaths (obtained from the model) in each month to obtain the standardized mortality ratio (SMR) for each month. We then multiplied each period’s SMR by the overall average crude mortality to generate monthly adjusted mortality rates. We calculated Poisson control limits and indicated points outside the control limits as significantly different.

In a second model (C statistic 0.84), we included month as a covariate and calculated average marginal effects (AME) for each time period by using the margins library in R,⁷ which uses a discrete first-difference in predicted outcomes to obtain the AME. The average marginal effect represents the percentage point difference between the reference period (March) and a subsequent time period in probability of death or discharge to hospice, for equivalent patients. We obtained lower and upper confidence intervals for the AME using a bootstrapping approach described in Green.⁸ Finally, we conducted two sensitivity analyses: one, restricting the analysis to only those patients with principal diagnosis of COVID-19, sepsis, or respiratory disease (see Appendix A for complete list of codes) and one restricting the analysis to only those with length of stay of at least 3 days.

All statistical analyses were conducted with R, version 4.0.2. All analyses used 2-sided statistical tests, and we considered a P value < .05 to be statistically significant without adjustment for multiple testing. The NYU institutional review board approved the study and granted a waiver of consent and a waiver of the Health Information Portability and Accountability Act.

We included 5,121 hospitalizations, of which 5,118 (99.94%) had known outcomes (death or hospital discharge). Peak hospitalizations occurred in late March to mid-April, which accounted for 53% of the hospitalizations. Median length of stay for patients who died or were discharged to hospice was 8 days (interquartile range, 4-15; max 140 days). The median age and the proportion male or with any comorbidity decreased over time (Table). For instance, the proportion with any chronic condition decreased from 81% in March to 72% in August.

horwitz11661023e_t1.jpg

Adjusted mortality dropped each month, from 25.6% in March to 7.6% in August (Table and Figure). The SMR declined progressively over time, from 1.26 (95% CI, 1.15-1.39) in March to 0.38 (95% CI, 0.12-0.88) in August (Table). The adjusted average marginal effect was also significantly lower than in March in every subsequent month, reaching a maximum of an average 18.2 (95% CI, 12.0-24.4) percentage point decrease in probability of death in August, accounting for changes in demographics and clinical severity (Table and Appendix B). The decrease in unadjusted mortality over time was observed across age groups (Appendix C).

horwitz11661023e_f1.jpg

Results of the two sensitivity analyses were similar (Appendices D and E), though attenuated in the case of the sepsis/respiratory cohort, with adjusted mortality falling from 31.4% to 14.4%, SMR decreasing from 1.28 (95% CI, 1.16-1.41) to 0.59 (95% CI, 0.16-1.50), and AME in August 17.0 percentage points (95% CI, 6.0-28.1).

In this study of COVID-19 mortality over 6 months at a single health system, we found that changes in demographics and severity of illness at presentation did not fully explain decreases in mortality seen over time. Even after risk adjustment for a variety of clinical and demographic factors, including severity of illness at presentation, mortality was significantly and progressively lower over the course of the study period.

Similar risk-adjusted results have been preliminarily reported among intensive care unit patients in a preprint from the United Kingdom.⁹ Incremental improvements in outcomes are likely a combination of increasing clinical experience, decreasing hospital volume, growing use of new pharmacologic treatments (such as systemic corticosteroids,¹⁰ remdesivir,¹¹ and anticytokine treatments), nonpharmacologic treatments (such as placing the patient in the prone position, or proning, rather than on their back), earlier intervention, community awareness, and, potentially, lower viral load exposure from increased mask wearing and social distancing.¹²

Strengths of this study include highly detailed electronic health record data on hospitalizations at three different hospitals, a diverse patient population,⁶ near-complete study outcomes, and a lengthy period of investigation of 6 months. However, this study does have limitations. All patients were from a single geographic region and treated within a single health system, though restricting data to one system reduces institution-level variability and allows us to assess how care may have evolved with growing experience. Aggregating data from numerous health systems that might be at different stages of local outbreaks, provide different quality of care, and contribute different numbers of patients in each period introduces its own biases. We were also unable to disentangle different potential explanatory factors given the observational nature of the study. Residual confounding, such as a higher proportion of particularly frail patients admitted in earlier periods, is also a possibility, though the fact that we observed declines across all age groups mitigates this concern. Thresholds for hospital admission may also have changed over time with less severely ill patients being admitted in the later time periods. While changing admission thresholds could have contributed to higher survival rates in the latter portions of the study, our inclusion of several highly predictive clinical and laboratory results likely captured many aspects of disease severity.

In summary, data from one health system suggest that COVID-19 remains a serious disease for high-risk patients, but that mortality rates are improving over time.

Early reports showed high mortality from coronavirus disease 2019 (COVID-19), while current United States data mortality rates are lower, raising hope that new treatments and management strategies have improved outcomes. For instance, Centers for Disease Control and Prevention data show that 6.7% of cases resulted in death in April, compared with 1.9% in September.¹ However, the demographics of those infected have also changed, and more available testing may mean more comprehensive identification and earlier treatment. Nationally, for instance, the median age of confirmed cases was 38 years at the end of August, down from 46 years at the start of May.² Therefore, whether decreasing COVID-19 mortality rates simply reflect changing demographics or represent actual improvements in clinical care is unknown. The objective of this analysis was to assess outcomes over time in a single health system, accounting for changes in demographics, clinical factors, and severity of disease at presentation.

We analyzed monthly mortality rates for admissions between March 1 and August 31, 2020, in a single health system in New York City. Outcomes were obtained as of October 8, 2020. We included all hospitalizations of people 18 years and older with laboratory-confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection identified during the hospitalization or in the prior 2 weeks, excluding those admitted to hospice care. Patients with multiple hospitalizations (N=208 patients, 229 hospitalizations, 4.4%) were included repeatedly if they continued to have laboratory-confirmed disease. Patients without admission vital signs (N=28) were excluded. Mortality was defined as in-hospital death or discharge to hospice care. In-house laboratory testing began March 16 and all inpatients were tested for SARS-CoV-2 by April 1; elective surgeries resumed May 4-11 and were only conducted on confirmed SARS-CoV-2–negative patients.

All data were obtained from the electronic health record (Epic Systems, Verona, Wisconsin). Diagnosis codes were obtained from the problem list, past medical history, and billing codes. In addition, we used objective data such as hemoglobin A1c, ejection fraction, outpatient creatinine, and outpatient blood pressure to augment problem list diagnoses where relevant.

Based on prior literature, we constructed multivariable logistic regression models for mortality adjusting for age; sex; self-reported race and ethnicity; body mass index; smoking history; presence of hypertension, heart failure, hyperlipidemia, coronary artery disease, diabetes, cancer, chronic kidney disease, dementia, or pulmonary disease individually as dummy variables; and admission oxygen saturation, D-dimer, ferritin, and C-reactive protein.^3-6 In the first model (C statistic 0.82), we did not include month of admission as a covariate and calculated the ratio of the sum of observed and expected deaths (obtained from the model) in each month to obtain the standardized mortality ratio (SMR) for each month. We then multiplied each period’s SMR by the overall average crude mortality to generate monthly adjusted mortality rates. We calculated Poisson control limits and indicated points outside the control limits as significantly different.

In a second model (C statistic 0.84), we included month as a covariate and calculated average marginal effects (AME) for each time period by using the margins library in R,⁷ which uses a discrete first-difference in predicted outcomes to obtain the AME. The average marginal effect represents the percentage point difference between the reference period (March) and a subsequent time period in probability of death or discharge to hospice, for equivalent patients. We obtained lower and upper confidence intervals for the AME using a bootstrapping approach described in Green.⁸ Finally, we conducted two sensitivity analyses: one, restricting the analysis to only those patients with principal diagnosis of COVID-19, sepsis, or respiratory disease (see Appendix A for complete list of codes) and one restricting the analysis to only those with length of stay of at least 3 days.

All statistical analyses were conducted with R, version 4.0.2. All analyses used 2-sided statistical tests, and we considered a P value < .05 to be statistically significant without adjustment for multiple testing. The NYU institutional review board approved the study and granted a waiver of consent and a waiver of the Health Information Portability and Accountability Act.

We included 5,121 hospitalizations, of which 5,118 (99.94%) had known outcomes (death or hospital discharge). Peak hospitalizations occurred in late March to mid-April, which accounted for 53% of the hospitalizations. Median length of stay for patients who died or were discharged to hospice was 8 days (interquartile range, 4-15; max 140 days). The median age and the proportion male or with any comorbidity decreased over time (Table). For instance, the proportion with any chronic condition decreased from 81% in March to 72% in August.

horwitz11661023e_t1.jpg

Adjusted mortality dropped each month, from 25.6% in March to 7.6% in August (Table and Figure). The SMR declined progressively over time, from 1.26 (95% CI, 1.15-1.39) in March to 0.38 (95% CI, 0.12-0.88) in August (Table). The adjusted average marginal effect was also significantly lower than in March in every subsequent month, reaching a maximum of an average 18.2 (95% CI, 12.0-24.4) percentage point decrease in probability of death in August, accounting for changes in demographics and clinical severity (Table and Appendix B). The decrease in unadjusted mortality over time was observed across age groups (Appendix C).

horwitz11661023e_f1.jpg

Results of the two sensitivity analyses were similar (Appendices D and E), though attenuated in the case of the sepsis/respiratory cohort, with adjusted mortality falling from 31.4% to 14.4%, SMR decreasing from 1.28 (95% CI, 1.16-1.41) to 0.59 (95% CI, 0.16-1.50), and AME in August 17.0 percentage points (95% CI, 6.0-28.1).

In this study of COVID-19 mortality over 6 months at a single health system, we found that changes in demographics and severity of illness at presentation did not fully explain decreases in mortality seen over time. Even after risk adjustment for a variety of clinical and demographic factors, including severity of illness at presentation, mortality was significantly and progressively lower over the course of the study period.

Similar risk-adjusted results have been preliminarily reported among intensive care unit patients in a preprint from the United Kingdom.⁹ Incremental improvements in outcomes are likely a combination of increasing clinical experience, decreasing hospital volume, growing use of new pharmacologic treatments (such as systemic corticosteroids,¹⁰ remdesivir,¹¹ and anticytokine treatments), nonpharmacologic treatments (such as placing the patient in the prone position, or proning, rather than on their back), earlier intervention, community awareness, and, potentially, lower viral load exposure from increased mask wearing and social distancing.¹²

Strengths of this study include highly detailed electronic health record data on hospitalizations at three different hospitals, a diverse patient population,⁶ near-complete study outcomes, and a lengthy period of investigation of 6 months. However, this study does have limitations. All patients were from a single geographic region and treated within a single health system, though restricting data to one system reduces institution-level variability and allows us to assess how care may have evolved with growing experience. Aggregating data from numerous health systems that might be at different stages of local outbreaks, provide different quality of care, and contribute different numbers of patients in each period introduces its own biases. We were also unable to disentangle different potential explanatory factors given the observational nature of the study. Residual confounding, such as a higher proportion of particularly frail patients admitted in earlier periods, is also a possibility, though the fact that we observed declines across all age groups mitigates this concern. Thresholds for hospital admission may also have changed over time with less severely ill patients being admitted in the later time periods. While changing admission thresholds could have contributed to higher survival rates in the latter portions of the study, our inclusion of several highly predictive clinical and laboratory results likely captured many aspects of disease severity.

In summary, data from one health system suggest that COVID-19 remains a serious disease for high-risk patients, but that mortality rates are improving over time.