Laura M. Grosso, PhD, MPH

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.

Chronic obstructive pulmonary disease (COPD) affects as many as 24 million individuals in the United States, is responsible for more than 700,000 annual hospital admissions, and is currently the nation's third leading cause of death, accounting for nearly $49.9 billion in medical spending in 2010.[1, 2] Reported in‐hospital mortality rates for patients hospitalized for exacerbations of COPD range from 2% to 5%.[3, 4, 5, 6, 7] Information about 30‐day mortality rates following hospitalization for COPD is more limited; however, international studies suggest that rates range from 3% to 9%,[8, 9] and 90‐day mortality rates exceed 15%.[10]

Despite this significant clinical and economic impact, there have been no large‐scale, sustained efforts to measure the quality or outcomes of hospital care for patients with COPD in the United States. What little is known about the treatment of patients with COPD suggests widespread opportunities to increase adherence to guideline‐recommended therapies, to reduce the use of ineffective treatments and tests, and to address variation in care across institutions.[5, 11, 12]

Public reporting of hospital performance is a key strategy for improving the quality and safety of hospital care, both in the United States and internationally.[13] Since 2007, the Centers for Medicare and Medicaid Services (CMS) has reported hospital mortality rates on the Hospital Compare Web site, and COPD is 1 of the conditions highlighted in the Affordable Care Act for future consideration.[14] Such initiatives rely on validated, risk‐adjusted performance measures for comparisons across institutions and to enable outcomes to be tracked over time. We present the development, validation, and results of a model intended for public reporting of risk‐standardized mortality rates for patients hospitalized with exacerbations of COPD that has been endorsed by the National Quality Forum.[15]

METHODS

RESULTS

Model Derivation

After exclusions were applied, the development sample included 150,035 admissions in 2008 at 4537 US hospitals (Figure 1). Factors that were most strongly associated with the risk of mortality included metastatic cancer (odds ratio [OR] 2.34), protein calorie malnutrition (OR 2.18), nonmetastatic cancers of the lung and upper digestive tract, (OR 1.80) cardiorespiratory failure and shock (OR 1.60), and congestive heart failure (OR 1.34) (Table 2).

Figure 1

Hospital Risk‐Standardized Mortality Rates

The mean unadjusted hospital 30‐day mortality rate was 8.6% and ranged from 0% to 100% (Figure 2a). Risk‐standardized mortality rates varied across hospitals (Figure 2b). The mean risk‐standardized mortality rate was 8.6% and ranged from 5.9% to 13.5%. The odds of mortality at a hospital 1 standard deviation above average was 1.20 times that of a hospital 1 standard deviation below average.

Figure 2

DISCUSSION

We present a hospital‐level risk‐standardized mortality measure for patients admitted with COPD based on administrative claims data that are intended for public reporting and that have achieved endorsement by the National Quality Forum, a voluntary consensus standards‐setting organization. Across more than 4500 US hospitals, the mean 30‐day risk‐standardized mortality rate in 2008 was 8.6%, and we observed considerable variation across institutions, despite adjustment for case mix, suggesting that improvement by lower‐performing institutions may be an achievable goal.

Although improving the delivery of evidence‐based care processes and outcomes of patients with acute myocardial infarction, heart failure, and pneumonia has been the focus of national quality improvement efforts for more than a decade, COPD has largely been overlooked.[23] Within this context, this analysis represents the first attempt to systematically measure, at the hospital level, 30‐day all‐cause mortality for patients admitted to US hospitals for exacerbation of COPD. The model we have developed and validated is intended to be used to compare the performance of hospitals while controlling for differences in the pretreatment risk of mortality of patients and accounting for the clustering of patients within hospitals, and will facilitate surveillance of hospital‐level risk‐adjusted outcomes over time.

In contrast to process‐based measures of quality, such as the percentage of patients with pneumonia who receive appropriate antibiotic therapy, performance measures based on patient outcomes provide a more comprehensive view of care and are more consistent with patients' goals.[24] Additionally, it is well established that hospital performance on individual and composite process measures explains only a small amount of the observed variation in patient outcomes between institutions.[25] In this regard, outcome measures incorporate important, but difficult to measure aspects of care, such as diagnostic accuracy and timing, communication and teamwork, the recognition and response to complications, care coordination at the time of transfers between levels of care, and care settings. Nevertheless, when used for making inferences about the quality of hospital care, individual measures such as the risk‐standardized hospital mortality rate should be interpreted in the context of other performance measures, including readmission, patient experience, and costs of care.

A number of prior investigators have described the outcomes of care for patients hospitalized with exacerbations of COPD, including identifying risk factors for mortality. Patil et al. carried out an analysis of the 1996 Nationwide Inpatient Sample and described an overall in‐hospital mortality rate of 2.5% among patients with COPD, and reported that a multivariable model containing sociodemographic characteristics about the patient and comorbidities had an area under the ROC curve of 0.70.[3] In contrast, this hospital‐level measure includes patients with a principal diagnosis of respiratory failure and focuses on 30‐day rather than inpatient mortality, accounting for the nearly 3‐fold higher mortality rate we observed. In a more recent study that used clinical from a large multistate database, Tabak et al. developed a prediction model for inpatient mortality for patients with COPD that contained only 4 factors: age, blood urea nitrogen, mental status, and pulse, and achieved an area under the ROC curve of 0.72.[4] The simplicity of such a model and its reliance on clinical measurements makes it particularly well suited for bedside application by clinicians, but less valuable for large‐scale public reporting programs that rely on administrative data. In the only other study identified that focused on the assessment of hospital mortality rates, Agabiti et al. analyzed the outcomes of 12,756 patients hospitalized for exacerbations of COPD, using similar ICD‐9‐CM diagnostic criteria as in this study, at 21 hospitals in Rome, Italy.[26] They reported an average crude 30‐day mortality rate of 3.8% among a group of 5 benchmark hospitals and an average mortality of 7.5% (range, 5.2%17.2%) among the remaining institutions.

To put the variation we observed in mortality rates into a broader context, the relative difference in the risk‐standardized hospital mortality rates across the 10th to 90th percentiles of hospital performance was 25% for acute myocardial infarction and 39% for heart failure, whereas rates varied 30% for COPD, from 7.6% to 9.9%.[27] Model discrimination in COPD (C statistic, 0.72) was also similar to that reported for models used for public reporting of hospital mortality in acute myocardial infarction (C statistic, 0.71) and pneumonia (C statistic, 0.72).

This study has a number of important strengths. First, the model was developed from a large sample of recent Medicare claims, achieved good discrimination, and was validated in samples not limited to Medicare beneficiaries. Second, by including patients with a principal diagnosis of COPD, as well as those with a principal diagnosis of acute respiratory failure when accompanied by a secondary diagnosis of COPD with acute exacerbation, this model can be used to assess hospital performance across the full spectrum of disease severity. This broad set of ICD‐9‐CM codes used to define the cohort also ensures that efforts to measure hospital performance will be less influenced by differences in documentation and coding practices across hospitals relating to the diagnosis or sequencing of acute respiratory failure diagnoses. Moreover, the inclusion of patients with respiratory failure is important because these patients have the greatest risk of mortality, and are those in whom efforts to improve the quality and safety of care may have the greatest impact. Third, rather than relying solely on information documented during the index admission, we used ambulatory and inpatient claims from the full year prior to the index admission to identify comorbidities and to distinguish them from potential complications of care. Finally, we did not include factors such as hospital characteristics (eg, number of beds, teaching status) in the model. Although they might have improved overall predictive ability, the goal of the hospital mortality measure is to enable comparisons of mortality rates among hospitals while controlling for differences in patient characteristics. To the extent that factors such as size or teaching status might be independently associated with hospital outcomes, it would be inappropriate to adjust away their effects, because mortality risk should not be influenced by hospital characteristics other than through their effects on quality.

These results should be viewed in light of several limitations. First, we used ICD‐9‐CM codes derived from claims files to define the patient populations included in the measure rather than collecting clinical or physiologic information prospectively or through manual review of medical records, such as the forced expiratory volume in 1 second or whether the patient required long‐term oxygen therapy. Nevertheless, we included a broad set of potential diagnosis codes to capture the full spectrum of COPD exacerbations and to minimize differences in coding across hospitals. Second, because the risk‐adjustment included diagnoses coded in the year prior to the index admission, it is potentially subject to bias due to regional differences in medical care utilization that are not driven by underlying differences in patient illness.[28] Third, using administrative claims data, we observed some paradoxical associations in the model that are difficult to explain on clinical grounds, such as a protective effect of substance and alcohol abuse or prior episodes of respiratory failure. Fourth, although we excluded patients from the analysis who were enrolled in hospice prior to, or on the day of, the index admission, we did not exclude those who choose to withdraw support, transition to comfort measures only, or enrolled in hospice care during a hospitalization. We do not seek to penalize hospitals for being sensitive to the preferences of patients at the end of life. At the same time, it is equally important that the measure is capable of detecting the outcomes of suboptimal care that may in some instances lead a patient or their family to withdraw support or choose hospice. Finally, we did not have the opportunity to validate the model against a clinical registry of patients with COPD, because such data do not currently exist. Nevertheless, the use of claims as a surrogate for chart data for risk adjustment has been validated for several conditions, including acute myocardial infarction, heart failure, and pneumonia.[29, 30]

CONCLUSIONS

Risk‐standardized 30‐day mortality rates for Medicare beneficiaries with COPD vary across hospitals in the US. Calculating and reporting hospital outcomes using validated performance measures may catalyze quality improvement activities and lead to better outcomes. Additional research would be helpful to confirm that hospitals with lower mortality rates achieve care that meets the goals of patients and their families better than at hospitals with higher mortality rates.

Disclosures

Peter K. Lindenauer, MD, MSc, is the guarantor of this article, taking responsibility for the integrity of the work as a whole, from inception to published article, and takes responsibility for the content of the manuscript, including the data and data analysis. All authors have made substantial contributions to the conception and design, or acquisition of data, or analysis and interpretation of data; have drafted the submitted article or revised it critically for important intellectual content; and have provided final approval of the version to be published. Preparation of this manuscript was completed under Contract Number: HHSM‐5002008‐0025I/HHSM‐500‐T0001, Modification No. 000007, Option Year 2 Measure Instrument Development and Support (MIDS). Sponsors did not contribute to the development of the research or manuscript. Dr. Au reports being an unpaid research consultant for Bosch Inc. He receives research funding from the NIH, Department of Veterans Affairs, AHRQ, and Gilead Sciences. The views of the this manuscript represent the authors and do not necessarily represent those of the Department of Veterans Affairs. Drs. Drye and Bernheim report receiving contract funding from CMS to develop and maintain quality measures.