Sharon‐Lise T. Normand, PhD

Data Sources

We used 20052006 claims data from Medicare inpatient, outpatient, and carrier (physician) Standard Analytic Files to develop and validate the administrative model. The Medicare Enrollment Database was used to determine Medicare fee‐for‐service enrollment and mortality statuses. A medical record model, used for additional validation of the administrative model, was developed using information abstracted from the charts of 75,616 pneumonia cases from 19982001 as part of the National Pneumonia Project, a CMS quality improvement initiative.18

Study Cohort

We identified hospitalizations of patients 65 years of age and older with a principal diagnosis of pneumonia (International Classification of Diseases, 9th Revision, Clinical Modification codes 480.XX, 481, 482.XX, 483.X, 485, 486, 487.0) as potential index pneumonia admissions. Because our focus was readmission for patients discharged from acute care settings, we excluded admissions in which patients died or were transferred to another acute care facility. Additionally, we restricted analysis to patients who had been enrolled in fee‐for‐service Medicare Parts A and B, for at least 12 months prior to their pneumonia hospitalization, so that we could use diagnostic codes from all inpatient and outpatient encounters during that period to enhance identification of comorbidities.

Outcome

The outcome was 30‐day readmission, defined as occurrence of at least one hospitalization for any cause within 30 days of discharge after an index admission. Readmissions were identified from hospital claims data, and were attributed to the hospital that had discharged the patient. A 30‐day time frame was selected because it is a clinically meaningful period during which hospitals can be expected to collaborate with other organizations and providers to implement measures to reduce the risk of rehospitalization.

Candidate and Final Model Variables

Candidate variables for the administrative claims model were selected by a clinician team from 189 diagnostic groups included in the Hierarchical Condition Category (HCC) clinical classification system.19 The HCC clinical classification system was developed for CMS in preparation for all‐encounter risk adjustment for Medicare Advantage (managed care). Under the HCC algorithm, the 15,000+ ICD‐9‐CM diagnosis codes are assigned to one of 189 clinically‐coherent condition categories (CCs). We used the April 2008 version of the ICD‐9‐CM to CC assignment map, which is maintained by CMS and posted at http://www.qualitynet.org. A total of 154 CCs were considered to be potentially relevant to readmission outcome and were included for further consideration. Some CCs were further combined into clinically coherent groupings of CCs. Our set of candidate variables ultimately included 97 CC‐based variables, two demographic variables (age and sex), and two procedure codes potentially relevant to readmission risk (history of percutaneous coronary intervention [PCI] and history of coronary artery bypass graft [CABG]).

The final risk‐adjustment model included 39 variables selected by the team of clinicians and analysts, primarily based on their clinical relevance but also with knowledge of the strength of their statistical association with readmission outcome (Table 1). For each patient, the presence or absence of these conditions was assessed from multiple sources, including secondary diagnoses during the index admission, principal and secondary diagnoses from hospital admissions in the 12 months prior to the index admission, and diagnoses from hospital outpatient and physician encounters 12 months before the index admission. A small number of CCs were considered to represent potential complications of care (eg, bleeding). Because we did not want to adjust for complications of care occurring during the index admission, a patient was not considered to have one of these conditions unless it was also present in at least one encounter prior to the index admission.

Model Derivation

For the development of the administrative claims model, we randomly sampled half of 2006 hospitalizations that met inclusion criteria. To assess model performance at the patient level, we calculated the area under the receiver operating curve (AUC), and calculated observed readmission rates in the lowest and highest deciles on the basis of predicted readmission probabilities. We also compared performance with a null model, a model that adjusted for age and sex, and a model that included all candidate variables.20

Risk‐Standardized Readmission Rates

Using hierarchical logistic regression, we modeled the log‐odds of readmission within 30 days of discharge from an index pneumonia admission as a function of patient demographic and clinical characteristics, and a random hospital‐specific intercept. This strategy accounts for within‐hospital correlation, or clustering, of observed outcomes, and models the assumption that underlying differences in quality among hospitals being evaluated lead to systematic differences in outcomes. We then calculated hospital‐specific readmission rates as the ratio of predicted‐to‐expected readmissions (similar to observed/expected ratio), multiplied by the national unadjusted ratea form of indirect standardization. Predicted number of readmissions in each hospital is estimated given the same patient mix and its estimated hospital‐specific intercept. Expected number of readmissions in each hospital is estimated using its patient mix and the average hospital‐specific intercept. To assess hospital performance in any given year, we re‐estimate model coefficients using that year's data.

Model Validation: Administrative Claims

We compared the model performance in the development sample with its performance in the sample from the 2006 data that was not selected for the development set, and separately among pneumonia admissions in 2005. The model was recalibrated in each validation set.

Model Validation: Medical Record Abstraction

We developed a separate medical record‐based model of readmission risk using information from charts that had previously been abstracted as part of CMS's National Pneumonia Project. To select variables for this model, the clinician team: 1) reviewed the list of variables that were included in a medical record model that was previously developed for validating the National Quality Forum‐approved pneumonia mortality measure; 2) reviewed a list of other potential candidate variables available in the National Pneumonia Project dataset; and 3) reviewed variables that emerged as potentially important predictors of readmission, based on a systematic review of the literature that was conducted as part of measure development. This selection process resulted in a final medical record model that included 35 variables.

We linked patients in the National Pneumonia Project cohort to their Medicare claims data, including claims from one year before the index hospitalization, so that we could calculate risk‐standardized readmission rates in this cohort separately using medical record and claims‐based models. This analysis was conducted at the state level, for the 50 states plus the District of Columbia and Puerto Rico, because medical record data were unavailable in sufficient numbers to permit hospital‐level comparisons. To examine the relationship between risk‐standardized rates obtained from medical record and administrative data models, we estimated a linear regression model describing the association between the two rates, weighting each state by number of index hospitalizations, and calculated the correlation coefficient and the intercept and slope of this equation. A slope close to 1 and an intercept close to 0 would provide evidence that risk‐standardized state readmission rates from the medical record and claims models were similar. We also calculated the difference between state risk‐standardized readmission rates from the two models.

Analyses were conducted with the use of SAS version 9.1.3 (SAS Institute Inc, Cary, NC). Models were fitted separately for the National Pneumonia Project and 2006 cohort. We estimated the hierarchical models using the GLIMMIX procedure in SAS. The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.

Model Derivation and Performance

After exclusions were applied, the 2006 sample included 453,251 pneumonia hospitalizations (Figure 1). The development sample consisted of 226,545 hospitalizations at 4675 hospitals, with an overall unadjusted 30‐day readmission rate of 17.4%. In 11,694 index cases (5.2%), the patient died within 30 days without being readmitted. Median readmission rate was 16.3%, 25th and 75th percentile rates were 11.1% and 21.3%, and at the 10th and 90th percentile, hospital readmission rates ranged from 4.6% to 26.7% (Figure 2).

Figure 1

Figure 2

The claims model included 39 variables (age, sex, and 37 clinical variables) (Table 1). The mean age of the cohort was 80.0 years, with 55.5% women and 11.1% nonwhite patients. Mean observed readmission rate in the development sample ranged from 9% in the lowest decile of predicted pneumonia readmission rates to 32% in the highest predicted decile, a range of 23%. The AUC was 0.63. For comparison, a model with only age and sex had an AUC of 0.51, and a model with all candidate variables had an AUC equal to 0.63 (Table 2).

Hospital Risk‐Standardized Readmission Rates

Risk‐standardized readmission rates varied across hospitals (Figure 3). Median risk‐standardized readmission rate was 17.3%, and the 25th and 75th percentiles were 16.9% and 17.9%, respectively. The 5th percentile was 16.0% and the 95th percentile was 19.1%. Odds of readmission for a hospital one standard deviation above average was 1.4 times that of a hospital one standard deviation below average.

Figure 3

Administrative Model Validation

In the remaining 50% of pneumonia index hospitalizations from 2006, and the entire 2005 cohort, regression coefficients and standard errors of model variables were similar to those in the development data set. Model performance using 2005 data was consistent with model performance using the 2006 development and validation half‐samples (Table 2).

Medical Record Validation

After exclusions, the medical record sample taken from the National Pneumonia Project included 47,429 cases, with an unadjusted 30‐day readmission rate of 17.0%. The final medical record risk‐adjustment model included a total of 35 variables, whose prevalence and association with readmission risk varied modestly (Table 3). Performance of the medical record and administrative models was similar (areas under the ROC curve 0.59 and 0.63, respectively) (Table 4). Additionally, in the administrative model, predicted readmission rates ranged from 8% in the lowest predicted decile to 30% in the highest predicted decile, while in the medical record model, the corresponding rates varied from 10% to 26%.

The correlation coefficient of the estimated state‐specific standardized readmission rates from the administrative and medical record models was 0.96, and the proportion of the variance explained by the model was 0.92 (Figure 4).

Figure 4

Data Sources

We used 20052006 claims data from Medicare inpatient, outpatient, and carrier (physician) Standard Analytic Files to develop and validate the administrative model. The Medicare Enrollment Database was used to determine Medicare fee‐for‐service enrollment and mortality statuses. A medical record model, used for additional validation of the administrative model, was developed using information abstracted from the charts of 75,616 pneumonia cases from 19982001 as part of the National Pneumonia Project, a CMS quality improvement initiative.18

Study Cohort

We identified hospitalizations of patients 65 years of age and older with a principal diagnosis of pneumonia (International Classification of Diseases, 9th Revision, Clinical Modification codes 480.XX, 481, 482.XX, 483.X, 485, 486, 487.0) as potential index pneumonia admissions. Because our focus was readmission for patients discharged from acute care settings, we excluded admissions in which patients died or were transferred to another acute care facility. Additionally, we restricted analysis to patients who had been enrolled in fee‐for‐service Medicare Parts A and B, for at least 12 months prior to their pneumonia hospitalization, so that we could use diagnostic codes from all inpatient and outpatient encounters during that period to enhance identification of comorbidities.

Outcome

The outcome was 30‐day readmission, defined as occurrence of at least one hospitalization for any cause within 30 days of discharge after an index admission. Readmissions were identified from hospital claims data, and were attributed to the hospital that had discharged the patient. A 30‐day time frame was selected because it is a clinically meaningful period during which hospitals can be expected to collaborate with other organizations and providers to implement measures to reduce the risk of rehospitalization.

Candidate and Final Model Variables

Candidate variables for the administrative claims model were selected by a clinician team from 189 diagnostic groups included in the Hierarchical Condition Category (HCC) clinical classification system.19 The HCC clinical classification system was developed for CMS in preparation for all‐encounter risk adjustment for Medicare Advantage (managed care). Under the HCC algorithm, the 15,000+ ICD‐9‐CM diagnosis codes are assigned to one of 189 clinically‐coherent condition categories (CCs). We used the April 2008 version of the ICD‐9‐CM to CC assignment map, which is maintained by CMS and posted at http://www.qualitynet.org. A total of 154 CCs were considered to be potentially relevant to readmission outcome and were included for further consideration. Some CCs were further combined into clinically coherent groupings of CCs. Our set of candidate variables ultimately included 97 CC‐based variables, two demographic variables (age and sex), and two procedure codes potentially relevant to readmission risk (history of percutaneous coronary intervention [PCI] and history of coronary artery bypass graft [CABG]).

The final risk‐adjustment model included 39 variables selected by the team of clinicians and analysts, primarily based on their clinical relevance but also with knowledge of the strength of their statistical association with readmission outcome (Table 1). For each patient, the presence or absence of these conditions was assessed from multiple sources, including secondary diagnoses during the index admission, principal and secondary diagnoses from hospital admissions in the 12 months prior to the index admission, and diagnoses from hospital outpatient and physician encounters 12 months before the index admission. A small number of CCs were considered to represent potential complications of care (eg, bleeding). Because we did not want to adjust for complications of care occurring during the index admission, a patient was not considered to have one of these conditions unless it was also present in at least one encounter prior to the index admission.

Model Derivation

For the development of the administrative claims model, we randomly sampled half of 2006 hospitalizations that met inclusion criteria. To assess model performance at the patient level, we calculated the area under the receiver operating curve (AUC), and calculated observed readmission rates in the lowest and highest deciles on the basis of predicted readmission probabilities. We also compared performance with a null model, a model that adjusted for age and sex, and a model that included all candidate variables.20

Risk‐Standardized Readmission Rates

Using hierarchical logistic regression, we modeled the log‐odds of readmission within 30 days of discharge from an index pneumonia admission as a function of patient demographic and clinical characteristics, and a random hospital‐specific intercept. This strategy accounts for within‐hospital correlation, or clustering, of observed outcomes, and models the assumption that underlying differences in quality among hospitals being evaluated lead to systematic differences in outcomes. We then calculated hospital‐specific readmission rates as the ratio of predicted‐to‐expected readmissions (similar to observed/expected ratio), multiplied by the national unadjusted ratea form of indirect standardization. Predicted number of readmissions in each hospital is estimated given the same patient mix and its estimated hospital‐specific intercept. Expected number of readmissions in each hospital is estimated using its patient mix and the average hospital‐specific intercept. To assess hospital performance in any given year, we re‐estimate model coefficients using that year's data.

Model Validation: Administrative Claims

We compared the model performance in the development sample with its performance in the sample from the 2006 data that was not selected for the development set, and separately among pneumonia admissions in 2005. The model was recalibrated in each validation set.

Model Validation: Medical Record Abstraction

We developed a separate medical record‐based model of readmission risk using information from charts that had previously been abstracted as part of CMS's National Pneumonia Project. To select variables for this model, the clinician team: 1) reviewed the list of variables that were included in a medical record model that was previously developed for validating the National Quality Forum‐approved pneumonia mortality measure; 2) reviewed a list of other potential candidate variables available in the National Pneumonia Project dataset; and 3) reviewed variables that emerged as potentially important predictors of readmission, based on a systematic review of the literature that was conducted as part of measure development. This selection process resulted in a final medical record model that included 35 variables.

We linked patients in the National Pneumonia Project cohort to their Medicare claims data, including claims from one year before the index hospitalization, so that we could calculate risk‐standardized readmission rates in this cohort separately using medical record and claims‐based models. This analysis was conducted at the state level, for the 50 states plus the District of Columbia and Puerto Rico, because medical record data were unavailable in sufficient numbers to permit hospital‐level comparisons. To examine the relationship between risk‐standardized rates obtained from medical record and administrative data models, we estimated a linear regression model describing the association between the two rates, weighting each state by number of index hospitalizations, and calculated the correlation coefficient and the intercept and slope of this equation. A slope close to 1 and an intercept close to 0 would provide evidence that risk‐standardized state readmission rates from the medical record and claims models were similar. We also calculated the difference between state risk‐standardized readmission rates from the two models.

Analyses were conducted with the use of SAS version 9.1.3 (SAS Institute Inc, Cary, NC). Models were fitted separately for the National Pneumonia Project and 2006 cohort. We estimated the hierarchical models using the GLIMMIX procedure in SAS. The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.

Model Derivation and Performance

After exclusions were applied, the 2006 sample included 453,251 pneumonia hospitalizations (Figure 1). The development sample consisted of 226,545 hospitalizations at 4675 hospitals, with an overall unadjusted 30‐day readmission rate of 17.4%. In 11,694 index cases (5.2%), the patient died within 30 days without being readmitted. Median readmission rate was 16.3%, 25th and 75th percentile rates were 11.1% and 21.3%, and at the 10th and 90th percentile, hospital readmission rates ranged from 4.6% to 26.7% (Figure 2).

Figure 1

Figure 2

The claims model included 39 variables (age, sex, and 37 clinical variables) (Table 1). The mean age of the cohort was 80.0 years, with 55.5% women and 11.1% nonwhite patients. Mean observed readmission rate in the development sample ranged from 9% in the lowest decile of predicted pneumonia readmission rates to 32% in the highest predicted decile, a range of 23%. The AUC was 0.63. For comparison, a model with only age and sex had an AUC of 0.51, and a model with all candidate variables had an AUC equal to 0.63 (Table 2).

Hospital Risk‐Standardized Readmission Rates

Risk‐standardized readmission rates varied across hospitals (Figure 3). Median risk‐standardized readmission rate was 17.3%, and the 25th and 75th percentiles were 16.9% and 17.9%, respectively. The 5th percentile was 16.0% and the 95th percentile was 19.1%. Odds of readmission for a hospital one standard deviation above average was 1.4 times that of a hospital one standard deviation below average.

Figure 3

Administrative Model Validation

In the remaining 50% of pneumonia index hospitalizations from 2006, and the entire 2005 cohort, regression coefficients and standard errors of model variables were similar to those in the development data set. Model performance using 2005 data was consistent with model performance using the 2006 development and validation half‐samples (Table 2).

Medical Record Validation

After exclusions, the medical record sample taken from the National Pneumonia Project included 47,429 cases, with an unadjusted 30‐day readmission rate of 17.0%. The final medical record risk‐adjustment model included a total of 35 variables, whose prevalence and association with readmission risk varied modestly (Table 3). Performance of the medical record and administrative models was similar (areas under the ROC curve 0.59 and 0.63, respectively) (Table 4). Additionally, in the administrative model, predicted readmission rates ranged from 8% in the lowest predicted decile to 30% in the highest predicted decile, while in the medical record model, the corresponding rates varied from 10% to 26%.

The correlation coefficient of the estimated state‐specific standardized readmission rates from the administrative and medical record models was 0.96, and the proportion of the variance explained by the model was 0.92 (Figure 4).

Figure 4

Data Sources

We used 20052006 claims data from Medicare inpatient, outpatient, and carrier (physician) Standard Analytic Files to develop and validate the administrative model. The Medicare Enrollment Database was used to determine Medicare fee‐for‐service enrollment and mortality statuses. A medical record model, used for additional validation of the administrative model, was developed using information abstracted from the charts of 75,616 pneumonia cases from 19982001 as part of the National Pneumonia Project, a CMS quality improvement initiative.18

Study Cohort

We identified hospitalizations of patients 65 years of age and older with a principal diagnosis of pneumonia (International Classification of Diseases, 9th Revision, Clinical Modification codes 480.XX, 481, 482.XX, 483.X, 485, 486, 487.0) as potential index pneumonia admissions. Because our focus was readmission for patients discharged from acute care settings, we excluded admissions in which patients died or were transferred to another acute care facility. Additionally, we restricted analysis to patients who had been enrolled in fee‐for‐service Medicare Parts A and B, for at least 12 months prior to their pneumonia hospitalization, so that we could use diagnostic codes from all inpatient and outpatient encounters during that period to enhance identification of comorbidities.

Outcome

The outcome was 30‐day readmission, defined as occurrence of at least one hospitalization for any cause within 30 days of discharge after an index admission. Readmissions were identified from hospital claims data, and were attributed to the hospital that had discharged the patient. A 30‐day time frame was selected because it is a clinically meaningful period during which hospitals can be expected to collaborate with other organizations and providers to implement measures to reduce the risk of rehospitalization.

Candidate and Final Model Variables

Candidate variables for the administrative claims model were selected by a clinician team from 189 diagnostic groups included in the Hierarchical Condition Category (HCC) clinical classification system.19 The HCC clinical classification system was developed for CMS in preparation for all‐encounter risk adjustment for Medicare Advantage (managed care). Under the HCC algorithm, the 15,000+ ICD‐9‐CM diagnosis codes are assigned to one of 189 clinically‐coherent condition categories (CCs). We used the April 2008 version of the ICD‐9‐CM to CC assignment map, which is maintained by CMS and posted at http://www.qualitynet.org. A total of 154 CCs were considered to be potentially relevant to readmission outcome and were included for further consideration. Some CCs were further combined into clinically coherent groupings of CCs. Our set of candidate variables ultimately included 97 CC‐based variables, two demographic variables (age and sex), and two procedure codes potentially relevant to readmission risk (history of percutaneous coronary intervention [PCI] and history of coronary artery bypass graft [CABG]).

The final risk‐adjustment model included 39 variables selected by the team of clinicians and analysts, primarily based on their clinical relevance but also with knowledge of the strength of their statistical association with readmission outcome (Table 1). For each patient, the presence or absence of these conditions was assessed from multiple sources, including secondary diagnoses during the index admission, principal and secondary diagnoses from hospital admissions in the 12 months prior to the index admission, and diagnoses from hospital outpatient and physician encounters 12 months before the index admission. A small number of CCs were considered to represent potential complications of care (eg, bleeding). Because we did not want to adjust for complications of care occurring during the index admission, a patient was not considered to have one of these conditions unless it was also present in at least one encounter prior to the index admission.

Model Derivation

For the development of the administrative claims model, we randomly sampled half of 2006 hospitalizations that met inclusion criteria. To assess model performance at the patient level, we calculated the area under the receiver operating curve (AUC), and calculated observed readmission rates in the lowest and highest deciles on the basis of predicted readmission probabilities. We also compared performance with a null model, a model that adjusted for age and sex, and a model that included all candidate variables.20

Risk‐Standardized Readmission Rates

Using hierarchical logistic regression, we modeled the log‐odds of readmission within 30 days of discharge from an index pneumonia admission as a function of patient demographic and clinical characteristics, and a random hospital‐specific intercept. This strategy accounts for within‐hospital correlation, or clustering, of observed outcomes, and models the assumption that underlying differences in quality among hospitals being evaluated lead to systematic differences in outcomes. We then calculated hospital‐specific readmission rates as the ratio of predicted‐to‐expected readmissions (similar to observed/expected ratio), multiplied by the national unadjusted ratea form of indirect standardization. Predicted number of readmissions in each hospital is estimated given the same patient mix and its estimated hospital‐specific intercept. Expected number of readmissions in each hospital is estimated using its patient mix and the average hospital‐specific intercept. To assess hospital performance in any given year, we re‐estimate model coefficients using that year's data.

Model Validation: Administrative Claims

We compared the model performance in the development sample with its performance in the sample from the 2006 data that was not selected for the development set, and separately among pneumonia admissions in 2005. The model was recalibrated in each validation set.

Model Validation: Medical Record Abstraction

We developed a separate medical record‐based model of readmission risk using information from charts that had previously been abstracted as part of CMS's National Pneumonia Project. To select variables for this model, the clinician team: 1) reviewed the list of variables that were included in a medical record model that was previously developed for validating the National Quality Forum‐approved pneumonia mortality measure; 2) reviewed a list of other potential candidate variables available in the National Pneumonia Project dataset; and 3) reviewed variables that emerged as potentially important predictors of readmission, based on a systematic review of the literature that was conducted as part of measure development. This selection process resulted in a final medical record model that included 35 variables.

We linked patients in the National Pneumonia Project cohort to their Medicare claims data, including claims from one year before the index hospitalization, so that we could calculate risk‐standardized readmission rates in this cohort separately using medical record and claims‐based models. This analysis was conducted at the state level, for the 50 states plus the District of Columbia and Puerto Rico, because medical record data were unavailable in sufficient numbers to permit hospital‐level comparisons. To examine the relationship between risk‐standardized rates obtained from medical record and administrative data models, we estimated a linear regression model describing the association between the two rates, weighting each state by number of index hospitalizations, and calculated the correlation coefficient and the intercept and slope of this equation. A slope close to 1 and an intercept close to 0 would provide evidence that risk‐standardized state readmission rates from the medical record and claims models were similar. We also calculated the difference between state risk‐standardized readmission rates from the two models.

Analyses were conducted with the use of SAS version 9.1.3 (SAS Institute Inc, Cary, NC). Models were fitted separately for the National Pneumonia Project and 2006 cohort. We estimated the hierarchical models using the GLIMMIX procedure in SAS. The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.

Model Derivation and Performance

After exclusions were applied, the 2006 sample included 453,251 pneumonia hospitalizations (Figure 1). The development sample consisted of 226,545 hospitalizations at 4675 hospitals, with an overall unadjusted 30‐day readmission rate of 17.4%. In 11,694 index cases (5.2%), the patient died within 30 days without being readmitted. Median readmission rate was 16.3%, 25th and 75th percentile rates were 11.1% and 21.3%, and at the 10th and 90th percentile, hospital readmission rates ranged from 4.6% to 26.7% (Figure 2).

Figure 1

Figure 2

The claims model included 39 variables (age, sex, and 37 clinical variables) (Table 1). The mean age of the cohort was 80.0 years, with 55.5% women and 11.1% nonwhite patients. Mean observed readmission rate in the development sample ranged from 9% in the lowest decile of predicted pneumonia readmission rates to 32% in the highest predicted decile, a range of 23%. The AUC was 0.63. For comparison, a model with only age and sex had an AUC of 0.51, and a model with all candidate variables had an AUC equal to 0.63 (Table 2).

Hospital Risk‐Standardized Readmission Rates

Risk‐standardized readmission rates varied across hospitals (Figure 3). Median risk‐standardized readmission rate was 17.3%, and the 25th and 75th percentiles were 16.9% and 17.9%, respectively. The 5th percentile was 16.0% and the 95th percentile was 19.1%. Odds of readmission for a hospital one standard deviation above average was 1.4 times that of a hospital one standard deviation below average.

Figure 3

Administrative Model Validation

In the remaining 50% of pneumonia index hospitalizations from 2006, and the entire 2005 cohort, regression coefficients and standard errors of model variables were similar to those in the development data set. Model performance using 2005 data was consistent with model performance using the 2006 development and validation half‐samples (Table 2).

Medical Record Validation

After exclusions, the medical record sample taken from the National Pneumonia Project included 47,429 cases, with an unadjusted 30‐day readmission rate of 17.0%. The final medical record risk‐adjustment model included a total of 35 variables, whose prevalence and association with readmission risk varied modestly (Table 3). Performance of the medical record and administrative models was similar (areas under the ROC curve 0.59 and 0.63, respectively) (Table 4). Additionally, in the administrative model, predicted readmission rates ranged from 8% in the lowest predicted decile to 30% in the highest predicted decile, while in the medical record model, the corresponding rates varied from 10% to 26%.

The correlation coefficient of the estimated state‐specific standardized readmission rates from the administrative and medical record models was 0.96, and the proportion of the variance explained by the model was 0.92 (Figure 4).

Figure 4

Design, Setting, Subjects

We conducted a cross‐sectional study at the hospital level of the outcomes of care of fee‐for‐service patients hospitalized for pneumonia between July 2006 and June 2009. Patients are eligible to be included in the measures if they are 65 years or older, have a principal diagnosis of pneumonia (International Classification of Diseases, Ninth Revision, Clinical Modification codes 480.X, 481, 482.XX, 483.X, 485, 486, and 487.0), and are cared for at a nonfederal acute care hospital in the US and its organized territories, including Puerto Rico, Guam, the US Virgin Islands, and the Northern Mariana Islands.

The mortality measure excludes patients enrolled in the Medicare hospice program in the year prior to, or on the day of admission, those in whom pneumonia is listed as a secondary diagnosis (to eliminate cases resulting from complications of hospitalization), those discharged against medical advice, and patients who are discharged alive but whose length of stay in the hospital is less than 1 day (because of concerns about the accuracy of the pneumonia diagnosis). Patients are also excluded if their administrative records for the period of analysis (1 year prior to hospitalization and 30 days following discharge) were not available or were incomplete, because these are needed to assess comorbid illness and outcomes. The readmission measure is similar, but does not exclude patients on the basis of hospice program enrollment (because these patients have been admitted and readmissions for hospice patients are likely unplanned events that can be measured and reduced), nor on the basis of hospital length of stay (because patients discharged within 24 hours may be at a heightened risk of readmission).11, 12

Information about patient comorbidities is derived from diagnoses recorded in the year prior to the index hospitalization as found in Medicare inpatient, outpatient, and carrier (physician) standard analytic files. Comorbidities are identified using the Condition Categories of the Hierarchical Condition Category grouper, which sorts the more than 15,000 possible diagnostic codes into 189 clinically‐coherent conditions and which was originally developed to support risk‐adjusted payments within Medicare managed care.13

Outcomes

The patient outcomes assessed include death from any cause within 30 days of admission and readmission for any cause within 30 days of discharge. All‐cause, rather than disease‐specific, readmission was chosen because hospital readmission as a consequence of suboptimal inpatient care or discharge coordination may manifest in many different diagnoses, and no validated method is available to distinguish related from unrelated readmissions. The measures use the Medicare Enrollment Database to determine mortality status, and acute care hospital inpatient claims are used to identify readmission events. For patients with multiple hospitalizations during the study period, the mortality measure randomly selects one hospitalization to use for determination of mortality. Admissions that are counted as readmissions (i.e., those that occurred within 30 days of discharge following hospitalization for pneumonia) are not also treated as index hospitalizations. In the case of patients who are transferred to or from another acute care facility, responsibility for deaths is assigned to the hospital that initially admitted the patient, while responsibility for readmissions is assigned to the hospital that ultimately discharges the patient to a nonacute setting (e.g., home, skilled nursing facilities).

Risk‐Standardization Methods

Hierarchical logistic regression is used to model the log‐odds of mortality or readmission within 30 days of admission or discharge from an index pneumonia admission as a function of patient demographic and clinical characteristics and a random hospital‐specific intercept. This strategy accounts for within‐hospital correlation of the observed outcomes, and reflects the assumption that underlying differences in quality among the hospitals being evaluated lead to systematic differences in outcomes. In contrast to nonhierarchical models which ignore hospital effects, this method attempts to measure the influence of the hospital on patient outcome after adjusting for patient characteristics. Comorbidities from the index admission that could represent potential complications of care are not included in the model unless they are also documented in the 12 months prior to admission. Hospital‐specific mortality and readmission rates are calculated as the ratio of predicted‐to‐expected events (similar to the observed/expected ratio), multiplied by the national unadjusted rate, a form of indirect standardization.

The model for mortality has a c‐statistic of 0.72 whereas a model based on medical record review that was developed for validation purposes had a c‐statistic of 0.77. The model for readmission has a c‐statistic of 0.63 whereas a model based on medical review had a c‐statistic of 0.59. The mortality and readmission models produce similar state‐level mortality and readmission rate estimates as the models derived from medical record review, and can therefore serve as reasonable surrogates. These methods, including their development and validation, have been described fully elsewhere,14, 15 and have been evaluated and subsequently endorsed by the National Quality Forum.16

Identification of Geographic Regions

To characterize patterns of performance geographically we identified the 306 hospital referral regions for each hospital in our analysis using definitions provided by the Dartmouth Atlas of Health Care project. Unlike a hospital‐level analysis, the hospital referral regions represent regional markets for tertiary care and are widely used to summarize variation in medical care inputs, utilization patterns, and health outcomes and provide a more detailed look at variation in outcomes than results at the state level.17

Analyses

Summary statistics were constructed using frequencies and proportions for categorical data, and means, medians and interquartile ranges for continuous variables. To characterize 30‐day risk‐standardized mortality and readmission rates at the hospital‐referral region level, we calculated means and percentiles by weighting each hospital's value by the inverse of the variance of the hospital's estimated rate. Hospitals with larger sample sizes, and therefore more precise estimates, lend more weight to the average. Hierarchical models were estimated using the SAS GLIMMIX procedure. Bayesian shrinkage was used to estimate rates in order to take into account the greater uncertainty in the true rates of hospitals with small caseloads. Using this technique, estimated rates at low volume institutions are shrunken toward the population mean, while hospitals with large caseloads have a relatively smaller amount of shrinkage and the estimate is closer to the hospital's observed rate.18

To determine whether a hospital's performance is significantly different than the national rate we measured whether the 95% interval estimate for the risk‐standardized rate overlapped with the national crude mortality or readmission rate. This information is used to categorize hospitals on Hospital Compare as better than the US national rate, worse than the US national rate, or no different than the US national rate. Hospitals with fewer than 25 cases in the 3‐year period, are excluded from this categorization on Hospital Compare.

Analyses were conducted with the use of SAS 9.1.3 (SAS Institute Inc, Cary, NC). We created the hospital referral region maps using ArcGIS version 9.3 (ESRI, Redlands, CA). The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.

Hospital‐Specific Risk‐Standardized 30‐Day Mortality and Readmission Rates

Of the 1,118,583 patients included in the mortality analysis 129,444 (11.6%) died within 30 days of hospital admission. The median (Q1, Q3) hospital 30‐day risk‐standardized mortality rate was 11.1% (10.0%, 12.3%), and ranged from 6.7% to 20.9% (Table 1, Figure 1). Hospitals at the 10th percentile had 30‐day risk‐standardized mortality rates of 9.0% while for those at the 90th percentile of performance the rate was 13.5%. The odds of all‐cause mortality for a patient treated at a hospital that was one standard deviation above the national average was 1.68 times higher than that of a patient treated at a hospital that was one standard deviation below the national average.

Figure 1

For the 3‐year period 2006 to 2009, 222 (4.7%) hospitals were categorized as having a mortality rate that was better than the national average, 3968 (83.7%) were no different than the national average, 221 (4.6%) were worse and 332 (7.0%) did not meet the minimum case threshold.

Among the 1,161,817 patients included in the readmission analysis 212,638 (18.3%) were readmitted within 30 days of hospital discharge. The median (Q1,Q3) 30‐day risk‐standardized readmission rate was 18.2% (17.2%, 19.2%) and ranged from 13.6% to 26.7% (Table 1, Figure 2). Hospitals at the 10th percentile had 30‐day risk‐standardized readmission rates of 16.4% while for those at the 90th percentile of performance the rate was 20.4%. The odds of all‐cause readmission for a patient treated at a hospital that was one standard deviation above the national average was 1.40 times higher than the odds of all‐cause readmission if treated at a hospital that was one standard deviation below the national average.

Figure 2

For the 3‐year period 2006 to 2009, 64 (1.3%) hospitals were categorized as having a readmission rate that was better than the national average, 4203 (88.2%) were no different than the national average, 163 (3.4%) were worse and 333 (7.0%) had less than 25 cases and were therefore not categorized.

While risk‐standardized readmission rates were substantially higher than risk‐standardized mortality rates, mortality rates varied more. For example, the top 10% of hospitals had a relative mortality rate that was 33% lower than those in the bottom 10%, as compared with just a 20% relative difference for readmission rates. The coefficient of variation, a normalized measure of dispersion that unlike the standard deviation is independent of the population mean, was 10.7 for risk‐standardized mortality rates and 4.9 for readmission rates.

Regional Risk‐Standardized 30‐Day Mortality and Readmission Rates

Figures 3 and 4 show the distribution of 30‐day risk‐standardized mortality and readmission rates among hospital referral regions by quintile. Highest mortality regions were found across the entire country, including parts of Northern New England, the Mid and South Atlantic, East and the West South Central, East and West North Central, and the Mountain and Pacific regions of the West. The lowest mortality rates were observed in Southern New England, parts of the Mid and South Atlantic, East and West South Central, and parts of the Mountain and Pacific regions of the West (Figure 3).

Figure 3

Figure 4

Readmission rates were higher in the eastern portions of the US (including the Northeast, Mid and South Atlantic, East South Central) as well as the East North Central, and small parts of the West North Central portions of the Midwest and in Central California. The lowest readmission rates were observed in the West (Mountain and Pacific regions), parts of the Midwest (East and West North Central) and small pockets within the South and Northeast (Figure 4).

Design, Setting, Subjects

We conducted a cross‐sectional study at the hospital level of the outcomes of care of fee‐for‐service patients hospitalized for pneumonia between July 2006 and June 2009. Patients are eligible to be included in the measures if they are 65 years or older, have a principal diagnosis of pneumonia (International Classification of Diseases, Ninth Revision, Clinical Modification codes 480.X, 481, 482.XX, 483.X, 485, 486, and 487.0), and are cared for at a nonfederal acute care hospital in the US and its organized territories, including Puerto Rico, Guam, the US Virgin Islands, and the Northern Mariana Islands.

The mortality measure excludes patients enrolled in the Medicare hospice program in the year prior to, or on the day of admission, those in whom pneumonia is listed as a secondary diagnosis (to eliminate cases resulting from complications of hospitalization), those discharged against medical advice, and patients who are discharged alive but whose length of stay in the hospital is less than 1 day (because of concerns about the accuracy of the pneumonia diagnosis). Patients are also excluded if their administrative records for the period of analysis (1 year prior to hospitalization and 30 days following discharge) were not available or were incomplete, because these are needed to assess comorbid illness and outcomes. The readmission measure is similar, but does not exclude patients on the basis of hospice program enrollment (because these patients have been admitted and readmissions for hospice patients are likely unplanned events that can be measured and reduced), nor on the basis of hospital length of stay (because patients discharged within 24 hours may be at a heightened risk of readmission).11, 12

Information about patient comorbidities is derived from diagnoses recorded in the year prior to the index hospitalization as found in Medicare inpatient, outpatient, and carrier (physician) standard analytic files. Comorbidities are identified using the Condition Categories of the Hierarchical Condition Category grouper, which sorts the more than 15,000 possible diagnostic codes into 189 clinically‐coherent conditions and which was originally developed to support risk‐adjusted payments within Medicare managed care.13

Outcomes

The patient outcomes assessed include death from any cause within 30 days of admission and readmission for any cause within 30 days of discharge. All‐cause, rather than disease‐specific, readmission was chosen because hospital readmission as a consequence of suboptimal inpatient care or discharge coordination may manifest in many different diagnoses, and no validated method is available to distinguish related from unrelated readmissions. The measures use the Medicare Enrollment Database to determine mortality status, and acute care hospital inpatient claims are used to identify readmission events. For patients with multiple hospitalizations during the study period, the mortality measure randomly selects one hospitalization to use for determination of mortality. Admissions that are counted as readmissions (i.e., those that occurred within 30 days of discharge following hospitalization for pneumonia) are not also treated as index hospitalizations. In the case of patients who are transferred to or from another acute care facility, responsibility for deaths is assigned to the hospital that initially admitted the patient, while responsibility for readmissions is assigned to the hospital that ultimately discharges the patient to a nonacute setting (e.g., home, skilled nursing facilities).

Risk‐Standardization Methods

Hierarchical logistic regression is used to model the log‐odds of mortality or readmission within 30 days of admission or discharge from an index pneumonia admission as a function of patient demographic and clinical characteristics and a random hospital‐specific intercept. This strategy accounts for within‐hospital correlation of the observed outcomes, and reflects the assumption that underlying differences in quality among the hospitals being evaluated lead to systematic differences in outcomes. In contrast to nonhierarchical models which ignore hospital effects, this method attempts to measure the influence of the hospital on patient outcome after adjusting for patient characteristics. Comorbidities from the index admission that could represent potential complications of care are not included in the model unless they are also documented in the 12 months prior to admission. Hospital‐specific mortality and readmission rates are calculated as the ratio of predicted‐to‐expected events (similar to the observed/expected ratio), multiplied by the national unadjusted rate, a form of indirect standardization.

The model for mortality has a c‐statistic of 0.72 whereas a model based on medical record review that was developed for validation purposes had a c‐statistic of 0.77. The model for readmission has a c‐statistic of 0.63 whereas a model based on medical review had a c‐statistic of 0.59. The mortality and readmission models produce similar state‐level mortality and readmission rate estimates as the models derived from medical record review, and can therefore serve as reasonable surrogates. These methods, including their development and validation, have been described fully elsewhere,14, 15 and have been evaluated and subsequently endorsed by the National Quality Forum.16

Identification of Geographic Regions

To characterize patterns of performance geographically we identified the 306 hospital referral regions for each hospital in our analysis using definitions provided by the Dartmouth Atlas of Health Care project. Unlike a hospital‐level analysis, the hospital referral regions represent regional markets for tertiary care and are widely used to summarize variation in medical care inputs, utilization patterns, and health outcomes and provide a more detailed look at variation in outcomes than results at the state level.17

Analyses

Summary statistics were constructed using frequencies and proportions for categorical data, and means, medians and interquartile ranges for continuous variables. To characterize 30‐day risk‐standardized mortality and readmission rates at the hospital‐referral region level, we calculated means and percentiles by weighting each hospital's value by the inverse of the variance of the hospital's estimated rate. Hospitals with larger sample sizes, and therefore more precise estimates, lend more weight to the average. Hierarchical models were estimated using the SAS GLIMMIX procedure. Bayesian shrinkage was used to estimate rates in order to take into account the greater uncertainty in the true rates of hospitals with small caseloads. Using this technique, estimated rates at low volume institutions are shrunken toward the population mean, while hospitals with large caseloads have a relatively smaller amount of shrinkage and the estimate is closer to the hospital's observed rate.18

To determine whether a hospital's performance is significantly different than the national rate we measured whether the 95% interval estimate for the risk‐standardized rate overlapped with the national crude mortality or readmission rate. This information is used to categorize hospitals on Hospital Compare as better than the US national rate, worse than the US national rate, or no different than the US national rate. Hospitals with fewer than 25 cases in the 3‐year period, are excluded from this categorization on Hospital Compare.

Analyses were conducted with the use of SAS 9.1.3 (SAS Institute Inc, Cary, NC). We created the hospital referral region maps using ArcGIS version 9.3 (ESRI, Redlands, CA). The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.

Hospital‐Specific Risk‐Standardized 30‐Day Mortality and Readmission Rates

Of the 1,118,583 patients included in the mortality analysis 129,444 (11.6%) died within 30 days of hospital admission. The median (Q1, Q3) hospital 30‐day risk‐standardized mortality rate was 11.1% (10.0%, 12.3%), and ranged from 6.7% to 20.9% (Table 1, Figure 1). Hospitals at the 10th percentile had 30‐day risk‐standardized mortality rates of 9.0% while for those at the 90th percentile of performance the rate was 13.5%. The odds of all‐cause mortality for a patient treated at a hospital that was one standard deviation above the national average was 1.68 times higher than that of a patient treated at a hospital that was one standard deviation below the national average.

Figure 1

For the 3‐year period 2006 to 2009, 222 (4.7%) hospitals were categorized as having a mortality rate that was better than the national average, 3968 (83.7%) were no different than the national average, 221 (4.6%) were worse and 332 (7.0%) did not meet the minimum case threshold.

Among the 1,161,817 patients included in the readmission analysis 212,638 (18.3%) were readmitted within 30 days of hospital discharge. The median (Q1,Q3) 30‐day risk‐standardized readmission rate was 18.2% (17.2%, 19.2%) and ranged from 13.6% to 26.7% (Table 1, Figure 2). Hospitals at the 10th percentile had 30‐day risk‐standardized readmission rates of 16.4% while for those at the 90th percentile of performance the rate was 20.4%. The odds of all‐cause readmission for a patient treated at a hospital that was one standard deviation above the national average was 1.40 times higher than the odds of all‐cause readmission if treated at a hospital that was one standard deviation below the national average.

Figure 2

For the 3‐year period 2006 to 2009, 64 (1.3%) hospitals were categorized as having a readmission rate that was better than the national average, 4203 (88.2%) were no different than the national average, 163 (3.4%) were worse and 333 (7.0%) had less than 25 cases and were therefore not categorized.

While risk‐standardized readmission rates were substantially higher than risk‐standardized mortality rates, mortality rates varied more. For example, the top 10% of hospitals had a relative mortality rate that was 33% lower than those in the bottom 10%, as compared with just a 20% relative difference for readmission rates. The coefficient of variation, a normalized measure of dispersion that unlike the standard deviation is independent of the population mean, was 10.7 for risk‐standardized mortality rates and 4.9 for readmission rates.

Regional Risk‐Standardized 30‐Day Mortality and Readmission Rates

Figures 3 and 4 show the distribution of 30‐day risk‐standardized mortality and readmission rates among hospital referral regions by quintile. Highest mortality regions were found across the entire country, including parts of Northern New England, the Mid and South Atlantic, East and the West South Central, East and West North Central, and the Mountain and Pacific regions of the West. The lowest mortality rates were observed in Southern New England, parts of the Mid and South Atlantic, East and West South Central, and parts of the Mountain and Pacific regions of the West (Figure 3).

Figure 3

Figure 4

Readmission rates were higher in the eastern portions of the US (including the Northeast, Mid and South Atlantic, East South Central) as well as the East North Central, and small parts of the West North Central portions of the Midwest and in Central California. The lowest readmission rates were observed in the West (Mountain and Pacific regions), parts of the Midwest (East and West North Central) and small pockets within the South and Northeast (Figure 4).

Design, Setting, Subjects

We conducted a cross‐sectional study at the hospital level of the outcomes of care of fee‐for‐service patients hospitalized for pneumonia between July 2006 and June 2009. Patients are eligible to be included in the measures if they are 65 years or older, have a principal diagnosis of pneumonia (International Classification of Diseases, Ninth Revision, Clinical Modification codes 480.X, 481, 482.XX, 483.X, 485, 486, and 487.0), and are cared for at a nonfederal acute care hospital in the US and its organized territories, including Puerto Rico, Guam, the US Virgin Islands, and the Northern Mariana Islands.

The mortality measure excludes patients enrolled in the Medicare hospice program in the year prior to, or on the day of admission, those in whom pneumonia is listed as a secondary diagnosis (to eliminate cases resulting from complications of hospitalization), those discharged against medical advice, and patients who are discharged alive but whose length of stay in the hospital is less than 1 day (because of concerns about the accuracy of the pneumonia diagnosis). Patients are also excluded if their administrative records for the period of analysis (1 year prior to hospitalization and 30 days following discharge) were not available or were incomplete, because these are needed to assess comorbid illness and outcomes. The readmission measure is similar, but does not exclude patients on the basis of hospice program enrollment (because these patients have been admitted and readmissions for hospice patients are likely unplanned events that can be measured and reduced), nor on the basis of hospital length of stay (because patients discharged within 24 hours may be at a heightened risk of readmission).11, 12

Information about patient comorbidities is derived from diagnoses recorded in the year prior to the index hospitalization as found in Medicare inpatient, outpatient, and carrier (physician) standard analytic files. Comorbidities are identified using the Condition Categories of the Hierarchical Condition Category grouper, which sorts the more than 15,000 possible diagnostic codes into 189 clinically‐coherent conditions and which was originally developed to support risk‐adjusted payments within Medicare managed care.13

Outcomes

The patient outcomes assessed include death from any cause within 30 days of admission and readmission for any cause within 30 days of discharge. All‐cause, rather than disease‐specific, readmission was chosen because hospital readmission as a consequence of suboptimal inpatient care or discharge coordination may manifest in many different diagnoses, and no validated method is available to distinguish related from unrelated readmissions. The measures use the Medicare Enrollment Database to determine mortality status, and acute care hospital inpatient claims are used to identify readmission events. For patients with multiple hospitalizations during the study period, the mortality measure randomly selects one hospitalization to use for determination of mortality. Admissions that are counted as readmissions (i.e., those that occurred within 30 days of discharge following hospitalization for pneumonia) are not also treated as index hospitalizations. In the case of patients who are transferred to or from another acute care facility, responsibility for deaths is assigned to the hospital that initially admitted the patient, while responsibility for readmissions is assigned to the hospital that ultimately discharges the patient to a nonacute setting (e.g., home, skilled nursing facilities).

Risk‐Standardization Methods

Hierarchical logistic regression is used to model the log‐odds of mortality or readmission within 30 days of admission or discharge from an index pneumonia admission as a function of patient demographic and clinical characteristics and a random hospital‐specific intercept. This strategy accounts for within‐hospital correlation of the observed outcomes, and reflects the assumption that underlying differences in quality among the hospitals being evaluated lead to systematic differences in outcomes. In contrast to nonhierarchical models which ignore hospital effects, this method attempts to measure the influence of the hospital on patient outcome after adjusting for patient characteristics. Comorbidities from the index admission that could represent potential complications of care are not included in the model unless they are also documented in the 12 months prior to admission. Hospital‐specific mortality and readmission rates are calculated as the ratio of predicted‐to‐expected events (similar to the observed/expected ratio), multiplied by the national unadjusted rate, a form of indirect standardization.

The model for mortality has a c‐statistic of 0.72 whereas a model based on medical record review that was developed for validation purposes had a c‐statistic of 0.77. The model for readmission has a c‐statistic of 0.63 whereas a model based on medical review had a c‐statistic of 0.59. The mortality and readmission models produce similar state‐level mortality and readmission rate estimates as the models derived from medical record review, and can therefore serve as reasonable surrogates. These methods, including their development and validation, have been described fully elsewhere,14, 15 and have been evaluated and subsequently endorsed by the National Quality Forum.16

Identification of Geographic Regions

To characterize patterns of performance geographically we identified the 306 hospital referral regions for each hospital in our analysis using definitions provided by the Dartmouth Atlas of Health Care project. Unlike a hospital‐level analysis, the hospital referral regions represent regional markets for tertiary care and are widely used to summarize variation in medical care inputs, utilization patterns, and health outcomes and provide a more detailed look at variation in outcomes than results at the state level.17

Analyses

Summary statistics were constructed using frequencies and proportions for categorical data, and means, medians and interquartile ranges for continuous variables. To characterize 30‐day risk‐standardized mortality and readmission rates at the hospital‐referral region level, we calculated means and percentiles by weighting each hospital's value by the inverse of the variance of the hospital's estimated rate. Hospitals with larger sample sizes, and therefore more precise estimates, lend more weight to the average. Hierarchical models were estimated using the SAS GLIMMIX procedure. Bayesian shrinkage was used to estimate rates in order to take into account the greater uncertainty in the true rates of hospitals with small caseloads. Using this technique, estimated rates at low volume institutions are shrunken toward the population mean, while hospitals with large caseloads have a relatively smaller amount of shrinkage and the estimate is closer to the hospital's observed rate.18

To determine whether a hospital's performance is significantly different than the national rate we measured whether the 95% interval estimate for the risk‐standardized rate overlapped with the national crude mortality or readmission rate. This information is used to categorize hospitals on Hospital Compare as better than the US national rate, worse than the US national rate, or no different than the US national rate. Hospitals with fewer than 25 cases in the 3‐year period, are excluded from this categorization on Hospital Compare.

Analyses were conducted with the use of SAS 9.1.3 (SAS Institute Inc, Cary, NC). We created the hospital referral region maps using ArcGIS version 9.3 (ESRI, Redlands, CA). The Human Investigation Committee at the Yale School of Medicine approved an exemption for the authors to use CMS claims and enrollment data for research analyses and publication.

Hospital‐Specific Risk‐Standardized 30‐Day Mortality and Readmission Rates

Of the 1,118,583 patients included in the mortality analysis 129,444 (11.6%) died within 30 days of hospital admission. The median (Q1, Q3) hospital 30‐day risk‐standardized mortality rate was 11.1% (10.0%, 12.3%), and ranged from 6.7% to 20.9% (Table 1, Figure 1). Hospitals at the 10th percentile had 30‐day risk‐standardized mortality rates of 9.0% while for those at the 90th percentile of performance the rate was 13.5%. The odds of all‐cause mortality for a patient treated at a hospital that was one standard deviation above the national average was 1.68 times higher than that of a patient treated at a hospital that was one standard deviation below the national average.

Figure 1

For the 3‐year period 2006 to 2009, 222 (4.7%) hospitals were categorized as having a mortality rate that was better than the national average, 3968 (83.7%) were no different than the national average, 221 (4.6%) were worse and 332 (7.0%) did not meet the minimum case threshold.

Among the 1,161,817 patients included in the readmission analysis 212,638 (18.3%) were readmitted within 30 days of hospital discharge. The median (Q1,Q3) 30‐day risk‐standardized readmission rate was 18.2% (17.2%, 19.2%) and ranged from 13.6% to 26.7% (Table 1, Figure 2). Hospitals at the 10th percentile had 30‐day risk‐standardized readmission rates of 16.4% while for those at the 90th percentile of performance the rate was 20.4%. The odds of all‐cause readmission for a patient treated at a hospital that was one standard deviation above the national average was 1.40 times higher than the odds of all‐cause readmission if treated at a hospital that was one standard deviation below the national average.

Figure 2

For the 3‐year period 2006 to 2009, 64 (1.3%) hospitals were categorized as having a readmission rate that was better than the national average, 4203 (88.2%) were no different than the national average, 163 (3.4%) were worse and 333 (7.0%) had less than 25 cases and were therefore not categorized.

While risk‐standardized readmission rates were substantially higher than risk‐standardized mortality rates, mortality rates varied more. For example, the top 10% of hospitals had a relative mortality rate that was 33% lower than those in the bottom 10%, as compared with just a 20% relative difference for readmission rates. The coefficient of variation, a normalized measure of dispersion that unlike the standard deviation is independent of the population mean, was 10.7 for risk‐standardized mortality rates and 4.9 for readmission rates.

Regional Risk‐Standardized 30‐Day Mortality and Readmission Rates

Figures 3 and 4 show the distribution of 30‐day risk‐standardized mortality and readmission rates among hospital referral regions by quintile. Highest mortality regions were found across the entire country, including parts of Northern New England, the Mid and South Atlantic, East and the West South Central, East and West North Central, and the Mountain and Pacific regions of the West. The lowest mortality rates were observed in Southern New England, parts of the Mid and South Atlantic, East and West South Central, and parts of the Mountain and Pacific regions of the West (Figure 3).

Figure 3

Figure 4

Readmission rates were higher in the eastern portions of the US (including the Northeast, Mid and South Atlantic, East South Central) as well as the East North Central, and small parts of the West North Central portions of the Midwest and in Central California. The lowest readmission rates were observed in the West (Mountain and Pacific regions), parts of the Midwest (East and West North Central) and small pockets within the South and Northeast (Figure 4).