Michael S. Leonard, MD, MS

The patient experience has become increasingly important to healthcare in the United States. It is now a metric used commonly to determine physician compensation and accounts for nearly 30% of the Centers for Medicare and Medicaid Services' (CMS) Value‐Based Purchasing (VBP) reimbursement for fiscal years 2015 and 2016.[1, 2]

In April 2015, CMS added a 5‐star patient experience score to its Hospital Compare website in an attempt to address the Affordable Care Act's call for transparent and easily understandable public reporting.[3] A hospital's principal score is the Summary Star Rating, which is based on responses to the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey. The formulas used to calculate Summary Star Ratings have been reported by CMS.[4]

Studies published over the past decade suggest that gender, age, education level, length of hospital stay, travel distance, and other factors may influence patient satisfaction.[5, 6, 7, 8] One study utilizing a national dataset suggested that higher patient satisfaction was associated with greater inpatient healthcare utilization and higher healthcare expenditures.[9] It is therefore possible that emphasizing patient experience scores could adversely impact healthcare resource utilization. However, positive patient experience may also be an important independent dimension of quality for patients and correlate with improved clinical outcomes.[10]

We know of no literature describing patient factors associated with the Summary Star Rating. Given that this rating is now used as a standard metric by which patient experience can be compared across more than 3,500 hospitals,[11] data describing the association between patient‐level factors and the Summary Star Rating may provide hospitals with an opportunity to target improvement efforts. We aimed to determine the degree to which resource utilization is associated with a satisfaction score based on the Summary Star Rating methodology.

METHODS

The study was conducted at the University of Rochester Medical Center (URMC), an 830‐bed tertiary care center in upstate New York. This was a retrospective review of all HCAHPS surveys returned to URMC over a 27‐month period from January 1, 2012 to April 1, 2014. URMC follows the standard CMS process for determining which patients receive surveys as follows. During the study timeframe, HCAHPS surveys were mailed to patients 18 years of age and older who had an inpatient stay spanning at least 1 midnight. Surveys were mailed within 5 days of discharge, and were generally returned within 6 weeks. URMC did not utilize telephone or email surveys during the study period. Surveys were not sent to patients who (1) were transferred to another facility, (2) were discharged to hospice, (3) died during the hospitalization, (4) received psychiatric or rehabilitative services during the hospitalization, (5) had an international address, and/or (6) were prisoners.

The survey vendor (Press Ganey, South Bend, IN) for URMC provided raw data for returned surveys with patient answers to questions. Administrative and billing databases were used to add demographic and clinical data for the corresponding hospitalization to the dataset. These data included age, gender, payer status (public, private, self, charity), length of stay, number of attendings who saw the patient (based on encounters documented in the electronic medical record (EMR)), all discharge International Classification of Diseases, 9th Revision (ICD‐9) diagnoses for the hospitalization, total charges for the hospitalization, and intensive care unit (ICU) utilization as evidenced by a documented encounter with a member of the Division of Critical Care/Pulmonary Medicine.

CMS analyzes surveys within 1 of 3 clinical service categories (medical, surgical, or obstetrics/gynecology) based on the discharging service. To parallel this approach, each returned survey was placed into 1 of these categories based on the clinical service of the discharging physician. Patients placed in the obstetrics/gynecology category (n = 1317, 13%) will be analyzed in a future analysis given inherent differences in patient characteristics that require evaluation of other variables.

RESULTS

Our initial search identified 10,007 returned surveys (29% of eligible patients returned surveys during the study period). Of these, 5059 (51%) were categorized as medical, 3630 (36%) as surgical, and 1317 (13%) as obstetrics/gynecology. One survey did not have the service of the discharging physician recorded and was excluded. Cohort demographics and relationship to RSRs in the top decile for the 8689 medical and surgical patients can be found in Table 1. The most common discharge diagnosis‐related groups (DRGs) for medical patients were 247, percutaneous cardiovascular procedure with drug‐eluding stent without major complications or comorbidities (MCC) (3.8%); 871, septicemia or severe sepsis without mechanical ventilation >96 hours with MCC (2.7%); and 392, esophagitis, gastroenteritis, and miscellaneous digestive disorders with MCC (2.3%). The most common DRGs for surgical patients were 460, spinal fusion except cervical without MCC (3.5%); 328, stomach, esophageal and duodenal procedure without complication or comorbidities or MCC (3.3%); and 491, back and neck procedure excluding spinal fusion without complication or comorbidities or MCC (3.1%).

Unadjusted analysis of medical and surgical patients identified significant associations of several variables with a top decile RSR (Table 2). Patients with longer lengths of stay (OR: 2.07, 95% CI: 1.72‐2.48), more attendings (OR: 1.44, 95% CI: 1.19‐1.73), and higher hospital charges (OR: 2.76, 95% CI: 2.19‐3.47) were more likely to report an RSR in the top decile. Patients without an ICU encounter (OR: 0.65, 95% CI: 0.55‐0.77) and on a medical service (OR: 0.57, 95% CI: 0.5‐ 0.66) were less likely to report an RSR in the top decile. Several associations were identified in only the medical or surgical cohorts. In the medical cohort, patients with the highest illness severity index (OR: 1.68, 95% CI: 1.12‐ 2.52) and with 7 different attending physicians (OR: 1.66, 95% CI: 1.27‐2.18) were more likely to report RSRs in the top decile. In the surgical cohort, patients <30 years of age (OR: 2.05, 95% CI 1.38‐3.07) were more likely to report an RSR in the top decile than patients >69 years of age. Insurance payer category and gender were not significantly associated with top decile RSRs.

Multivariable modeling (Table 3) for all patients without an ICU encounter suggested that (1) patients aged <30 years, 30 to 49 years, and 50 to 69 years were more likely to report top decile RSRs when compared to patients 70 years and older (OR: 1.61, 95% CI: 1.09‐2.36; OR: 1.44, 95% CI: 1.08‐1.93; and OR: 1.39, 95% CI: 1.13‐1.71, respectively) and (2), when compared to patients with extreme resource intensity scores, patients with higher resource intensity scores were more likely to report top decile RSRs (moderate [OR: 1.42, 95% CI: 1.11‐1.83], major [OR: 1.56, 95% CI: 1.22‐2.01], and extreme [OR: 2.29, 95% CI: 1.8‐2.92]. These results were relatively consistent within medical and surgical subgroups (Table 3).

In those with at least 1 ICU attending encounter (see Supporting Table 1 in the online version of this article), no variables demonstrated significant association with top decile RSRs in the overall group or in the medical subgroup. For surgical patients with at least 1 ICU attending encounter (see Supporting Table 1 in the online version of this article), patients aged 30 to 49 and 50 to 69 years were more likely to provide top decile RSRs (OR: 1.93, 95% CI: 1.08‐3.46 and OR: 1.65, 95% CI 1.07‐2.53, respectively). Resource intensity was not significantly associated with top decile RSRs.

DISCUSSION

Our analysis suggests that, for patients on the general care floors, resource utilization is associated with the RSR and, therefore, potentially the CMS Summary Star Rating. Adjusting for severity of illness, patients with higher resource utilization were more likely to report top decile RSRs.

Prior data regarding utilization and satisfaction are mixed. In a 2‐year, prospective, national examination, patients in the highest quartile of patient satisfaction had increased healthcare and prescription drug expenditures as well as increased rates of hospitalization when compared with patients in the lowest quartile of patient satisfaction.[9] However, a recent national study of surgical administrative databases suggested hospitals with high patient satisfaction provided more efficient care.[13]

One reason for the conflicting data may be that large, national evaluations are unable to control for between‐hospital confounders (ie, hospital quality of care). By capturing all eligible returned surveys at 1 institution, our design allowed us to collect granular data. We found that in 1 hospital setting, patient population, facilities, and food services, patients receiving more clinical resources generally assigned higher ratings than patients receiving less.

It is possible that utilization is a proxy for serious illness, and that patients with serious illness receive more attention during hospitalization and are more satisfied when discharged in a good state of health. However, we did adjust for severity of illness in our model using the Charlson‐Deyo index and we suggest that, other factors being equal, hospitals with higher per‐patient expenditures may be assigned higher Summary Star Ratings.

CMS has recently implemented a number of metrics designed to decrease healthcare costs by improving quality, safety, and efficiency. Concurrently, CMS has also prioritized patient experience. The Summary Star Rating was created to provide healthcare consumers with an easy way to compare the patient experience between hospitals[4]; however, our data suggest that this metric may be at odds with inpatient cost savings and efficiency metrics.

Per‐patient spending becomes particularly salient when considering that in fiscal year 2016, CMS' hospital VBP reimbursement will include 2 metrics: an efficiency outcome measure labeled Medicare spending per beneficiary, and a patient experience outcome measure based on HCAHPS survey dimensions.[2] Together, these 2 metrics will comprise nearly half of the total VBP performance score used to determine reimbursement. Although our data suggest that these 2 VBP metrics may be correlated, it should be noted that we measured inpatient hospital charges, whereas the CMS efficiency outcome measure includes costs for episode of care spanning 3 days prior to hospitalization to 30 days after hospitalization.

Patient expectations likely play a role in satisfaction.[14, 15, 16] In an outpatient setting, physician fulfillment of patient requests has been associated with positive patient evaluations of care.[17] However, patients appear to value education, shared decision making, and provider empathy more than testing and intervention.[14, 18, 19, 20, 21, 22, 23] Perhaps, in the absence of the former attributes, patients use additional resource expenditure as a proxy.

It is not clear that higher resource expenditure improves outcomes. A landmark study of nearly 1 million Medicare enrollees by Fisher et al. suggests that, although Medicare patients in higher‐spending regions receive more care than those in lower‐spending regions, this does not result in better health outcomes, specifically with regard to mortality.[24, 25] Patients who live in areas of high hospital capacity use the hospital more frequently than do patients in areas of low hospital capacity, but this does not appear to result in improved mortality rates.[26] In fact, physicians in areas of high healthcare capacity report more difficulty maintaining high‐quality patient relationships and feel less able to provide high‐quality care than physicians in lower‐capacity areas.[27]

We hypothesize the cause of the association between resource utilization and patient satisfaction could be that patients (1) perceive that a doctor who allows them to stay longer in the hospital or who performs additional testing cares more about their well‐being and (2) that these patients feel more strongly that their concerns are being heard and addressed by their physicians. A systematic review of primary care patients identified many studies that found a positive association between meeting patient expectations and satisfaction with care, but also suggested that although patients frequently expect information, physicians misperceive this as an expectation of specific action.[28] A separate systematic review found that patient education in the form of decision aides can help patients develop more reasonable expectations and reduce utilization of certain discretionary procedures such as elective surgeries and prostate‐specific antigen testing.[29]

We did not specifically address clinical outcomes in our analysis because the clinical outcomes on which CMS currently adjusts VBP reimbursement focus on 30‐day mortality for specific diagnoses, nosocomial infections, and iatrogenic events.[30] Our data include only returned surveys from living patients, and it is likely that 30‐day mortality was similar throughout all subsets of patients. Additionally, the nosocomial and iatrogenic outcome measures used by CMS are sufficiently rare on the general floors and are unlikely to have significantly influenced our results.[31]

Our study has several strengths. Nearly all medical and surgical patient surveys returned during the study period were included, and therefore our calculations are likely to accurately reflect the Summary Star Rating that would have been assigned for the period. Second, the large sample size helps attenuate potential differences in commonly used outcome metrics. Third, by adjusting for a variety of demographic and clinical variables, we were able to decrease the likelihood of unidentified confounders.

Notably, we identified 38 (0.4%) surveys returned for patients under 18 years of age at admission. These surveys were included in our analysis because, to the best of our knowledge, they would have existed in the pool of surveys CMS could have used to assign a Summary Star Rating.

Our study also has limitations. First, geographically diverse data are needed to ensure generalizability. Second, we used the Charlson‐Deyo Comorbidity Index to describe the degree of illness for each patient. This index represents a patient's total illness burden but may not describe the relative severity of the patient's current illness relative to another patient. Third, we selected variables we felt were most likely to be associated with patient experience, but unidentified confounding remains possible. Fourth, attendings caring for ICU patients fall within the Division of Critical Care/Pulmonary Medicine. Therefore, we may have inadvertently placed patients into the ICU cohort who received a pulmonary/critical care consult on the general floors. Fifth, our data describe associations only for patients who returned surveys. Although there may be inherent biases in patients who return surveys, HCAHPS survey responses are used by CMS to determine a hospital's overall satisfaction score.

CONCLUSION

For patients who return HCAHPS surveys, resource utilization may be positively associated with a hospital's Summary Star Rating. These data suggest that hospitals with higher per‐patient expenditures may receive higher Summary Star Ratings, which could result in hospitals with higher per‐patient resource utilization appearing more attractive to healthcare consumers. Future studies should attempt to confirm our findings at other institutions and to determine causative factors.

The patient experience has become increasingly important to healthcare in the United States. It is now a metric used commonly to determine physician compensation and accounts for nearly 30% of the Centers for Medicare and Medicaid Services' (CMS) Value‐Based Purchasing (VBP) reimbursement for fiscal years 2015 and 2016.[1, 2]

In April 2015, CMS added a 5‐star patient experience score to its Hospital Compare website in an attempt to address the Affordable Care Act's call for transparent and easily understandable public reporting.[3] A hospital's principal score is the Summary Star Rating, which is based on responses to the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey. The formulas used to calculate Summary Star Ratings have been reported by CMS.[4]

Studies published over the past decade suggest that gender, age, education level, length of hospital stay, travel distance, and other factors may influence patient satisfaction.[5, 6, 7, 8] One study utilizing a national dataset suggested that higher patient satisfaction was associated with greater inpatient healthcare utilization and higher healthcare expenditures.[9] It is therefore possible that emphasizing patient experience scores could adversely impact healthcare resource utilization. However, positive patient experience may also be an important independent dimension of quality for patients and correlate with improved clinical outcomes.[10]

We know of no literature describing patient factors associated with the Summary Star Rating. Given that this rating is now used as a standard metric by which patient experience can be compared across more than 3,500 hospitals,[11] data describing the association between patient‐level factors and the Summary Star Rating may provide hospitals with an opportunity to target improvement efforts. We aimed to determine the degree to which resource utilization is associated with a satisfaction score based on the Summary Star Rating methodology.

METHODS

The study was conducted at the University of Rochester Medical Center (URMC), an 830‐bed tertiary care center in upstate New York. This was a retrospective review of all HCAHPS surveys returned to URMC over a 27‐month period from January 1, 2012 to April 1, 2014. URMC follows the standard CMS process for determining which patients receive surveys as follows. During the study timeframe, HCAHPS surveys were mailed to patients 18 years of age and older who had an inpatient stay spanning at least 1 midnight. Surveys were mailed within 5 days of discharge, and were generally returned within 6 weeks. URMC did not utilize telephone or email surveys during the study period. Surveys were not sent to patients who (1) were transferred to another facility, (2) were discharged to hospice, (3) died during the hospitalization, (4) received psychiatric or rehabilitative services during the hospitalization, (5) had an international address, and/or (6) were prisoners.

The survey vendor (Press Ganey, South Bend, IN) for URMC provided raw data for returned surveys with patient answers to questions. Administrative and billing databases were used to add demographic and clinical data for the corresponding hospitalization to the dataset. These data included age, gender, payer status (public, private, self, charity), length of stay, number of attendings who saw the patient (based on encounters documented in the electronic medical record (EMR)), all discharge International Classification of Diseases, 9th Revision (ICD‐9) diagnoses for the hospitalization, total charges for the hospitalization, and intensive care unit (ICU) utilization as evidenced by a documented encounter with a member of the Division of Critical Care/Pulmonary Medicine.

CMS analyzes surveys within 1 of 3 clinical service categories (medical, surgical, or obstetrics/gynecology) based on the discharging service. To parallel this approach, each returned survey was placed into 1 of these categories based on the clinical service of the discharging physician. Patients placed in the obstetrics/gynecology category (n = 1317, 13%) will be analyzed in a future analysis given inherent differences in patient characteristics that require evaluation of other variables.

RESULTS

Our initial search identified 10,007 returned surveys (29% of eligible patients returned surveys during the study period). Of these, 5059 (51%) were categorized as medical, 3630 (36%) as surgical, and 1317 (13%) as obstetrics/gynecology. One survey did not have the service of the discharging physician recorded and was excluded. Cohort demographics and relationship to RSRs in the top decile for the 8689 medical and surgical patients can be found in Table 1. The most common discharge diagnosis‐related groups (DRGs) for medical patients were 247, percutaneous cardiovascular procedure with drug‐eluding stent without major complications or comorbidities (MCC) (3.8%); 871, septicemia or severe sepsis without mechanical ventilation >96 hours with MCC (2.7%); and 392, esophagitis, gastroenteritis, and miscellaneous digestive disorders with MCC (2.3%). The most common DRGs for surgical patients were 460, spinal fusion except cervical without MCC (3.5%); 328, stomach, esophageal and duodenal procedure without complication or comorbidities or MCC (3.3%); and 491, back and neck procedure excluding spinal fusion without complication or comorbidities or MCC (3.1%).

Unadjusted analysis of medical and surgical patients identified significant associations of several variables with a top decile RSR (Table 2). Patients with longer lengths of stay (OR: 2.07, 95% CI: 1.72‐2.48), more attendings (OR: 1.44, 95% CI: 1.19‐1.73), and higher hospital charges (OR: 2.76, 95% CI: 2.19‐3.47) were more likely to report an RSR in the top decile. Patients without an ICU encounter (OR: 0.65, 95% CI: 0.55‐0.77) and on a medical service (OR: 0.57, 95% CI: 0.5‐ 0.66) were less likely to report an RSR in the top decile. Several associations were identified in only the medical or surgical cohorts. In the medical cohort, patients with the highest illness severity index (OR: 1.68, 95% CI: 1.12‐ 2.52) and with 7 different attending physicians (OR: 1.66, 95% CI: 1.27‐2.18) were more likely to report RSRs in the top decile. In the surgical cohort, patients <30 years of age (OR: 2.05, 95% CI 1.38‐3.07) were more likely to report an RSR in the top decile than patients >69 years of age. Insurance payer category and gender were not significantly associated with top decile RSRs.

Multivariable modeling (Table 3) for all patients without an ICU encounter suggested that (1) patients aged <30 years, 30 to 49 years, and 50 to 69 years were more likely to report top decile RSRs when compared to patients 70 years and older (OR: 1.61, 95% CI: 1.09‐2.36; OR: 1.44, 95% CI: 1.08‐1.93; and OR: 1.39, 95% CI: 1.13‐1.71, respectively) and (2), when compared to patients with extreme resource intensity scores, patients with higher resource intensity scores were more likely to report top decile RSRs (moderate [OR: 1.42, 95% CI: 1.11‐1.83], major [OR: 1.56, 95% CI: 1.22‐2.01], and extreme [OR: 2.29, 95% CI: 1.8‐2.92]. These results were relatively consistent within medical and surgical subgroups (Table 3).

In those with at least 1 ICU attending encounter (see Supporting Table 1 in the online version of this article), no variables demonstrated significant association with top decile RSRs in the overall group or in the medical subgroup. For surgical patients with at least 1 ICU attending encounter (see Supporting Table 1 in the online version of this article), patients aged 30 to 49 and 50 to 69 years were more likely to provide top decile RSRs (OR: 1.93, 95% CI: 1.08‐3.46 and OR: 1.65, 95% CI 1.07‐2.53, respectively). Resource intensity was not significantly associated with top decile RSRs.

DISCUSSION

Our analysis suggests that, for patients on the general care floors, resource utilization is associated with the RSR and, therefore, potentially the CMS Summary Star Rating. Adjusting for severity of illness, patients with higher resource utilization were more likely to report top decile RSRs.

Prior data regarding utilization and satisfaction are mixed. In a 2‐year, prospective, national examination, patients in the highest quartile of patient satisfaction had increased healthcare and prescription drug expenditures as well as increased rates of hospitalization when compared with patients in the lowest quartile of patient satisfaction.[9] However, a recent national study of surgical administrative databases suggested hospitals with high patient satisfaction provided more efficient care.[13]

One reason for the conflicting data may be that large, national evaluations are unable to control for between‐hospital confounders (ie, hospital quality of care). By capturing all eligible returned surveys at 1 institution, our design allowed us to collect granular data. We found that in 1 hospital setting, patient population, facilities, and food services, patients receiving more clinical resources generally assigned higher ratings than patients receiving less.

It is possible that utilization is a proxy for serious illness, and that patients with serious illness receive more attention during hospitalization and are more satisfied when discharged in a good state of health. However, we did adjust for severity of illness in our model using the Charlson‐Deyo index and we suggest that, other factors being equal, hospitals with higher per‐patient expenditures may be assigned higher Summary Star Ratings.

CMS has recently implemented a number of metrics designed to decrease healthcare costs by improving quality, safety, and efficiency. Concurrently, CMS has also prioritized patient experience. The Summary Star Rating was created to provide healthcare consumers with an easy way to compare the patient experience between hospitals[4]; however, our data suggest that this metric may be at odds with inpatient cost savings and efficiency metrics.

Per‐patient spending becomes particularly salient when considering that in fiscal year 2016, CMS' hospital VBP reimbursement will include 2 metrics: an efficiency outcome measure labeled Medicare spending per beneficiary, and a patient experience outcome measure based on HCAHPS survey dimensions.[2] Together, these 2 metrics will comprise nearly half of the total VBP performance score used to determine reimbursement. Although our data suggest that these 2 VBP metrics may be correlated, it should be noted that we measured inpatient hospital charges, whereas the CMS efficiency outcome measure includes costs for episode of care spanning 3 days prior to hospitalization to 30 days after hospitalization.

Patient expectations likely play a role in satisfaction.[14, 15, 16] In an outpatient setting, physician fulfillment of patient requests has been associated with positive patient evaluations of care.[17] However, patients appear to value education, shared decision making, and provider empathy more than testing and intervention.[14, 18, 19, 20, 21, 22, 23] Perhaps, in the absence of the former attributes, patients use additional resource expenditure as a proxy.

It is not clear that higher resource expenditure improves outcomes. A landmark study of nearly 1 million Medicare enrollees by Fisher et al. suggests that, although Medicare patients in higher‐spending regions receive more care than those in lower‐spending regions, this does not result in better health outcomes, specifically with regard to mortality.[24, 25] Patients who live in areas of high hospital capacity use the hospital more frequently than do patients in areas of low hospital capacity, but this does not appear to result in improved mortality rates.[26] In fact, physicians in areas of high healthcare capacity report more difficulty maintaining high‐quality patient relationships and feel less able to provide high‐quality care than physicians in lower‐capacity areas.[27]

We hypothesize the cause of the association between resource utilization and patient satisfaction could be that patients (1) perceive that a doctor who allows them to stay longer in the hospital or who performs additional testing cares more about their well‐being and (2) that these patients feel more strongly that their concerns are being heard and addressed by their physicians. A systematic review of primary care patients identified many studies that found a positive association between meeting patient expectations and satisfaction with care, but also suggested that although patients frequently expect information, physicians misperceive this as an expectation of specific action.[28] A separate systematic review found that patient education in the form of decision aides can help patients develop more reasonable expectations and reduce utilization of certain discretionary procedures such as elective surgeries and prostate‐specific antigen testing.[29]

We did not specifically address clinical outcomes in our analysis because the clinical outcomes on which CMS currently adjusts VBP reimbursement focus on 30‐day mortality for specific diagnoses, nosocomial infections, and iatrogenic events.[30] Our data include only returned surveys from living patients, and it is likely that 30‐day mortality was similar throughout all subsets of patients. Additionally, the nosocomial and iatrogenic outcome measures used by CMS are sufficiently rare on the general floors and are unlikely to have significantly influenced our results.[31]

Our study has several strengths. Nearly all medical and surgical patient surveys returned during the study period were included, and therefore our calculations are likely to accurately reflect the Summary Star Rating that would have been assigned for the period. Second, the large sample size helps attenuate potential differences in commonly used outcome metrics. Third, by adjusting for a variety of demographic and clinical variables, we were able to decrease the likelihood of unidentified confounders.

Notably, we identified 38 (0.4%) surveys returned for patients under 18 years of age at admission. These surveys were included in our analysis because, to the best of our knowledge, they would have existed in the pool of surveys CMS could have used to assign a Summary Star Rating.

Our study also has limitations. First, geographically diverse data are needed to ensure generalizability. Second, we used the Charlson‐Deyo Comorbidity Index to describe the degree of illness for each patient. This index represents a patient's total illness burden but may not describe the relative severity of the patient's current illness relative to another patient. Third, we selected variables we felt were most likely to be associated with patient experience, but unidentified confounding remains possible. Fourth, attendings caring for ICU patients fall within the Division of Critical Care/Pulmonary Medicine. Therefore, we may have inadvertently placed patients into the ICU cohort who received a pulmonary/critical care consult on the general floors. Fifth, our data describe associations only for patients who returned surveys. Although there may be inherent biases in patients who return surveys, HCAHPS survey responses are used by CMS to determine a hospital's overall satisfaction score.

CONCLUSION

For patients who return HCAHPS surveys, resource utilization may be positively associated with a hospital's Summary Star Rating. These data suggest that hospitals with higher per‐patient expenditures may receive higher Summary Star Ratings, which could result in hospitals with higher per‐patient resource utilization appearing more attractive to healthcare consumers. Future studies should attempt to confirm our findings at other institutions and to determine causative factors.

The patient experience has become increasingly important to healthcare in the United States. It is now a metric used commonly to determine physician compensation and accounts for nearly 30% of the Centers for Medicare and Medicaid Services' (CMS) Value‐Based Purchasing (VBP) reimbursement for fiscal years 2015 and 2016.[1, 2]

In April 2015, CMS added a 5‐star patient experience score to its Hospital Compare website in an attempt to address the Affordable Care Act's call for transparent and easily understandable public reporting.[3] A hospital's principal score is the Summary Star Rating, which is based on responses to the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey. The formulas used to calculate Summary Star Ratings have been reported by CMS.[4]

Studies published over the past decade suggest that gender, age, education level, length of hospital stay, travel distance, and other factors may influence patient satisfaction.[5, 6, 7, 8] One study utilizing a national dataset suggested that higher patient satisfaction was associated with greater inpatient healthcare utilization and higher healthcare expenditures.[9] It is therefore possible that emphasizing patient experience scores could adversely impact healthcare resource utilization. However, positive patient experience may also be an important independent dimension of quality for patients and correlate with improved clinical outcomes.[10]

We know of no literature describing patient factors associated with the Summary Star Rating. Given that this rating is now used as a standard metric by which patient experience can be compared across more than 3,500 hospitals,[11] data describing the association between patient‐level factors and the Summary Star Rating may provide hospitals with an opportunity to target improvement efforts. We aimed to determine the degree to which resource utilization is associated with a satisfaction score based on the Summary Star Rating methodology.

METHODS

The study was conducted at the University of Rochester Medical Center (URMC), an 830‐bed tertiary care center in upstate New York. This was a retrospective review of all HCAHPS surveys returned to URMC over a 27‐month period from January 1, 2012 to April 1, 2014. URMC follows the standard CMS process for determining which patients receive surveys as follows. During the study timeframe, HCAHPS surveys were mailed to patients 18 years of age and older who had an inpatient stay spanning at least 1 midnight. Surveys were mailed within 5 days of discharge, and were generally returned within 6 weeks. URMC did not utilize telephone or email surveys during the study period. Surveys were not sent to patients who (1) were transferred to another facility, (2) were discharged to hospice, (3) died during the hospitalization, (4) received psychiatric or rehabilitative services during the hospitalization, (5) had an international address, and/or (6) were prisoners.

The survey vendor (Press Ganey, South Bend, IN) for URMC provided raw data for returned surveys with patient answers to questions. Administrative and billing databases were used to add demographic and clinical data for the corresponding hospitalization to the dataset. These data included age, gender, payer status (public, private, self, charity), length of stay, number of attendings who saw the patient (based on encounters documented in the electronic medical record (EMR)), all discharge International Classification of Diseases, 9th Revision (ICD‐9) diagnoses for the hospitalization, total charges for the hospitalization, and intensive care unit (ICU) utilization as evidenced by a documented encounter with a member of the Division of Critical Care/Pulmonary Medicine.

CMS analyzes surveys within 1 of 3 clinical service categories (medical, surgical, or obstetrics/gynecology) based on the discharging service. To parallel this approach, each returned survey was placed into 1 of these categories based on the clinical service of the discharging physician. Patients placed in the obstetrics/gynecology category (n = 1317, 13%) will be analyzed in a future analysis given inherent differences in patient characteristics that require evaluation of other variables.

RESULTS

Our initial search identified 10,007 returned surveys (29% of eligible patients returned surveys during the study period). Of these, 5059 (51%) were categorized as medical, 3630 (36%) as surgical, and 1317 (13%) as obstetrics/gynecology. One survey did not have the service of the discharging physician recorded and was excluded. Cohort demographics and relationship to RSRs in the top decile for the 8689 medical and surgical patients can be found in Table 1. The most common discharge diagnosis‐related groups (DRGs) for medical patients were 247, percutaneous cardiovascular procedure with drug‐eluding stent without major complications or comorbidities (MCC) (3.8%); 871, septicemia or severe sepsis without mechanical ventilation >96 hours with MCC (2.7%); and 392, esophagitis, gastroenteritis, and miscellaneous digestive disorders with MCC (2.3%). The most common DRGs for surgical patients were 460, spinal fusion except cervical without MCC (3.5%); 328, stomach, esophageal and duodenal procedure without complication or comorbidities or MCC (3.3%); and 491, back and neck procedure excluding spinal fusion without complication or comorbidities or MCC (3.1%).

Unadjusted analysis of medical and surgical patients identified significant associations of several variables with a top decile RSR (Table 2). Patients with longer lengths of stay (OR: 2.07, 95% CI: 1.72‐2.48), more attendings (OR: 1.44, 95% CI: 1.19‐1.73), and higher hospital charges (OR: 2.76, 95% CI: 2.19‐3.47) were more likely to report an RSR in the top decile. Patients without an ICU encounter (OR: 0.65, 95% CI: 0.55‐0.77) and on a medical service (OR: 0.57, 95% CI: 0.5‐ 0.66) were less likely to report an RSR in the top decile. Several associations were identified in only the medical or surgical cohorts. In the medical cohort, patients with the highest illness severity index (OR: 1.68, 95% CI: 1.12‐ 2.52) and with 7 different attending physicians (OR: 1.66, 95% CI: 1.27‐2.18) were more likely to report RSRs in the top decile. In the surgical cohort, patients <30 years of age (OR: 2.05, 95% CI 1.38‐3.07) were more likely to report an RSR in the top decile than patients >69 years of age. Insurance payer category and gender were not significantly associated with top decile RSRs.

Multivariable modeling (Table 3) for all patients without an ICU encounter suggested that (1) patients aged <30 years, 30 to 49 years, and 50 to 69 years were more likely to report top decile RSRs when compared to patients 70 years and older (OR: 1.61, 95% CI: 1.09‐2.36; OR: 1.44, 95% CI: 1.08‐1.93; and OR: 1.39, 95% CI: 1.13‐1.71, respectively) and (2), when compared to patients with extreme resource intensity scores, patients with higher resource intensity scores were more likely to report top decile RSRs (moderate [OR: 1.42, 95% CI: 1.11‐1.83], major [OR: 1.56, 95% CI: 1.22‐2.01], and extreme [OR: 2.29, 95% CI: 1.8‐2.92]. These results were relatively consistent within medical and surgical subgroups (Table 3).

In those with at least 1 ICU attending encounter (see Supporting Table 1 in the online version of this article), no variables demonstrated significant association with top decile RSRs in the overall group or in the medical subgroup. For surgical patients with at least 1 ICU attending encounter (see Supporting Table 1 in the online version of this article), patients aged 30 to 49 and 50 to 69 years were more likely to provide top decile RSRs (OR: 1.93, 95% CI: 1.08‐3.46 and OR: 1.65, 95% CI 1.07‐2.53, respectively). Resource intensity was not significantly associated with top decile RSRs.

DISCUSSION

Our analysis suggests that, for patients on the general care floors, resource utilization is associated with the RSR and, therefore, potentially the CMS Summary Star Rating. Adjusting for severity of illness, patients with higher resource utilization were more likely to report top decile RSRs.

Prior data regarding utilization and satisfaction are mixed. In a 2‐year, prospective, national examination, patients in the highest quartile of patient satisfaction had increased healthcare and prescription drug expenditures as well as increased rates of hospitalization when compared with patients in the lowest quartile of patient satisfaction.[9] However, a recent national study of surgical administrative databases suggested hospitals with high patient satisfaction provided more efficient care.[13]

One reason for the conflicting data may be that large, national evaluations are unable to control for between‐hospital confounders (ie, hospital quality of care). By capturing all eligible returned surveys at 1 institution, our design allowed us to collect granular data. We found that in 1 hospital setting, patient population, facilities, and food services, patients receiving more clinical resources generally assigned higher ratings than patients receiving less.

It is possible that utilization is a proxy for serious illness, and that patients with serious illness receive more attention during hospitalization and are more satisfied when discharged in a good state of health. However, we did adjust for severity of illness in our model using the Charlson‐Deyo index and we suggest that, other factors being equal, hospitals with higher per‐patient expenditures may be assigned higher Summary Star Ratings.

CMS has recently implemented a number of metrics designed to decrease healthcare costs by improving quality, safety, and efficiency. Concurrently, CMS has also prioritized patient experience. The Summary Star Rating was created to provide healthcare consumers with an easy way to compare the patient experience between hospitals[4]; however, our data suggest that this metric may be at odds with inpatient cost savings and efficiency metrics.

Per‐patient spending becomes particularly salient when considering that in fiscal year 2016, CMS' hospital VBP reimbursement will include 2 metrics: an efficiency outcome measure labeled Medicare spending per beneficiary, and a patient experience outcome measure based on HCAHPS survey dimensions.[2] Together, these 2 metrics will comprise nearly half of the total VBP performance score used to determine reimbursement. Although our data suggest that these 2 VBP metrics may be correlated, it should be noted that we measured inpatient hospital charges, whereas the CMS efficiency outcome measure includes costs for episode of care spanning 3 days prior to hospitalization to 30 days after hospitalization.

Patient expectations likely play a role in satisfaction.[14, 15, 16] In an outpatient setting, physician fulfillment of patient requests has been associated with positive patient evaluations of care.[17] However, patients appear to value education, shared decision making, and provider empathy more than testing and intervention.[14, 18, 19, 20, 21, 22, 23] Perhaps, in the absence of the former attributes, patients use additional resource expenditure as a proxy.

It is not clear that higher resource expenditure improves outcomes. A landmark study of nearly 1 million Medicare enrollees by Fisher et al. suggests that, although Medicare patients in higher‐spending regions receive more care than those in lower‐spending regions, this does not result in better health outcomes, specifically with regard to mortality.[24, 25] Patients who live in areas of high hospital capacity use the hospital more frequently than do patients in areas of low hospital capacity, but this does not appear to result in improved mortality rates.[26] In fact, physicians in areas of high healthcare capacity report more difficulty maintaining high‐quality patient relationships and feel less able to provide high‐quality care than physicians in lower‐capacity areas.[27]

We hypothesize the cause of the association between resource utilization and patient satisfaction could be that patients (1) perceive that a doctor who allows them to stay longer in the hospital or who performs additional testing cares more about their well‐being and (2) that these patients feel more strongly that their concerns are being heard and addressed by their physicians. A systematic review of primary care patients identified many studies that found a positive association between meeting patient expectations and satisfaction with care, but also suggested that although patients frequently expect information, physicians misperceive this as an expectation of specific action.[28] A separate systematic review found that patient education in the form of decision aides can help patients develop more reasonable expectations and reduce utilization of certain discretionary procedures such as elective surgeries and prostate‐specific antigen testing.[29]

We did not specifically address clinical outcomes in our analysis because the clinical outcomes on which CMS currently adjusts VBP reimbursement focus on 30‐day mortality for specific diagnoses, nosocomial infections, and iatrogenic events.[30] Our data include only returned surveys from living patients, and it is likely that 30‐day mortality was similar throughout all subsets of patients. Additionally, the nosocomial and iatrogenic outcome measures used by CMS are sufficiently rare on the general floors and are unlikely to have significantly influenced our results.[31]

Our study has several strengths. Nearly all medical and surgical patient surveys returned during the study period were included, and therefore our calculations are likely to accurately reflect the Summary Star Rating that would have been assigned for the period. Second, the large sample size helps attenuate potential differences in commonly used outcome metrics. Third, by adjusting for a variety of demographic and clinical variables, we were able to decrease the likelihood of unidentified confounders.

Notably, we identified 38 (0.4%) surveys returned for patients under 18 years of age at admission. These surveys were included in our analysis because, to the best of our knowledge, they would have existed in the pool of surveys CMS could have used to assign a Summary Star Rating.

Our study also has limitations. First, geographically diverse data are needed to ensure generalizability. Second, we used the Charlson‐Deyo Comorbidity Index to describe the degree of illness for each patient. This index represents a patient's total illness burden but may not describe the relative severity of the patient's current illness relative to another patient. Third, we selected variables we felt were most likely to be associated with patient experience, but unidentified confounding remains possible. Fourth, attendings caring for ICU patients fall within the Division of Critical Care/Pulmonary Medicine. Therefore, we may have inadvertently placed patients into the ICU cohort who received a pulmonary/critical care consult on the general floors. Fifth, our data describe associations only for patients who returned surveys. Although there may be inherent biases in patients who return surveys, HCAHPS survey responses are used by CMS to determine a hospital's overall satisfaction score.

CONCLUSION

For patients who return HCAHPS surveys, resource utilization may be positively associated with a hospital's Summary Star Rating. These data suggest that hospitals with higher per‐patient expenditures may receive higher Summary Star Ratings, which could result in hospitals with higher per‐patient resource utilization appearing more attractive to healthcare consumers. Future studies should attempt to confirm our findings at other institutions and to determine causative factors.

Maintaining high‐quality patient care, optimizing patient safety, and providing adequate trainee supervision has been an area of debate in medical education recently, and many physicians remain concerned that excessive regulation and duty hour restrictions may prevent residents from obtaining sufficient experience and developing an appropriate sense of autonomy.[1, 2, 3, 4] However, pediatric hospital medicine (PHM) has seen dramatic increases in evening and nighttime in‐house attending coverage, and the trend is expected to continue.[5, 6] Whether it be for financial, educational, or patient‐centered reasons, increased in‐house attending coverage at an academic medical setting, almost by definition, increases direct resident supervision.[7]

Increased supervision may result in better educational outcomes,[8] but many forces, such as night float systems and electronic medical records (EMRs), pull residents away from the bedside, leaving them with fewer opportunities to make decisions and a reduced sense of personal responsibility and patient ownership. Experiential learning is of great value in medical training, and without this, residents may exit their training with less confidence and competence, only rarely having been able to make important medical decisions on their own.[9, 10]

Counter to the shift toward increased supervision, we recently amended our process for pediatric admissions to the PHM service by transitioning from mandatory to on‐demand attending input during the admissions process. We hypothesized that this would improve its efficiency by encouraging residents to develop an increased sense of patient ownership and would not significantly impact patient care.

METHODS

Process Change

Prior to the change, a pediatric emergency department (ED) provider at GCH directly contacted the PHM attending for all admissions to the PHM service (Figure 1). If the PHM attending accepted the admission, the ED provider then notified the pediatric admitting officer (PAO), a third‐year pediatric or fourth‐year medicinepediatric resident, who either performed or delegated the admission duties (eg, history and physical exam, admission orders).

Figure 1

On June 18, 2012, a new process for pediatric admissions was implemented (Figure 1). The ED provider now called the PAO, and not the attending, to discuss an admission to the PHM service. The PAO was empowered to accept the patient on behalf of the PHM attending, and perform or delegate the admission duties. During daytime hours (7:00 _am5:00 _pm), the PAO was expected to alert the PHM attending of the admission to allow the attending to see the patient on the day of admission. The PHM attending discussed the case with the admitting resident after the resident had an opportunity to assess the patient and formulate a management plan. During evening hours (5:00 _pm10:00 _pm), the admitting resident was expected to contact the PHM attending on call after evaluating the patient and developing a plan. Overnight (10:00 _pm7:00 _am), the PAO was given discretion as to whether she/he needed to contact the PHM attending on call; the PHM service attending then saw the patient in the morning. Residents were strongly encouraged to call the PHM attending with any questions or concerns or if they did not feel an admission was appropriate to the PHM service.

RESULTS

The initial search identified 532 admissions. Of these, 140 were excluded (72 were via route other than the ED, 44 were not admitted to PHM, 14 were outside the study period, and 10 did not meet age criteria). Therefore, 182 admissions in the 2011 cohort and 210 admissions in the 2012 cohort were included. For all patients in the 2012 cohort, the correct admission process was followed.

Demographic characteristics between cohorts were similar (Table 1). Data for ROM and SOI were available for 141 (78%) 2011 patients and for 169 (81%) 2012 patients. The distribution of patients over the study months differed between cohorts. Age, gender, ROM, and SOI were not significantly different.

The median difference in time from the ED provider admission request to the first inpatient resident order was roughly half as long in 2012 than in 2011 (123 vs 62 minutes, P<0.001) (Table 2). There were 12 cases in which the inpatient order came prior to the ED admission request in 2012 and 2 cases in 2011, and these were excluded from the order time difference analysis. LOS was not significantly different between groups (P=0.348). There were no differences in the frequency of antibiotic changes when all patients were considered or in the subgroup in whom antibiotics were prescribed by either the resident or attending. The number of cases for which the admitting resident's plan was deemed not to have met standard of care were few and not significantly different (P=1). None of these patients experienced harm as a result, and in all cases, SOC was determined to have been provided by the admitting PHM attending. The frequency of RRT calls within the first 24 hours of admission on PHM patients was not significantly different (P=0.114).

When only patients admitted during the night in 2011 and 2012 were compared, results were consistent with the overall finding that there was a shorter time to inpatient admission order without a difference in other studied variables (Table 3).

DISCUSSION

The purpose of this study was to evaluate an admission process that removed an ineffective method of attending oversight and allowed residents an opportunity to develop patient care plans prior to attending input. The key change from the original process was removing the step in which the ED provider contacted the PHM attending for new admissions, thus eliminating mandatory inpatient attending input, removing an impediment to workflow, and empowering inpatient pediatric residents to assess new patients and develop management plans. Our data show a reduction in the time difference between the ED admission request and the inpatient resident's first order by more than an hour, indicating a more efficient admission process. Although one might expect that eliminating the act of a phone call would shorten this time by a few minutes, it cannot account for the extent of the difference we found. We postulate that an increased sense of accountability motivated inpatient residents to evaluate and begin management sooner, a topic that requires further exploration.

A more efficient admission process benefits emergency medicine residents and other ED providers as well. It is well documented that ED crowding is associated with decreased quality of care,[11, 12] and ED efficiency is receiving increased attention with newly reportable quality metrics such as Admit Decision Time to Emergency Department Departure Time for Admitted Patients.[13]

Our data do not attenuate the importance of hospitalists in patient care, as evidenced by the fact that PHM attendings continued to frequently amend the residents' antibiotic choicethe only variable we evaluated in terms of change in planand recognized several cases in which the residents' plan did not meet standard of care. Furthermore, attendings continued to be available by phone and pager for guidance and education when needed or requested by the residents. Instead, our data show that removing mandated attending input at the time of admission did not significantly impact major patient outcomes, which may partly be attributable to the general safety of the inpatient pediatric wards.[14, 15] In our study, a comprehensive analysis of patient harm was not possible given the variable list and infrequency with which SOC was not met or RRTs were called. Furthermore, our residency program continues to comply with national pediatric residency requirements for nighttime supervision.[7]

Our PHM division, which had previously allocated 2 hours of attending clinical time per call night, now averages <15 minutes. These data conflict with the current trend in PHM toward more, rather than less, direct attending oversight. Many PHM divisions have moved toward 24/7 in‐house coverage,[5] a situation that often results in shiftwork and multiple handoffs. Removing the in‐house attending overnight would allow for the rapidly growing PHM subspecialty to allocate hospitalists elsewhere depending on their scholarly needs, particularly as divisions seek to become increasingly involved in medical education, research, and hospital leadership.[16, 17] Although one might posit a financial benefit to having in‐house attendings determine the appropriateness of an admission overnight, we identified no case in which the insurance denied an admission.

Safety equivalence of an in‐house to on‐call attending is poorly studied in PHM. However, even in intensive care units, where the majority of morbidity and mortality occur, it is unclear that the presence of an attending, let alone mandating phone calls, positively impacts survival. One prospective trial failed to demonstrate a difference in patient outcomes in the critical care setting when comparing mandated attending in‐house involvement to optional attending availability by phone.[18] Furthermore, several studies have found no association with time of admission and mortality, implying there is no criticality specifically requiring nighttime coverage.[19, 20]

One adult study of nocturnists showed that residents felt they had more contact with attendings who were in‐house than attendings taking home calls.[21] However, when the residents were asked why they did not contact the attending, the only difference between at‐home and in‐house attendings was that for attendings available by phone, residents were less likely to know who to call and were hesitant to wake the attending.

This study had several limitations. First, we could not effectively blind reviewers; a salient point given that the reviewers benefited from the new system with a reduced nighttime workload. We attempted to minimize this bias by employing multiple independent evaluations followed by group consensus whenever possible. Second, even though we had 3 hospitalists independently review each 2012 RRT to determine whether it was preventable by the prior system, this task was prone to retrospective bias. Third, there was a significant difference in the month of admission between cohorts. Rather than biasing toward our observed time difference, the fact that more patients were admitted in July 2012the beginning of the academic yearmay have decreased our observed difference given that residents were less experienced. Forth, this study used certain measurable outcomes as proxies for quality of care and patient harm and was likely underpowered to truly detect a difference in some of the more infrequent variables. Furthermore, we did not evaluate other potential harms, such as cost. Fifth, we did not evaluate whether or not the new process changed ED provider behavior (ie, an ED provider may wait longer to request admission overnight given that the PHM attending is not mandated to provide input until the morning). Finally, although LOS was used as a balancing measure, it would likely have taken major events or omissions during the admission process to cause it to change significantly, and therefore the lack of statistical difference in this metric does not necessarily imply that more subtle aspects of care were the same between groups. We also chose not to include readmission rate for this reason, as any change could not conclusively be attributed to the new admission process.

CONCLUSION

Increasing resident autonomy by removing mandated input during PHM admissions makes the process more efficient and results in no significant changes to major patient outcomes. These data may be used by rapidly growing PHM divisions to redefine faculty clinical responsibilities, particularly at night.

Maintaining high‐quality patient care, optimizing patient safety, and providing adequate trainee supervision has been an area of debate in medical education recently, and many physicians remain concerned that excessive regulation and duty hour restrictions may prevent residents from obtaining sufficient experience and developing an appropriate sense of autonomy.[1, 2, 3, 4] However, pediatric hospital medicine (PHM) has seen dramatic increases in evening and nighttime in‐house attending coverage, and the trend is expected to continue.[5, 6] Whether it be for financial, educational, or patient‐centered reasons, increased in‐house attending coverage at an academic medical setting, almost by definition, increases direct resident supervision.[7]

Increased supervision may result in better educational outcomes,[8] but many forces, such as night float systems and electronic medical records (EMRs), pull residents away from the bedside, leaving them with fewer opportunities to make decisions and a reduced sense of personal responsibility and patient ownership. Experiential learning is of great value in medical training, and without this, residents may exit their training with less confidence and competence, only rarely having been able to make important medical decisions on their own.[9, 10]

Counter to the shift toward increased supervision, we recently amended our process for pediatric admissions to the PHM service by transitioning from mandatory to on‐demand attending input during the admissions process. We hypothesized that this would improve its efficiency by encouraging residents to develop an increased sense of patient ownership and would not significantly impact patient care.

METHODS

Process Change

Prior to the change, a pediatric emergency department (ED) provider at GCH directly contacted the PHM attending for all admissions to the PHM service (Figure 1). If the PHM attending accepted the admission, the ED provider then notified the pediatric admitting officer (PAO), a third‐year pediatric or fourth‐year medicinepediatric resident, who either performed or delegated the admission duties (eg, history and physical exam, admission orders).

Figure 1

On June 18, 2012, a new process for pediatric admissions was implemented (Figure 1). The ED provider now called the PAO, and not the attending, to discuss an admission to the PHM service. The PAO was empowered to accept the patient on behalf of the PHM attending, and perform or delegate the admission duties. During daytime hours (7:00 _am5:00 _pm), the PAO was expected to alert the PHM attending of the admission to allow the attending to see the patient on the day of admission. The PHM attending discussed the case with the admitting resident after the resident had an opportunity to assess the patient and formulate a management plan. During evening hours (5:00 _pm10:00 _pm), the admitting resident was expected to contact the PHM attending on call after evaluating the patient and developing a plan. Overnight (10:00 _pm7:00 _am), the PAO was given discretion as to whether she/he needed to contact the PHM attending on call; the PHM service attending then saw the patient in the morning. Residents were strongly encouraged to call the PHM attending with any questions or concerns or if they did not feel an admission was appropriate to the PHM service.

RESULTS

The initial search identified 532 admissions. Of these, 140 were excluded (72 were via route other than the ED, 44 were not admitted to PHM, 14 were outside the study period, and 10 did not meet age criteria). Therefore, 182 admissions in the 2011 cohort and 210 admissions in the 2012 cohort were included. For all patients in the 2012 cohort, the correct admission process was followed.

Demographic characteristics between cohorts were similar (Table 1). Data for ROM and SOI were available for 141 (78%) 2011 patients and for 169 (81%) 2012 patients. The distribution of patients over the study months differed between cohorts. Age, gender, ROM, and SOI were not significantly different.

The median difference in time from the ED provider admission request to the first inpatient resident order was roughly half as long in 2012 than in 2011 (123 vs 62 minutes, P<0.001) (Table 2). There were 12 cases in which the inpatient order came prior to the ED admission request in 2012 and 2 cases in 2011, and these were excluded from the order time difference analysis. LOS was not significantly different between groups (P=0.348). There were no differences in the frequency of antibiotic changes when all patients were considered or in the subgroup in whom antibiotics were prescribed by either the resident or attending. The number of cases for which the admitting resident's plan was deemed not to have met standard of care were few and not significantly different (P=1). None of these patients experienced harm as a result, and in all cases, SOC was determined to have been provided by the admitting PHM attending. The frequency of RRT calls within the first 24 hours of admission on PHM patients was not significantly different (P=0.114).

When only patients admitted during the night in 2011 and 2012 were compared, results were consistent with the overall finding that there was a shorter time to inpatient admission order without a difference in other studied variables (Table 3).

DISCUSSION

The purpose of this study was to evaluate an admission process that removed an ineffective method of attending oversight and allowed residents an opportunity to develop patient care plans prior to attending input. The key change from the original process was removing the step in which the ED provider contacted the PHM attending for new admissions, thus eliminating mandatory inpatient attending input, removing an impediment to workflow, and empowering inpatient pediatric residents to assess new patients and develop management plans. Our data show a reduction in the time difference between the ED admission request and the inpatient resident's first order by more than an hour, indicating a more efficient admission process. Although one might expect that eliminating the act of a phone call would shorten this time by a few minutes, it cannot account for the extent of the difference we found. We postulate that an increased sense of accountability motivated inpatient residents to evaluate and begin management sooner, a topic that requires further exploration.

A more efficient admission process benefits emergency medicine residents and other ED providers as well. It is well documented that ED crowding is associated with decreased quality of care,[11, 12] and ED efficiency is receiving increased attention with newly reportable quality metrics such as Admit Decision Time to Emergency Department Departure Time for Admitted Patients.[13]

Our data do not attenuate the importance of hospitalists in patient care, as evidenced by the fact that PHM attendings continued to frequently amend the residents' antibiotic choicethe only variable we evaluated in terms of change in planand recognized several cases in which the residents' plan did not meet standard of care. Furthermore, attendings continued to be available by phone and pager for guidance and education when needed or requested by the residents. Instead, our data show that removing mandated attending input at the time of admission did not significantly impact major patient outcomes, which may partly be attributable to the general safety of the inpatient pediatric wards.[14, 15] In our study, a comprehensive analysis of patient harm was not possible given the variable list and infrequency with which SOC was not met or RRTs were called. Furthermore, our residency program continues to comply with national pediatric residency requirements for nighttime supervision.[7]

Our PHM division, which had previously allocated 2 hours of attending clinical time per call night, now averages <15 minutes. These data conflict with the current trend in PHM toward more, rather than less, direct attending oversight. Many PHM divisions have moved toward 24/7 in‐house coverage,[5] a situation that often results in shiftwork and multiple handoffs. Removing the in‐house attending overnight would allow for the rapidly growing PHM subspecialty to allocate hospitalists elsewhere depending on their scholarly needs, particularly as divisions seek to become increasingly involved in medical education, research, and hospital leadership.[16, 17] Although one might posit a financial benefit to having in‐house attendings determine the appropriateness of an admission overnight, we identified no case in which the insurance denied an admission.

Safety equivalence of an in‐house to on‐call attending is poorly studied in PHM. However, even in intensive care units, where the majority of morbidity and mortality occur, it is unclear that the presence of an attending, let alone mandating phone calls, positively impacts survival. One prospective trial failed to demonstrate a difference in patient outcomes in the critical care setting when comparing mandated attending in‐house involvement to optional attending availability by phone.[18] Furthermore, several studies have found no association with time of admission and mortality, implying there is no criticality specifically requiring nighttime coverage.[19, 20]

One adult study of nocturnists showed that residents felt they had more contact with attendings who were in‐house than attendings taking home calls.[21] However, when the residents were asked why they did not contact the attending, the only difference between at‐home and in‐house attendings was that for attendings available by phone, residents were less likely to know who to call and were hesitant to wake the attending.

This study had several limitations. First, we could not effectively blind reviewers; a salient point given that the reviewers benefited from the new system with a reduced nighttime workload. We attempted to minimize this bias by employing multiple independent evaluations followed by group consensus whenever possible. Second, even though we had 3 hospitalists independently review each 2012 RRT to determine whether it was preventable by the prior system, this task was prone to retrospective bias. Third, there was a significant difference in the month of admission between cohorts. Rather than biasing toward our observed time difference, the fact that more patients were admitted in July 2012the beginning of the academic yearmay have decreased our observed difference given that residents were less experienced. Forth, this study used certain measurable outcomes as proxies for quality of care and patient harm and was likely underpowered to truly detect a difference in some of the more infrequent variables. Furthermore, we did not evaluate other potential harms, such as cost. Fifth, we did not evaluate whether or not the new process changed ED provider behavior (ie, an ED provider may wait longer to request admission overnight given that the PHM attending is not mandated to provide input until the morning). Finally, although LOS was used as a balancing measure, it would likely have taken major events or omissions during the admission process to cause it to change significantly, and therefore the lack of statistical difference in this metric does not necessarily imply that more subtle aspects of care were the same between groups. We also chose not to include readmission rate for this reason, as any change could not conclusively be attributed to the new admission process.

CONCLUSION

Increasing resident autonomy by removing mandated input during PHM admissions makes the process more efficient and results in no significant changes to major patient outcomes. These data may be used by rapidly growing PHM divisions to redefine faculty clinical responsibilities, particularly at night.

Maintaining high‐quality patient care, optimizing patient safety, and providing adequate trainee supervision has been an area of debate in medical education recently, and many physicians remain concerned that excessive regulation and duty hour restrictions may prevent residents from obtaining sufficient experience and developing an appropriate sense of autonomy.[1, 2, 3, 4] However, pediatric hospital medicine (PHM) has seen dramatic increases in evening and nighttime in‐house attending coverage, and the trend is expected to continue.[5, 6] Whether it be for financial, educational, or patient‐centered reasons, increased in‐house attending coverage at an academic medical setting, almost by definition, increases direct resident supervision.[7]

Increased supervision may result in better educational outcomes,[8] but many forces, such as night float systems and electronic medical records (EMRs), pull residents away from the bedside, leaving them with fewer opportunities to make decisions and a reduced sense of personal responsibility and patient ownership. Experiential learning is of great value in medical training, and without this, residents may exit their training with less confidence and competence, only rarely having been able to make important medical decisions on their own.[9, 10]

Counter to the shift toward increased supervision, we recently amended our process for pediatric admissions to the PHM service by transitioning from mandatory to on‐demand attending input during the admissions process. We hypothesized that this would improve its efficiency by encouraging residents to develop an increased sense of patient ownership and would not significantly impact patient care.

METHODS

Process Change

Prior to the change, a pediatric emergency department (ED) provider at GCH directly contacted the PHM attending for all admissions to the PHM service (Figure 1). If the PHM attending accepted the admission, the ED provider then notified the pediatric admitting officer (PAO), a third‐year pediatric or fourth‐year medicinepediatric resident, who either performed or delegated the admission duties (eg, history and physical exam, admission orders).

Figure 1

On June 18, 2012, a new process for pediatric admissions was implemented (Figure 1). The ED provider now called the PAO, and not the attending, to discuss an admission to the PHM service. The PAO was empowered to accept the patient on behalf of the PHM attending, and perform or delegate the admission duties. During daytime hours (7:00 _am5:00 _pm), the PAO was expected to alert the PHM attending of the admission to allow the attending to see the patient on the day of admission. The PHM attending discussed the case with the admitting resident after the resident had an opportunity to assess the patient and formulate a management plan. During evening hours (5:00 _pm10:00 _pm), the admitting resident was expected to contact the PHM attending on call after evaluating the patient and developing a plan. Overnight (10:00 _pm7:00 _am), the PAO was given discretion as to whether she/he needed to contact the PHM attending on call; the PHM service attending then saw the patient in the morning. Residents were strongly encouraged to call the PHM attending with any questions or concerns or if they did not feel an admission was appropriate to the PHM service.

RESULTS

The initial search identified 532 admissions. Of these, 140 were excluded (72 were via route other than the ED, 44 were not admitted to PHM, 14 were outside the study period, and 10 did not meet age criteria). Therefore, 182 admissions in the 2011 cohort and 210 admissions in the 2012 cohort were included. For all patients in the 2012 cohort, the correct admission process was followed.

Demographic characteristics between cohorts were similar (Table 1). Data for ROM and SOI were available for 141 (78%) 2011 patients and for 169 (81%) 2012 patients. The distribution of patients over the study months differed between cohorts. Age, gender, ROM, and SOI were not significantly different.

The median difference in time from the ED provider admission request to the first inpatient resident order was roughly half as long in 2012 than in 2011 (123 vs 62 minutes, P<0.001) (Table 2). There were 12 cases in which the inpatient order came prior to the ED admission request in 2012 and 2 cases in 2011, and these were excluded from the order time difference analysis. LOS was not significantly different between groups (P=0.348). There were no differences in the frequency of antibiotic changes when all patients were considered or in the subgroup in whom antibiotics were prescribed by either the resident or attending. The number of cases for which the admitting resident's plan was deemed not to have met standard of care were few and not significantly different (P=1). None of these patients experienced harm as a result, and in all cases, SOC was determined to have been provided by the admitting PHM attending. The frequency of RRT calls within the first 24 hours of admission on PHM patients was not significantly different (P=0.114).

When only patients admitted during the night in 2011 and 2012 were compared, results were consistent with the overall finding that there was a shorter time to inpatient admission order without a difference in other studied variables (Table 3).

DISCUSSION

The purpose of this study was to evaluate an admission process that removed an ineffective method of attending oversight and allowed residents an opportunity to develop patient care plans prior to attending input. The key change from the original process was removing the step in which the ED provider contacted the PHM attending for new admissions, thus eliminating mandatory inpatient attending input, removing an impediment to workflow, and empowering inpatient pediatric residents to assess new patients and develop management plans. Our data show a reduction in the time difference between the ED admission request and the inpatient resident's first order by more than an hour, indicating a more efficient admission process. Although one might expect that eliminating the act of a phone call would shorten this time by a few minutes, it cannot account for the extent of the difference we found. We postulate that an increased sense of accountability motivated inpatient residents to evaluate and begin management sooner, a topic that requires further exploration.

A more efficient admission process benefits emergency medicine residents and other ED providers as well. It is well documented that ED crowding is associated with decreased quality of care,[11, 12] and ED efficiency is receiving increased attention with newly reportable quality metrics such as Admit Decision Time to Emergency Department Departure Time for Admitted Patients.[13]

Our data do not attenuate the importance of hospitalists in patient care, as evidenced by the fact that PHM attendings continued to frequently amend the residents' antibiotic choicethe only variable we evaluated in terms of change in planand recognized several cases in which the residents' plan did not meet standard of care. Furthermore, attendings continued to be available by phone and pager for guidance and education when needed or requested by the residents. Instead, our data show that removing mandated attending input at the time of admission did not significantly impact major patient outcomes, which may partly be attributable to the general safety of the inpatient pediatric wards.[14, 15] In our study, a comprehensive analysis of patient harm was not possible given the variable list and infrequency with which SOC was not met or RRTs were called. Furthermore, our residency program continues to comply with national pediatric residency requirements for nighttime supervision.[7]

Our PHM division, which had previously allocated 2 hours of attending clinical time per call night, now averages <15 minutes. These data conflict with the current trend in PHM toward more, rather than less, direct attending oversight. Many PHM divisions have moved toward 24/7 in‐house coverage,[5] a situation that often results in shiftwork and multiple handoffs. Removing the in‐house attending overnight would allow for the rapidly growing PHM subspecialty to allocate hospitalists elsewhere depending on their scholarly needs, particularly as divisions seek to become increasingly involved in medical education, research, and hospital leadership.[16, 17] Although one might posit a financial benefit to having in‐house attendings determine the appropriateness of an admission overnight, we identified no case in which the insurance denied an admission.

Safety equivalence of an in‐house to on‐call attending is poorly studied in PHM. However, even in intensive care units, where the majority of morbidity and mortality occur, it is unclear that the presence of an attending, let alone mandating phone calls, positively impacts survival. One prospective trial failed to demonstrate a difference in patient outcomes in the critical care setting when comparing mandated attending in‐house involvement to optional attending availability by phone.[18] Furthermore, several studies have found no association with time of admission and mortality, implying there is no criticality specifically requiring nighttime coverage.[19, 20]

One adult study of nocturnists showed that residents felt they had more contact with attendings who were in‐house than attendings taking home calls.[21] However, when the residents were asked why they did not contact the attending, the only difference between at‐home and in‐house attendings was that for attendings available by phone, residents were less likely to know who to call and were hesitant to wake the attending.

This study had several limitations. First, we could not effectively blind reviewers; a salient point given that the reviewers benefited from the new system with a reduced nighttime workload. We attempted to minimize this bias by employing multiple independent evaluations followed by group consensus whenever possible. Second, even though we had 3 hospitalists independently review each 2012 RRT to determine whether it was preventable by the prior system, this task was prone to retrospective bias. Third, there was a significant difference in the month of admission between cohorts. Rather than biasing toward our observed time difference, the fact that more patients were admitted in July 2012the beginning of the academic yearmay have decreased our observed difference given that residents were less experienced. Forth, this study used certain measurable outcomes as proxies for quality of care and patient harm and was likely underpowered to truly detect a difference in some of the more infrequent variables. Furthermore, we did not evaluate other potential harms, such as cost. Fifth, we did not evaluate whether or not the new process changed ED provider behavior (ie, an ED provider may wait longer to request admission overnight given that the PHM attending is not mandated to provide input until the morning). Finally, although LOS was used as a balancing measure, it would likely have taken major events or omissions during the admission process to cause it to change significantly, and therefore the lack of statistical difference in this metric does not necessarily imply that more subtle aspects of care were the same between groups. We also chose not to include readmission rate for this reason, as any change could not conclusively be attributed to the new admission process.

CONCLUSION

Increasing resident autonomy by removing mandated input during PHM admissions makes the process more efficient and results in no significant changes to major patient outcomes. These data may be used by rapidly growing PHM divisions to redefine faculty clinical responsibilities, particularly at night.