Affiliations
Department of Medicine, Johns Hopkins University School of Medicine
Given name(s)
Jenna
Family name
Swann
Degrees
MS

Outcomes after 2011 Residency Reform

Article Type
Article
Changed
Sun, 05/21/2017 - 14:15
Display Headline
Inpatient safety outcomes following the 2011 residency work‐hour reform
Author(s)
Lauren Block, MD, MPH
Marian Jarlenski, MPH
Albert W. Wu, MD, MPH
Leonard Feldman, MD
Joseph Conigliaro, MD, MPH
Jenna Swann, MS
Sanjay V. Desai, MD

The Accreditation Council for Graduate Medical Education (ACGME) Common Program Requirements implemented in July 2011 increased supervision requirements and limited continuous work hours for first‐year residents.[1] Similar to the 2003 mandates, these requirements were introduced to improve patient safety and education at academic medical centers.[2] Work‐hour reforms have been associated with decreased resident burnout and improved sleep.[3, 4, 5] However, national observational studies and systematic reviews of the impact of the 2003 reforms on patient safety and quality of care have been varied in terms of outcome.[6, 7, 8, 9, 10] Small studies of the 2011 recommendations have shown increased sleep duration and decreased burnout, but also an increased number of handoffs and increased resident concerns about making a serious medical error.[11, 12, 13, 14] Although national surveys of residents and program directors have not indicated improvements in education or quality of life, 1 observational study did show improvement in clinical exposure and conference attendance.[15, 16, 17, 18] The impact of the 2011 reforms on patient safety remains unclear.[19, 20]

The objective of this study was to evaluate the association between implementation of the 2011 residency work‐hour mandates and patient safety outcomes at a large academic medical center.

METHODS

Study Design

This observational study used a quasi‐experimental difference‐in‐differences approach to evaluate whether residency work‐hour changes were associated with patient safety outcomes among general medicine inpatients. We compared safety outcomes among adult patients discharged from resident general medical services (referred to as resident) to safety outcomes among patients discharged by the hospitalist general medical service (referred to as hospitalist) before and after the 2011 residency work‐hour reforms at a large academic medical center. Differences in outcomes for the resident group were compared to differences observed in the hospitalist group, with adjustment for relevant demographic and case mix factors.[21] We used the hospitalist service as a control group, because ACGME changes applied only to resident services. The strength of this design is that it controls for secular trends that are correlated with patient safety, impacting both residents and hospitalists similarly.[9]

Approval for this study and a Health Insurance Portability and Accountability Act waiver were granted by the Johns Hopkins University School of Medicine institutional review board. We retrospectively examined administrative data on all patient discharges from the general medicine services at Johns Hopkins Hospital between July 1, 2008 and June 30, 2012 that were identified as pertaining to resident or hospitalist services.

Patient Allocation and Physician Scheduling

Patient admission to the resident or hospitalist service was decided by a number of factors. To maintain continuity of care, patients were preferentially admitted to the same service as for prior admissions. New patients were admitted to a service based on bed availability, nurse staffing, patient gender, isolation precautions, and cardiac monitor availability.

The inpatient resident services were staffed prior to July 2011 using a traditional 30‐hour overnight call system. Following July 2011, the inpatient resident services were staffed using a modified overnight call system, in which interns took overnight calls from 8 pm until 12 pm the following day, once every 5 nights with supervision by upper‐level residents. These interns rotated through daytime admitting and coverage roles on the intervening days. The hospitalist service was organized into a 3‐physician rotation of day shift, evening shift, and overnight shift.

Data and Outcomes

Twenty‐nine percent of patients in the sample were admitted more than once during the study period, and patients were generally admitted to the same resident team during each admission. Patients with multiple admissions were counted multiple times in the model. We categorized admissions as prereform (July 1, 2008June 30, 2011) and postreform (July 1, 2011June 30, 2012). Outcomes evaluated included hospital length of stay, 30‐day readmission, intensive care unit stay (ICU) stay, inpatient mortality, and number of Maryland Hospital Acquired Conditions (MHACs). ICU stay pertained to any ICU admission including initial admission and transfer from the inpatient floor. MHACs are a set of inpatient performance indicators derived from a list of 64 inpatient Potentially Preventable Complications developed by 3M Health Information Systems.[22] MHACs are used by the Maryland Health Services Cost Review Commission to link hospital payment to performance for costly, preventable, and clinically relevant complications. MHACs were coded in our analysis as a dichotomous variable. Independent variables included patient age at admission, race, gender, and case mix index. Case mix index (CMI) is a numeric score that measures resource utilization for a specific patient population. CMI is a weighted value assigned to patients based on resource utilization and All Patient Refined Diagnostic Related Group and was included as an indicator of patient illness severity and risk of mortality.[23] Data were obtained from administrative records from the case mix research team at Johns Hopkins Medicine.

To account for transitional differences that may have coincided with the opening of a new hospital wing in late April 2012, we conducted a sensitivity analysis, in which we excluded from analysis any visits that took place in May 2012 to June 2012.

Data Analysis

Based on historical studies, we calculated that a sample size of at least 3600 discharges would allow us to detect a difference of 5% between the pre‐ and postreform period assuming baseline 20% occurrence of dichotomous outcomes (=0.05; =0.2; r=4).[21]

The primary unit of analysis was the hospital discharge. Similar to Horwitz et al., we analyzed data using a difference‐in‐differences estimation strategy.[21] We used multivariable linear regression for length of stay measured as a continuous variable, and multivariable logistic regression for inpatient mortality, 30‐day readmission, MHACs coded as a dichotomous variable, and ICU stay coded as a dichotomous variable.[9] The difference‐in‐differences estimation was used to determine whether the postreform period relative to prereform period was associated with differences in outcomes comparing resident and hospitalist services. In the regression models, the independent variables of interest included an indicator variable for whether a patient was treated on a resident service, an indicator variable for whether a patient was discharged in the postreform period, and the interaction of these 2 variables (resident*postreform). The interaction term can be interpreted as a differential change over time comparing resident and hospitalist services. In all models, we adjusted for patient age, gender, race, and case mix index.

To determine whether prereform trends were similar among the resident and hospitalist services, we performed a test of controls as described by Volpp and colleagues.[6] Interaction terms for resident service and prereform years 2010 and 2011 were added to the model. A Wald test was then used to test for improved model fit, which would indicate differential trends among resident and hospitalist services during the prereform period. Where such trends were found, postreform results were compared only to 2011 rather than the 2009 to 2011 prereform period.[6]

To account for correlation within patients who had multiple discharges, we used a clustering approach and estimated robust variances.[24] From the regression model results, we calculated predicted probabilities adjusted for relevant covariates and prepost differences, and used linear probability models to estimate percentage‐point differences in outcomes, comparing residents and hospitalists in the pre‐ and postreform periods.[25] All analyses were performed using Stata/IC version 11 (StataCorp, College Station, TX).

RESULTS

In the 3 years before the 2011 residency work‐hour reforms were implemented (prereform), there were a total of 15,688 discharges for 8983 patients to the resident services and 4622 discharges for 3649 patients to the hospitalist services. In the year following implementation of residency work‐hour changes (postreform), there were 5253 discharges for 3805 patients to the resident services and 1767 discharges for 1454 patients to the hospitalist service. Table 1 shows the characteristics of patients discharged from the resident and hospitalist services in the pre‐ and postreform periods. Patients discharged from the resident services were more likely to be older, male, African American, and have a higher CMI.

Demographics and Case Mix Index of Patients Discharged From Resident and Hospitalist (Nonresident) General Medicine Services 20092012 at Johns Hopkins Hospital
 Resident ServicesHospitalist Service 
 20092010201120122009201020112012P Valuea

  • NOTE: Abbreviations: SD, standard deviation.

  • Comparing patients admitted to resident versus hospitalist service over the length of the study period 2009 to 2012. Case mix index range for this sample was 0.2 to 21.9 (SD 0.9). Higher case mix index indicates higher risk of mortality.

Discharges, n53455299504452531366149217641767 
Unique patients, n30822968293338051106118013631454 
Age, y, mean (SD)55.1 (17.7)55.7 (17.4)56.4 (17.9)56.7 (17.1)55.9 (17.9)56.2 (18.4)55.5 (18.8)54 (18.7)0.02
Sex male, n (%)1503 (48.8)1397 (47.1)1432 (48.8)1837 (48.3)520 (47)550 (46.6)613 (45)654 (45)<0.01
Race         
African American, n (%)2072 (67.2)1922 (64.8)1820 (62.1)2507 (65.9)500 (45.2)592 (50.2)652 (47.8)747 (51.4)<0.01
White, n (%)897 (29.1)892 (30.1)957 (32.6)1118 (29.4)534 (48.3)527 (44.7)621 (45.6)619 (42.6) 
Asian, n (%)19 (.6%)35 (1.2)28 (1)32 (.8)11 (1)7 (.6)25 (1.8)12 (.8) 
Other, n (%)94 (3.1)119 (4)128 (4.4)148 (3.9)61 (5.5)54 (4.6)65 (4.8)76 (5.2) 
Case mix index, mean (SD)1.2 (1)1.1 (0.9)1.1 (0.9)1.1 (1.2)1.2 (1)1.1 (1)1.1 (1)1 (0.7)<0.01

Differences in Outcomes Among Resident and Hospitalist Services Pre‐ and Postreform

Table 2 shows unadjusted results. Patients discharged from the resident services in the postreform period as compared to the prereform period had a higher likelihood of an ICU stay (5.9% vs 4.5%, P<0.01), and lower likelihood of 30‐day readmission (17.1% vs 20.1%, P<0.01). Patients discharged from the hospitalist service in the postreform period as compared to the prereform period had a significantly shorter mean length of stay (4.51 vs 4.88 days, P=0.03)

Unadjusted Patient Safety Outcomes by Year and Service
 Resident ServicesHospitalist Service
OutcomePrereformaPostreformP ValuePrereformaPostreformP Value

  • NOTE: Abbreviations: ICU, intensive care unit; MHACs, Maryland Hospital Acquired Conditions.

  • For the outcomes length of stay and ICU admission, the postreform period was compared to 2011 only. For MHACs, readmissions, and mortality the postreform period was compared to 2009 to 2011.

Length of stay (mean)4.55 (5.39)4.50 (5.47)0.614.88 (5.36)4.51 (4.64)0.03
Any ICU stay (%)225 (4.5%)310 (5.9%)<0.0182 (4.7%)83 (4.7%)0.95
Any MHACs (%)560 (3.6%)180 (3.4%)0.62210 (4.5%)64 (3.6%)0.09
Readmit in 30 days (%)3155 (20.1%)900 (17.1%)<0.01852 (18.4%)296 (16.8%)0.11
Inpatient mortality (%)71 (0.5%)28 (0.5%)0.4818 (0.4%)7 (0.4%)0.97

Table 3 presents the results of regression analyses examining correlates of patient safety outcomes, adjusted for age, gender, race, and CMI. As the test of controls indicated differential prereform trends for ICU admission and length of stay, the prereform period was limited to 2011 for these outcomes. After adjustment for covariates, the probability of an ICU stay remained greater, and the 30‐day readmission rate was lower among patients discharged from resident services in the postreform period than the prereform period. Among patients discharged from the hospitalist services, there were no significant differences in length of stay, readmissions, ICU admissions, MHACs, or inpatient mortality comparing the pre‐ and postreform periods.

Adjusted Changes in Patient Safety Outcomes by Year and Service
 Resident ServicesHospitalist ServiceDifference in Differences
OutcomePrereformaPostreformDifferencePrereformPostreformDifference(ResidentHospitalist)

  • NOTE: Predicted probabilities and 95% confidence intervals were obtained via margins command. Logistic regression was used for dichotomous outcomes and linear regression for continuous outcomes, adjusted for case mix index, age, race, gender, and clustering at patient level.

  • Abbreviations: ICU, intensive care unit; MHACs, Maryland Hospital Acquired Conditions.

  • For the outcomes length of stay and ICU admission, the postreform period was compared to 2011 only. For MHACs, readmissions, and mortality, the postreform period was compared to 2009 to 2011.

ICU stay4.5% (4.0% to 5.1%)5.7% (5.1% to 6.3%)1.4% (0.5% to 2.2%)4.4% (3.5% to 5.3%)5.3% (4.3% to 6.3%)1.1% (0.2 to 2.4%)0.3% (1.1% to 1.8%)
Inpatient mortality0.5% (0.4% to 0.6%)0.5% (0.3% to 0.7%)0 (0.2% to 0.2%)0.3% (0.2% to 0.6%)0.5% (0.1% to 0.8%)0.1% (0.3% to 0.5%)0.1% (0.5% to 0.3%)
MHACs3.6% (3.3% to 3.9%)3.3% (2.9% to 3.7%)0.4% (0.9 to 0.2%)4.5% (3.9% to 5.1%)4.1% (3.2% to 5.1%)0.3% (1.4% to 0.7%)0.2% (1.0% to 1.3%)
Readmit 30 days20.1% (19.1% to 21.1%)17.2% (15.9% to 18.5%)2.8% (4.3% to 1.3%)18.4% (16.5% to 20.2%)16.6% (14.7% to 18.5%)1.7% (4.1% to 0.8%)1.8% (0.2% to 3.7%)
Length of stay4.6 (4.4 to 4.7)4.4 (4.3 to 4.6)0.1 (0.3 to 0.1)4.9 (4.6 to 5.1)4.7 (4.5 to 5.0)0.1 (0.4 to 0.2)0.01 (0.37 to 0.34)

Differences in Outcomes Comparing Resident and Hospitalist Services Pre‐ and Postreform

Comparing pre‐ and postreform periods in the resident and hospitalist services, there were no significant differences in ICU admission, length of stay, MHACs, 30‐day readmissions, or inpatient mortality. In the sensitivity analysis, in which we excluded all discharges in May 2012 to June 2012, results were not significantly different for any of the outcomes examined.

DISCUSSION

Using difference‐in‐differences estimation, we evaluated whether the implementation of the 2011 residency work‐hour mandate was associated with differences in patient safety outcomes including length of stay, 30‐day readmission, inpatient mortality, MHACs, and ICU admissions comparing resident and hospitalist services at a large academic medical center. Adjusting for patient age, race, gender, and clinical complexity, we found no significant changes in any of the patient safety outcomes indicators in the postreform period comparing resident to hospitalist services.

Our quasiexperimental study design allowed us to gauge differences in patient safety outcomes, while reducing bias due to unmeasured confounders that might impact patient safety indicators.[9] We were able to examine all discharges from the resident and hospitalist general medicine services during the academic years 2009 to 2012, while adjusting for age, race, gender, and clinical complexity. Though ICU admission was higher and readmission rates were lower on the resident services post‐2011, we did not observe a significant difference in ICU admission or 30‐day readmission rates in the postreform period comparing patients discharged from the resident and hospitalist services and all patients in the prereform period.

Our neutral findings differ from some other single‐institution evaluations of reduced resident work hours, several of which have shown improved quality of life, education, and patient safety indicators.[18, 21, 26, 27, 28] It is unclear why improvements in patient safety were not identified in the current study. The 2011 reforms were more broad‐based than some of the preliminary studies of reduced work hours, and therefore additional variables may be at play. For instance, challenges related to decreased work hours, including the increased number of handoffs in care and work compression, may require specific interventions to produce sustained improvements in patient safety.[3, 14, 29, 30]

Improving patient safety requires more than changing resident work hours. Blum et al. recommended enhanced funding to increase supervision, decrease resident caseload, and incentivize achievement of quality indicators to achieve the goal of improved patient safety within work‐hour reform.[31] Schumacher et al. proposed a focus on supervision, professionalism, safe transitions of care, and optimizing workloads as a means to improve patient safety and education within the new residency training paradigm.[29]

Limitations of this study include limited follow‐up time after implementation of the work‐hour reforms. It may take more time to optimize systems of care to see benefits in patient safety indicators. This was a single‐institution study of a limited number of outcomes in a single department, which limits generalizability and may reflect local experience rather than broader trends. The call schedule on the resident service in this study differs from programs that have adopted night float schedules. [27] This may have had an effect on patient care outcomes.[32] In an attempt to conduct a timely study of inpatient safety indicators following the 2011 changes, our study was not powered to detect small changes in low‐frequency outcomes such as mortality; longer‐term studies at multiple institutions will be needed to answer these key questions. We limited the prereform period where our test of controls indicated differential prereform trends, which reduced power.

As this was an observational study rather than an experiment, there may have been both measured and unmeasured differences in patient characteristics and comorbidity between the intervention and control group. For example, CMI was lower on the hospitalist service than the resident services. Demographics varied somewhat between services; male and African American patients were more likely to be discharged from resident services than hospitalist services for unknown reasons. Although we adjusted for demographics and CMI in our model, there may be residual confounding. Limitations in data collection did not allow us to separate patients initially admitted to the ICU from patients transferred to the ICU from the inpatient floors. We attempted to overcome this limitation through use of a difference‐in‐differences model to account for secular trends, but factors other than residency work hours may have impacted the resident and hospitalist services differentially. For example, hospital quality‐improvement programs or provider‐level factors may have differentially impacted the resident versus hospitalist services during the study period.

Work‐hour limitations for residents were established to improve residency education and patient safety. As noted by the Institute of Medicine, improving patient safety will require significant investment by program directors, hospitals, and the public to keep resident caseloads manageable, ensure adequate supervision of first‐year residents, train residents on safe handoffs in care, and conduct ongoing evaluations of patient safety and any unintended consequences of the regulations.[33] In the first year after implementation of the 2011 work‐hour reforms, we found no change in ICU admission, inpatient mortality, 30‐day readmission rates, length of stay, or MHACs compared with patients treated by hospitalists. Studies of the long‐term impact of residency work‐hour reform are necessary to determine whether changes in work hours have been associated with improvement in resident education and patient safety.

Disclosure: Nothing to report.

Files
Supplementary information (1)
References
  1. Accreditation Council for Graduate Medical Education. Common program requirements effective: July 1, 2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramResources/Common_Program_Requirements_07012011[1].pdf. Accessed February 10, 2014.
  2. Nasca TJ, Day SH, Amis ES. The new recommendations on duty hours from the ACGME Task Force. N Engl J Med. 2010;363:e3.
  3. Landrigan CP, Barger LK, Cade BE, Ayas NT, Czeisler CA. Interns' compliance with Accreditation Council for Graduate Medical Education work‐hour limits. JAMA. 2006;296(9):1063–1070.
  4. Fletcher KE, Underwood W, Davis SQ, Mangulkar RS, McMahon LF, Saint S. Effects of work hour reduction on residents' lives: a systematic review. JAMA. 2005;294(9):1088–1100.
  5. Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects of the ACGME duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122(2):250–258.
  6. Volpp KG, Small DS, Romano PS. Teaching hospital five‐year mortality trends in the wake of duty hour reforms. J Gen Intern Med. 2013;28(8):1048–1055.
  7. Philibert I, Nasca T, Brigham T, Shapiro J. Duty hour limits and patient care and resident outcomes: can high‐quality studies offer insight into complex relationships? Ann Rev Med. 2013;64:467–483.
  8. Fletcher KE, Reed DA, Arora VM. Patient safety, resident education and resident well‐being following implementation of the 2003 ACGME duty hour rules. J Gen Intern Med. 2011;26(8):907–919.
  9. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298(9):975–983.
  10. Rosen AK, Loveland SA, Romano PS, et al. Effects of resident duty hour reform on surgical and procedural patient safety indicators among hospitalized Veterans Health Administration and Medicare patients. Med Care. 2009;47(7):723–731.
  11. Schuh LA, Khan MA, Harle H, et al. Pilot trial of IOM duty hour recommendations in neurology residency programs. Neurology. 2011;77(9):883–887.
  12. McCoy CP, Halvorsen AJ, Loftus CG, et al. Effect of 16‐hour duty periods of patient care and resident education. Mayo Clin Proc. 2011;86:192–196.
  13. Sen S, Kranzler HR, Didwania AK, et al. Effects of the 2011 duty hour reforms on interns and their patients: a prospective longitudinal cohort study. JAMA Intern Med. 2013;173(8):657–662.
  14. Desai SV, Feldman L, Brown L, et al. Effect of the 2011 vs 2003 duty hour regulation—compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff. JAMA Intern Med. 2013;173(8):649–655.
  15. Drolet BC, Christopher DA, Fischer SA. Residents' response to duty‐hour regulations—a follow‐up national survey. N Engl J Med. 2012;366:e35.
  16. Drolet BS, Sangisetty S, Tracy TF, Cioffi WG. Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations. JAMA Surg. 2013;148(5):427–433.
  17. Drolet BC, Khokhar MT, Fischer SA. The 2011 duty hour requirements—a survey of residency program directors. N Engl J Med. 2013;368:694–697.
  18. Theobald CN, Stover DG, Choma NN, et al. The effect of reducing maximum shift lengths to 16 hours on internal medicine interns' educational opportunities. Acad Med. 2013;88(4):512–518.
  19. Nuckols TK, Escarce JJ. Residency work‐hours reform. A cost analysis including preventable adverse events. J Gen Intern Med. 2005;20(10):873–878.
  20. Nuckols TK, Bhattacharya J, Wolman DM, Ulmer C, Escarce JJ. Cost implications of reduced work hours and workloads for resident physicians. N Engl J Med. 2009;360:2202–2215.
  21. Horwitz LI, Kosiborod M, Lin Z, Krumholz HM. Changes in outcomes for internal medicine inpatients after work‐hour regulations. Ann Intern Med. 2007;147:97–103.
  22. .Maryland Health Services Cost Review Commission. Complications: Maryland Hospital Acquired Conditions. Available at: http://www.hscrc.state.md.us/init_qi_MHAC.cfm. Accessed May 23, 2013.
  23. Averill R, Goldfield N, Hughes J, et al. What are APR‐DRGs? An introduction to severity of illness and risk of mortality adjustment methodology. 3M Health Information Systems. Available at: http://solutions.3m.com/3MContentRetrievalAPI/BlobServlet?locale=it_IT44(4):1049–1060.
  24. Ross JS, Wang R, Long JB, Gross CP, Ma X. Impact of the 2008 US Preventive Services Task Force Recommendation to discontinue prostate cancer screening among male Medicare beneficiaries. Arch Intern Med. 2012;172(20):1601–1603.
  25. Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns' work hour on serious medical errors in intensive care units. N Engl J Med. 2004;351(18):1838–1848.
  26. Levine AC, Adusumilli J, Landrigan CP. Effects of reducing or eliminating resident work shifts over 16 hours: a systematic review. Sleep. 2010;33(8):1043–1053.
  27. Bhavsar J, Montgomery D, Li J, et al. Impact of duty hours restrictions on quality of care and clinical outcomes. Am J Med. 2007;120(11):968–974.
  28. Schumacher DJ, Slovein SR, Riebschleger MP, Englander R, Hicks P, Carraccio C. Beyond counting hours: the importance of supervision, professionalism, transitions in care, and workload in residency training. Acad Med. 2012;87(7):883–888.
  29. Tessing S, Amendt A, Jennings J, Thomson J, Auger KA, Gonzalez del Rey JA. One possible future for resident hours: interns' perspective on a one‐month trial of the Institute of Medicine recommended duty hour limits. J Grad Med Educ. 2009;1(2):185–187.
  30. Blum AB, Shea S, Czeisler CA, Landrigan CP, Leape L. Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety. Nature Sci Sleep. 2001;3:47–85.
  31. Bricker DA, Markert RJ. Night float teaching and learning: perceptions of residents and faculty. J Grad Med Educ. 2010;2(2):236–241.
  32. Institute of Medicine. Resident duty hours: enhancing sleep, supervision, and safety. Report brief. Washington, DC: National Academies; 2008. Available at: http://www.iom.edu/∼/media/Files/Report Files/2008/Resident‐Duty‐Hours/residency hours revised for web.pdf. Accessed May 23, 2013.
Article PDF
jhm2171.pdf
Issue
Journal of Hospital Medicine - 9(6)
Publications
Journal of Hospital Medicine
Page Number
347-352
Sections
Original Research
Author(s)
Lauren Block, MD, MPH
Marian Jarlenski, MPH
Albert W. Wu, MD, MPH
Leonard Feldman, MD
Joseph Conigliaro, MD, MPH
Jenna Swann, MS
Sanjay V. Desai, MD
Author(s)
Lauren Block, MD, MPH
Marian Jarlenski, MPH
Albert W. Wu, MD, MPH
Leonard Feldman, MD
Joseph Conigliaro, MD, MPH
Jenna Swann, MS
Sanjay V. Desai, MD
Files
Supplementary information (1)
Files
Supplementary information (1)
Article PDF
jhm2171.pdf
Article PDF
jhm2171.pdf

The Accreditation Council for Graduate Medical Education (ACGME) Common Program Requirements implemented in July 2011 increased supervision requirements and limited continuous work hours for first‐year residents.[1] Similar to the 2003 mandates, these requirements were introduced to improve patient safety and education at academic medical centers.[2] Work‐hour reforms have been associated with decreased resident burnout and improved sleep.[3, 4, 5] However, national observational studies and systematic reviews of the impact of the 2003 reforms on patient safety and quality of care have been varied in terms of outcome.[6, 7, 8, 9, 10] Small studies of the 2011 recommendations have shown increased sleep duration and decreased burnout, but also an increased number of handoffs and increased resident concerns about making a serious medical error.[11, 12, 13, 14] Although national surveys of residents and program directors have not indicated improvements in education or quality of life, 1 observational study did show improvement in clinical exposure and conference attendance.[15, 16, 17, 18] The impact of the 2011 reforms on patient safety remains unclear.[19, 20]

The objective of this study was to evaluate the association between implementation of the 2011 residency work‐hour mandates and patient safety outcomes at a large academic medical center.

METHODS

Study Design

This observational study used a quasi‐experimental difference‐in‐differences approach to evaluate whether residency work‐hour changes were associated with patient safety outcomes among general medicine inpatients. We compared safety outcomes among adult patients discharged from resident general medical services (referred to as resident) to safety outcomes among patients discharged by the hospitalist general medical service (referred to as hospitalist) before and after the 2011 residency work‐hour reforms at a large academic medical center. Differences in outcomes for the resident group were compared to differences observed in the hospitalist group, with adjustment for relevant demographic and case mix factors.[21] We used the hospitalist service as a control group, because ACGME changes applied only to resident services. The strength of this design is that it controls for secular trends that are correlated with patient safety, impacting both residents and hospitalists similarly.[9]

Approval for this study and a Health Insurance Portability and Accountability Act waiver were granted by the Johns Hopkins University School of Medicine institutional review board. We retrospectively examined administrative data on all patient discharges from the general medicine services at Johns Hopkins Hospital between July 1, 2008 and June 30, 2012 that were identified as pertaining to resident or hospitalist services.

Patient Allocation and Physician Scheduling

Patient admission to the resident or hospitalist service was decided by a number of factors. To maintain continuity of care, patients were preferentially admitted to the same service as for prior admissions. New patients were admitted to a service based on bed availability, nurse staffing, patient gender, isolation precautions, and cardiac monitor availability.

The inpatient resident services were staffed prior to July 2011 using a traditional 30‐hour overnight call system. Following July 2011, the inpatient resident services were staffed using a modified overnight call system, in which interns took overnight calls from 8 pm until 12 pm the following day, once every 5 nights with supervision by upper‐level residents. These interns rotated through daytime admitting and coverage roles on the intervening days. The hospitalist service was organized into a 3‐physician rotation of day shift, evening shift, and overnight shift.

Data and Outcomes

Twenty‐nine percent of patients in the sample were admitted more than once during the study period, and patients were generally admitted to the same resident team during each admission. Patients with multiple admissions were counted multiple times in the model. We categorized admissions as prereform (July 1, 2008June 30, 2011) and postreform (July 1, 2011June 30, 2012). Outcomes evaluated included hospital length of stay, 30‐day readmission, intensive care unit stay (ICU) stay, inpatient mortality, and number of Maryland Hospital Acquired Conditions (MHACs). ICU stay pertained to any ICU admission including initial admission and transfer from the inpatient floor. MHACs are a set of inpatient performance indicators derived from a list of 64 inpatient Potentially Preventable Complications developed by 3M Health Information Systems.[22] MHACs are used by the Maryland Health Services Cost Review Commission to link hospital payment to performance for costly, preventable, and clinically relevant complications. MHACs were coded in our analysis as a dichotomous variable. Independent variables included patient age at admission, race, gender, and case mix index. Case mix index (CMI) is a numeric score that measures resource utilization for a specific patient population. CMI is a weighted value assigned to patients based on resource utilization and All Patient Refined Diagnostic Related Group and was included as an indicator of patient illness severity and risk of mortality.[23] Data were obtained from administrative records from the case mix research team at Johns Hopkins Medicine.

To account for transitional differences that may have coincided with the opening of a new hospital wing in late April 2012, we conducted a sensitivity analysis, in which we excluded from analysis any visits that took place in May 2012 to June 2012.

Data Analysis

Based on historical studies, we calculated that a sample size of at least 3600 discharges would allow us to detect a difference of 5% between the pre‐ and postreform period assuming baseline 20% occurrence of dichotomous outcomes (=0.05; =0.2; r=4).[21]

The primary unit of analysis was the hospital discharge. Similar to Horwitz et al., we analyzed data using a difference‐in‐differences estimation strategy.[21] We used multivariable linear regression for length of stay measured as a continuous variable, and multivariable logistic regression for inpatient mortality, 30‐day readmission, MHACs coded as a dichotomous variable, and ICU stay coded as a dichotomous variable.[9] The difference‐in‐differences estimation was used to determine whether the postreform period relative to prereform period was associated with differences in outcomes comparing resident and hospitalist services. In the regression models, the independent variables of interest included an indicator variable for whether a patient was treated on a resident service, an indicator variable for whether a patient was discharged in the postreform period, and the interaction of these 2 variables (resident*postreform). The interaction term can be interpreted as a differential change over time comparing resident and hospitalist services. In all models, we adjusted for patient age, gender, race, and case mix index.

To determine whether prereform trends were similar among the resident and hospitalist services, we performed a test of controls as described by Volpp and colleagues.[6] Interaction terms for resident service and prereform years 2010 and 2011 were added to the model. A Wald test was then used to test for improved model fit, which would indicate differential trends among resident and hospitalist services during the prereform period. Where such trends were found, postreform results were compared only to 2011 rather than the 2009 to 2011 prereform period.[6]

To account for correlation within patients who had multiple discharges, we used a clustering approach and estimated robust variances.[24] From the regression model results, we calculated predicted probabilities adjusted for relevant covariates and prepost differences, and used linear probability models to estimate percentage‐point differences in outcomes, comparing residents and hospitalists in the pre‐ and postreform periods.[25] All analyses were performed using Stata/IC version 11 (StataCorp, College Station, TX).

RESULTS

In the 3 years before the 2011 residency work‐hour reforms were implemented (prereform), there were a total of 15,688 discharges for 8983 patients to the resident services and 4622 discharges for 3649 patients to the hospitalist services. In the year following implementation of residency work‐hour changes (postreform), there were 5253 discharges for 3805 patients to the resident services and 1767 discharges for 1454 patients to the hospitalist service. Table 1 shows the characteristics of patients discharged from the resident and hospitalist services in the pre‐ and postreform periods. Patients discharged from the resident services were more likely to be older, male, African American, and have a higher CMI.

Demographics and Case Mix Index of Patients Discharged From Resident and Hospitalist (Nonresident) General Medicine Services 20092012 at Johns Hopkins Hospital
 Resident ServicesHospitalist Service 
 20092010201120122009201020112012P Valuea

  • NOTE: Abbreviations: SD, standard deviation.

  • Comparing patients admitted to resident versus hospitalist service over the length of the study period 2009 to 2012. Case mix index range for this sample was 0.2 to 21.9 (SD 0.9). Higher case mix index indicates higher risk of mortality.

Discharges, n53455299504452531366149217641767 
Unique patients, n30822968293338051106118013631454 
Age, y, mean (SD)55.1 (17.7)55.7 (17.4)56.4 (17.9)56.7 (17.1)55.9 (17.9)56.2 (18.4)55.5 (18.8)54 (18.7)0.02
Sex male, n (%)1503 (48.8)1397 (47.1)1432 (48.8)1837 (48.3)520 (47)550 (46.6)613 (45)654 (45)<0.01
Race         
African American, n (%)2072 (67.2)1922 (64.8)1820 (62.1)2507 (65.9)500 (45.2)592 (50.2)652 (47.8)747 (51.4)<0.01
White, n (%)897 (29.1)892 (30.1)957 (32.6)1118 (29.4)534 (48.3)527 (44.7)621 (45.6)619 (42.6) 
Asian, n (%)19 (.6%)35 (1.2)28 (1)32 (.8)11 (1)7 (.6)25 (1.8)12 (.8) 
Other, n (%)94 (3.1)119 (4)128 (4.4)148 (3.9)61 (5.5)54 (4.6)65 (4.8)76 (5.2) 
Case mix index, mean (SD)1.2 (1)1.1 (0.9)1.1 (0.9)1.1 (1.2)1.2 (1)1.1 (1)1.1 (1)1 (0.7)<0.01

Differences in Outcomes Among Resident and Hospitalist Services Pre‐ and Postreform

Table 2 shows unadjusted results. Patients discharged from the resident services in the postreform period as compared to the prereform period had a higher likelihood of an ICU stay (5.9% vs 4.5%, P<0.01), and lower likelihood of 30‐day readmission (17.1% vs 20.1%, P<0.01). Patients discharged from the hospitalist service in the postreform period as compared to the prereform period had a significantly shorter mean length of stay (4.51 vs 4.88 days, P=0.03)

Unadjusted Patient Safety Outcomes by Year and Service
 Resident ServicesHospitalist Service
OutcomePrereformaPostreformP ValuePrereformaPostreformP Value

  • NOTE: Abbreviations: ICU, intensive care unit; MHACs, Maryland Hospital Acquired Conditions.

  • For the outcomes length of stay and ICU admission, the postreform period was compared to 2011 only. For MHACs, readmissions, and mortality the postreform period was compared to 2009 to 2011.

Length of stay (mean)4.55 (5.39)4.50 (5.47)0.614.88 (5.36)4.51 (4.64)0.03
Any ICU stay (%)225 (4.5%)310 (5.9%)<0.0182 (4.7%)83 (4.7%)0.95
Any MHACs (%)560 (3.6%)180 (3.4%)0.62210 (4.5%)64 (3.6%)0.09
Readmit in 30 days (%)3155 (20.1%)900 (17.1%)<0.01852 (18.4%)296 (16.8%)0.11
Inpatient mortality (%)71 (0.5%)28 (0.5%)0.4818 (0.4%)7 (0.4%)0.97

Table 3 presents the results of regression analyses examining correlates of patient safety outcomes, adjusted for age, gender, race, and CMI. As the test of controls indicated differential prereform trends for ICU admission and length of stay, the prereform period was limited to 2011 for these outcomes. After adjustment for covariates, the probability of an ICU stay remained greater, and the 30‐day readmission rate was lower among patients discharged from resident services in the postreform period than the prereform period. Among patients discharged from the hospitalist services, there were no significant differences in length of stay, readmissions, ICU admissions, MHACs, or inpatient mortality comparing the pre‐ and postreform periods.

Adjusted Changes in Patient Safety Outcomes by Year and Service
 Resident ServicesHospitalist ServiceDifference in Differences
OutcomePrereformaPostreformDifferencePrereformPostreformDifference(ResidentHospitalist)

  • NOTE: Predicted probabilities and 95% confidence intervals were obtained via margins command. Logistic regression was used for dichotomous outcomes and linear regression for continuous outcomes, adjusted for case mix index, age, race, gender, and clustering at patient level.

  • Abbreviations: ICU, intensive care unit; MHACs, Maryland Hospital Acquired Conditions.

  • For the outcomes length of stay and ICU admission, the postreform period was compared to 2011 only. For MHACs, readmissions, and mortality, the postreform period was compared to 2009 to 2011.

ICU stay4.5% (4.0% to 5.1%)5.7% (5.1% to 6.3%)1.4% (0.5% to 2.2%)4.4% (3.5% to 5.3%)5.3% (4.3% to 6.3%)1.1% (0.2 to 2.4%)0.3% (1.1% to 1.8%)
Inpatient mortality0.5% (0.4% to 0.6%)0.5% (0.3% to 0.7%)0 (0.2% to 0.2%)0.3% (0.2% to 0.6%)0.5% (0.1% to 0.8%)0.1% (0.3% to 0.5%)0.1% (0.5% to 0.3%)
MHACs3.6% (3.3% to 3.9%)3.3% (2.9% to 3.7%)0.4% (0.9 to 0.2%)4.5% (3.9% to 5.1%)4.1% (3.2% to 5.1%)0.3% (1.4% to 0.7%)0.2% (1.0% to 1.3%)
Readmit 30 days20.1% (19.1% to 21.1%)17.2% (15.9% to 18.5%)2.8% (4.3% to 1.3%)18.4% (16.5% to 20.2%)16.6% (14.7% to 18.5%)1.7% (4.1% to 0.8%)1.8% (0.2% to 3.7%)
Length of stay4.6 (4.4 to 4.7)4.4 (4.3 to 4.6)0.1 (0.3 to 0.1)4.9 (4.6 to 5.1)4.7 (4.5 to 5.0)0.1 (0.4 to 0.2)0.01 (0.37 to 0.34)

Differences in Outcomes Comparing Resident and Hospitalist Services Pre‐ and Postreform

Comparing pre‐ and postreform periods in the resident and hospitalist services, there were no significant differences in ICU admission, length of stay, MHACs, 30‐day readmissions, or inpatient mortality. In the sensitivity analysis, in which we excluded all discharges in May 2012 to June 2012, results were not significantly different for any of the outcomes examined.

DISCUSSION

Using difference‐in‐differences estimation, we evaluated whether the implementation of the 2011 residency work‐hour mandate was associated with differences in patient safety outcomes including length of stay, 30‐day readmission, inpatient mortality, MHACs, and ICU admissions comparing resident and hospitalist services at a large academic medical center. Adjusting for patient age, race, gender, and clinical complexity, we found no significant changes in any of the patient safety outcomes indicators in the postreform period comparing resident to hospitalist services.

Our quasiexperimental study design allowed us to gauge differences in patient safety outcomes, while reducing bias due to unmeasured confounders that might impact patient safety indicators.[9] We were able to examine all discharges from the resident and hospitalist general medicine services during the academic years 2009 to 2012, while adjusting for age, race, gender, and clinical complexity. Though ICU admission was higher and readmission rates were lower on the resident services post‐2011, we did not observe a significant difference in ICU admission or 30‐day readmission rates in the postreform period comparing patients discharged from the resident and hospitalist services and all patients in the prereform period.

Our neutral findings differ from some other single‐institution evaluations of reduced resident work hours, several of which have shown improved quality of life, education, and patient safety indicators.[18, 21, 26, 27, 28] It is unclear why improvements in patient safety were not identified in the current study. The 2011 reforms were more broad‐based than some of the preliminary studies of reduced work hours, and therefore additional variables may be at play. For instance, challenges related to decreased work hours, including the increased number of handoffs in care and work compression, may require specific interventions to produce sustained improvements in patient safety.[3, 14, 29, 30]

Improving patient safety requires more than changing resident work hours. Blum et al. recommended enhanced funding to increase supervision, decrease resident caseload, and incentivize achievement of quality indicators to achieve the goal of improved patient safety within work‐hour reform.[31] Schumacher et al. proposed a focus on supervision, professionalism, safe transitions of care, and optimizing workloads as a means to improve patient safety and education within the new residency training paradigm.[29]

Limitations of this study include limited follow‐up time after implementation of the work‐hour reforms. It may take more time to optimize systems of care to see benefits in patient safety indicators. This was a single‐institution study of a limited number of outcomes in a single department, which limits generalizability and may reflect local experience rather than broader trends. The call schedule on the resident service in this study differs from programs that have adopted night float schedules. [27] This may have had an effect on patient care outcomes.[32] In an attempt to conduct a timely study of inpatient safety indicators following the 2011 changes, our study was not powered to detect small changes in low‐frequency outcomes such as mortality; longer‐term studies at multiple institutions will be needed to answer these key questions. We limited the prereform period where our test of controls indicated differential prereform trends, which reduced power.

As this was an observational study rather than an experiment, there may have been both measured and unmeasured differences in patient characteristics and comorbidity between the intervention and control group. For example, CMI was lower on the hospitalist service than the resident services. Demographics varied somewhat between services; male and African American patients were more likely to be discharged from resident services than hospitalist services for unknown reasons. Although we adjusted for demographics and CMI in our model, there may be residual confounding. Limitations in data collection did not allow us to separate patients initially admitted to the ICU from patients transferred to the ICU from the inpatient floors. We attempted to overcome this limitation through use of a difference‐in‐differences model to account for secular trends, but factors other than residency work hours may have impacted the resident and hospitalist services differentially. For example, hospital quality‐improvement programs or provider‐level factors may have differentially impacted the resident versus hospitalist services during the study period.

Work‐hour limitations for residents were established to improve residency education and patient safety. As noted by the Institute of Medicine, improving patient safety will require significant investment by program directors, hospitals, and the public to keep resident caseloads manageable, ensure adequate supervision of first‐year residents, train residents on safe handoffs in care, and conduct ongoing evaluations of patient safety and any unintended consequences of the regulations.[33] In the first year after implementation of the 2011 work‐hour reforms, we found no change in ICU admission, inpatient mortality, 30‐day readmission rates, length of stay, or MHACs compared with patients treated by hospitalists. Studies of the long‐term impact of residency work‐hour reform are necessary to determine whether changes in work hours have been associated with improvement in resident education and patient safety.

Disclosure: Nothing to report.

The Accreditation Council for Graduate Medical Education (ACGME) Common Program Requirements implemented in July 2011 increased supervision requirements and limited continuous work hours for first‐year residents.[1] Similar to the 2003 mandates, these requirements were introduced to improve patient safety and education at academic medical centers.[2] Work‐hour reforms have been associated with decreased resident burnout and improved sleep.[3, 4, 5] However, national observational studies and systematic reviews of the impact of the 2003 reforms on patient safety and quality of care have been varied in terms of outcome.[6, 7, 8, 9, 10] Small studies of the 2011 recommendations have shown increased sleep duration and decreased burnout, but also an increased number of handoffs and increased resident concerns about making a serious medical error.[11, 12, 13, 14] Although national surveys of residents and program directors have not indicated improvements in education or quality of life, 1 observational study did show improvement in clinical exposure and conference attendance.[15, 16, 17, 18] The impact of the 2011 reforms on patient safety remains unclear.[19, 20]

The objective of this study was to evaluate the association between implementation of the 2011 residency work‐hour mandates and patient safety outcomes at a large academic medical center.

METHODS

Study Design

This observational study used a quasi‐experimental difference‐in‐differences approach to evaluate whether residency work‐hour changes were associated with patient safety outcomes among general medicine inpatients. We compared safety outcomes among adult patients discharged from resident general medical services (referred to as resident) to safety outcomes among patients discharged by the hospitalist general medical service (referred to as hospitalist) before and after the 2011 residency work‐hour reforms at a large academic medical center. Differences in outcomes for the resident group were compared to differences observed in the hospitalist group, with adjustment for relevant demographic and case mix factors.[21] We used the hospitalist service as a control group, because ACGME changes applied only to resident services. The strength of this design is that it controls for secular trends that are correlated with patient safety, impacting both residents and hospitalists similarly.[9]

Approval for this study and a Health Insurance Portability and Accountability Act waiver were granted by the Johns Hopkins University School of Medicine institutional review board. We retrospectively examined administrative data on all patient discharges from the general medicine services at Johns Hopkins Hospital between July 1, 2008 and June 30, 2012 that were identified as pertaining to resident or hospitalist services.

Patient Allocation and Physician Scheduling

Patient admission to the resident or hospitalist service was decided by a number of factors. To maintain continuity of care, patients were preferentially admitted to the same service as for prior admissions. New patients were admitted to a service based on bed availability, nurse staffing, patient gender, isolation precautions, and cardiac monitor availability.

The inpatient resident services were staffed prior to July 2011 using a traditional 30‐hour overnight call system. Following July 2011, the inpatient resident services were staffed using a modified overnight call system, in which interns took overnight calls from 8 pm until 12 pm the following day, once every 5 nights with supervision by upper‐level residents. These interns rotated through daytime admitting and coverage roles on the intervening days. The hospitalist service was organized into a 3‐physician rotation of day shift, evening shift, and overnight shift.

Data and Outcomes

Twenty‐nine percent of patients in the sample were admitted more than once during the study period, and patients were generally admitted to the same resident team during each admission. Patients with multiple admissions were counted multiple times in the model. We categorized admissions as prereform (July 1, 2008June 30, 2011) and postreform (July 1, 2011June 30, 2012). Outcomes evaluated included hospital length of stay, 30‐day readmission, intensive care unit stay (ICU) stay, inpatient mortality, and number of Maryland Hospital Acquired Conditions (MHACs). ICU stay pertained to any ICU admission including initial admission and transfer from the inpatient floor. MHACs are a set of inpatient performance indicators derived from a list of 64 inpatient Potentially Preventable Complications developed by 3M Health Information Systems.[22] MHACs are used by the Maryland Health Services Cost Review Commission to link hospital payment to performance for costly, preventable, and clinically relevant complications. MHACs were coded in our analysis as a dichotomous variable. Independent variables included patient age at admission, race, gender, and case mix index. Case mix index (CMI) is a numeric score that measures resource utilization for a specific patient population. CMI is a weighted value assigned to patients based on resource utilization and All Patient Refined Diagnostic Related Group and was included as an indicator of patient illness severity and risk of mortality.[23] Data were obtained from administrative records from the case mix research team at Johns Hopkins Medicine.

To account for transitional differences that may have coincided with the opening of a new hospital wing in late April 2012, we conducted a sensitivity analysis, in which we excluded from analysis any visits that took place in May 2012 to June 2012.

Data Analysis

Based on historical studies, we calculated that a sample size of at least 3600 discharges would allow us to detect a difference of 5% between the pre‐ and postreform period assuming baseline 20% occurrence of dichotomous outcomes (=0.05; =0.2; r=4).[21]

The primary unit of analysis was the hospital discharge. Similar to Horwitz et al., we analyzed data using a difference‐in‐differences estimation strategy.[21] We used multivariable linear regression for length of stay measured as a continuous variable, and multivariable logistic regression for inpatient mortality, 30‐day readmission, MHACs coded as a dichotomous variable, and ICU stay coded as a dichotomous variable.[9] The difference‐in‐differences estimation was used to determine whether the postreform period relative to prereform period was associated with differences in outcomes comparing resident and hospitalist services. In the regression models, the independent variables of interest included an indicator variable for whether a patient was treated on a resident service, an indicator variable for whether a patient was discharged in the postreform period, and the interaction of these 2 variables (resident*postreform). The interaction term can be interpreted as a differential change over time comparing resident and hospitalist services. In all models, we adjusted for patient age, gender, race, and case mix index.

To determine whether prereform trends were similar among the resident and hospitalist services, we performed a test of controls as described by Volpp and colleagues.[6] Interaction terms for resident service and prereform years 2010 and 2011 were added to the model. A Wald test was then used to test for improved model fit, which would indicate differential trends among resident and hospitalist services during the prereform period. Where such trends were found, postreform results were compared only to 2011 rather than the 2009 to 2011 prereform period.[6]

To account for correlation within patients who had multiple discharges, we used a clustering approach and estimated robust variances.[24] From the regression model results, we calculated predicted probabilities adjusted for relevant covariates and prepost differences, and used linear probability models to estimate percentage‐point differences in outcomes, comparing residents and hospitalists in the pre‐ and postreform periods.[25] All analyses were performed using Stata/IC version 11 (StataCorp, College Station, TX).

RESULTS

In the 3 years before the 2011 residency work‐hour reforms were implemented (prereform), there were a total of 15,688 discharges for 8983 patients to the resident services and 4622 discharges for 3649 patients to the hospitalist services. In the year following implementation of residency work‐hour changes (postreform), there were 5253 discharges for 3805 patients to the resident services and 1767 discharges for 1454 patients to the hospitalist service. Table 1 shows the characteristics of patients discharged from the resident and hospitalist services in the pre‐ and postreform periods. Patients discharged from the resident services were more likely to be older, male, African American, and have a higher CMI.

Demographics and Case Mix Index of Patients Discharged From Resident and Hospitalist (Nonresident) General Medicine Services 20092012 at Johns Hopkins Hospital
 Resident ServicesHospitalist Service 
 20092010201120122009201020112012P Valuea

  • NOTE: Abbreviations: SD, standard deviation.

  • Comparing patients admitted to resident versus hospitalist service over the length of the study period 2009 to 2012. Case mix index range for this sample was 0.2 to 21.9 (SD 0.9). Higher case mix index indicates higher risk of mortality.

Discharges, n53455299504452531366149217641767 
Unique patients, n30822968293338051106118013631454 
Age, y, mean (SD)55.1 (17.7)55.7 (17.4)56.4 (17.9)56.7 (17.1)55.9 (17.9)56.2 (18.4)55.5 (18.8)54 (18.7)0.02
Sex male, n (%)1503 (48.8)1397 (47.1)1432 (48.8)1837 (48.3)520 (47)550 (46.6)613 (45)654 (45)<0.01
Race         
African American, n (%)2072 (67.2)1922 (64.8)1820 (62.1)2507 (65.9)500 (45.2)592 (50.2)652 (47.8)747 (51.4)<0.01
White, n (%)897 (29.1)892 (30.1)957 (32.6)1118 (29.4)534 (48.3)527 (44.7)621 (45.6)619 (42.6) 
Asian, n (%)19 (.6%)35 (1.2)28 (1)32 (.8)11 (1)7 (.6)25 (1.8)12 (.8) 
Other, n (%)94 (3.1)119 (4)128 (4.4)148 (3.9)61 (5.5)54 (4.6)65 (4.8)76 (5.2) 
Case mix index, mean (SD)1.2 (1)1.1 (0.9)1.1 (0.9)1.1 (1.2)1.2 (1)1.1 (1)1.1 (1)1 (0.7)<0.01

Differences in Outcomes Among Resident and Hospitalist Services Pre‐ and Postreform

Table 2 shows unadjusted results. Patients discharged from the resident services in the postreform period as compared to the prereform period had a higher likelihood of an ICU stay (5.9% vs 4.5%, P<0.01), and lower likelihood of 30‐day readmission (17.1% vs 20.1%, P<0.01). Patients discharged from the hospitalist service in the postreform period as compared to the prereform period had a significantly shorter mean length of stay (4.51 vs 4.88 days, P=0.03)

Unadjusted Patient Safety Outcomes by Year and Service
 Resident ServicesHospitalist Service
OutcomePrereformaPostreformP ValuePrereformaPostreformP Value

  • NOTE: Abbreviations: ICU, intensive care unit; MHACs, Maryland Hospital Acquired Conditions.

  • For the outcomes length of stay and ICU admission, the postreform period was compared to 2011 only. For MHACs, readmissions, and mortality the postreform period was compared to 2009 to 2011.

Length of stay (mean)4.55 (5.39)4.50 (5.47)0.614.88 (5.36)4.51 (4.64)0.03
Any ICU stay (%)225 (4.5%)310 (5.9%)<0.0182 (4.7%)83 (4.7%)0.95
Any MHACs (%)560 (3.6%)180 (3.4%)0.62210 (4.5%)64 (3.6%)0.09
Readmit in 30 days (%)3155 (20.1%)900 (17.1%)<0.01852 (18.4%)296 (16.8%)0.11
Inpatient mortality (%)71 (0.5%)28 (0.5%)0.4818 (0.4%)7 (0.4%)0.97

Table 3 presents the results of regression analyses examining correlates of patient safety outcomes, adjusted for age, gender, race, and CMI. As the test of controls indicated differential prereform trends for ICU admission and length of stay, the prereform period was limited to 2011 for these outcomes. After adjustment for covariates, the probability of an ICU stay remained greater, and the 30‐day readmission rate was lower among patients discharged from resident services in the postreform period than the prereform period. Among patients discharged from the hospitalist services, there were no significant differences in length of stay, readmissions, ICU admissions, MHACs, or inpatient mortality comparing the pre‐ and postreform periods.

Adjusted Changes in Patient Safety Outcomes by Year and Service
 Resident ServicesHospitalist ServiceDifference in Differences
OutcomePrereformaPostreformDifferencePrereformPostreformDifference(ResidentHospitalist)

  • NOTE: Predicted probabilities and 95% confidence intervals were obtained via margins command. Logistic regression was used for dichotomous outcomes and linear regression for continuous outcomes, adjusted for case mix index, age, race, gender, and clustering at patient level.

  • Abbreviations: ICU, intensive care unit; MHACs, Maryland Hospital Acquired Conditions.

  • For the outcomes length of stay and ICU admission, the postreform period was compared to 2011 only. For MHACs, readmissions, and mortality, the postreform period was compared to 2009 to 2011.

ICU stay4.5% (4.0% to 5.1%)5.7% (5.1% to 6.3%)1.4% (0.5% to 2.2%)4.4% (3.5% to 5.3%)5.3% (4.3% to 6.3%)1.1% (0.2 to 2.4%)0.3% (1.1% to 1.8%)
Inpatient mortality0.5% (0.4% to 0.6%)0.5% (0.3% to 0.7%)0 (0.2% to 0.2%)0.3% (0.2% to 0.6%)0.5% (0.1% to 0.8%)0.1% (0.3% to 0.5%)0.1% (0.5% to 0.3%)
MHACs3.6% (3.3% to 3.9%)3.3% (2.9% to 3.7%)0.4% (0.9 to 0.2%)4.5% (3.9% to 5.1%)4.1% (3.2% to 5.1%)0.3% (1.4% to 0.7%)0.2% (1.0% to 1.3%)
Readmit 30 days20.1% (19.1% to 21.1%)17.2% (15.9% to 18.5%)2.8% (4.3% to 1.3%)18.4% (16.5% to 20.2%)16.6% (14.7% to 18.5%)1.7% (4.1% to 0.8%)1.8% (0.2% to 3.7%)
Length of stay4.6 (4.4 to 4.7)4.4 (4.3 to 4.6)0.1 (0.3 to 0.1)4.9 (4.6 to 5.1)4.7 (4.5 to 5.0)0.1 (0.4 to 0.2)0.01 (0.37 to 0.34)

Differences in Outcomes Comparing Resident and Hospitalist Services Pre‐ and Postreform

Comparing pre‐ and postreform periods in the resident and hospitalist services, there were no significant differences in ICU admission, length of stay, MHACs, 30‐day readmissions, or inpatient mortality. In the sensitivity analysis, in which we excluded all discharges in May 2012 to June 2012, results were not significantly different for any of the outcomes examined.

DISCUSSION

Using difference‐in‐differences estimation, we evaluated whether the implementation of the 2011 residency work‐hour mandate was associated with differences in patient safety outcomes including length of stay, 30‐day readmission, inpatient mortality, MHACs, and ICU admissions comparing resident and hospitalist services at a large academic medical center. Adjusting for patient age, race, gender, and clinical complexity, we found no significant changes in any of the patient safety outcomes indicators in the postreform period comparing resident to hospitalist services.

Our quasiexperimental study design allowed us to gauge differences in patient safety outcomes, while reducing bias due to unmeasured confounders that might impact patient safety indicators.[9] We were able to examine all discharges from the resident and hospitalist general medicine services during the academic years 2009 to 2012, while adjusting for age, race, gender, and clinical complexity. Though ICU admission was higher and readmission rates were lower on the resident services post‐2011, we did not observe a significant difference in ICU admission or 30‐day readmission rates in the postreform period comparing patients discharged from the resident and hospitalist services and all patients in the prereform period.

Our neutral findings differ from some other single‐institution evaluations of reduced resident work hours, several of which have shown improved quality of life, education, and patient safety indicators.[18, 21, 26, 27, 28] It is unclear why improvements in patient safety were not identified in the current study. The 2011 reforms were more broad‐based than some of the preliminary studies of reduced work hours, and therefore additional variables may be at play. For instance, challenges related to decreased work hours, including the increased number of handoffs in care and work compression, may require specific interventions to produce sustained improvements in patient safety.[3, 14, 29, 30]

Improving patient safety requires more than changing resident work hours. Blum et al. recommended enhanced funding to increase supervision, decrease resident caseload, and incentivize achievement of quality indicators to achieve the goal of improved patient safety within work‐hour reform.[31] Schumacher et al. proposed a focus on supervision, professionalism, safe transitions of care, and optimizing workloads as a means to improve patient safety and education within the new residency training paradigm.[29]

Limitations of this study include limited follow‐up time after implementation of the work‐hour reforms. It may take more time to optimize systems of care to see benefits in patient safety indicators. This was a single‐institution study of a limited number of outcomes in a single department, which limits generalizability and may reflect local experience rather than broader trends. The call schedule on the resident service in this study differs from programs that have adopted night float schedules. [27] This may have had an effect on patient care outcomes.[32] In an attempt to conduct a timely study of inpatient safety indicators following the 2011 changes, our study was not powered to detect small changes in low‐frequency outcomes such as mortality; longer‐term studies at multiple institutions will be needed to answer these key questions. We limited the prereform period where our test of controls indicated differential prereform trends, which reduced power.

As this was an observational study rather than an experiment, there may have been both measured and unmeasured differences in patient characteristics and comorbidity between the intervention and control group. For example, CMI was lower on the hospitalist service than the resident services. Demographics varied somewhat between services; male and African American patients were more likely to be discharged from resident services than hospitalist services for unknown reasons. Although we adjusted for demographics and CMI in our model, there may be residual confounding. Limitations in data collection did not allow us to separate patients initially admitted to the ICU from patients transferred to the ICU from the inpatient floors. We attempted to overcome this limitation through use of a difference‐in‐differences model to account for secular trends, but factors other than residency work hours may have impacted the resident and hospitalist services differentially. For example, hospital quality‐improvement programs or provider‐level factors may have differentially impacted the resident versus hospitalist services during the study period.

Work‐hour limitations for residents were established to improve residency education and patient safety. As noted by the Institute of Medicine, improving patient safety will require significant investment by program directors, hospitals, and the public to keep resident caseloads manageable, ensure adequate supervision of first‐year residents, train residents on safe handoffs in care, and conduct ongoing evaluations of patient safety and any unintended consequences of the regulations.[33] In the first year after implementation of the 2011 work‐hour reforms, we found no change in ICU admission, inpatient mortality, 30‐day readmission rates, length of stay, or MHACs compared with patients treated by hospitalists. Studies of the long‐term impact of residency work‐hour reform are necessary to determine whether changes in work hours have been associated with improvement in resident education and patient safety.

Disclosure: Nothing to report.

References
  1. Accreditation Council for Graduate Medical Education. Common program requirements effective: July 1, 2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramResources/Common_Program_Requirements_07012011[1].pdf. Accessed February 10, 2014.
  2. Nasca TJ, Day SH, Amis ES. The new recommendations on duty hours from the ACGME Task Force. N Engl J Med. 2010;363:e3.
  3. Landrigan CP, Barger LK, Cade BE, Ayas NT, Czeisler CA. Interns' compliance with Accreditation Council for Graduate Medical Education work‐hour limits. JAMA. 2006;296(9):1063–1070.
  4. Fletcher KE, Underwood W, Davis SQ, Mangulkar RS, McMahon LF, Saint S. Effects of work hour reduction on residents' lives: a systematic review. JAMA. 2005;294(9):1088–1100.
  5. Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects of the ACGME duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122(2):250–258.
  6. Volpp KG, Small DS, Romano PS. Teaching hospital five‐year mortality trends in the wake of duty hour reforms. J Gen Intern Med. 2013;28(8):1048–1055.
  7. Philibert I, Nasca T, Brigham T, Shapiro J. Duty hour limits and patient care and resident outcomes: can high‐quality studies offer insight into complex relationships? Ann Rev Med. 2013;64:467–483.
  8. Fletcher KE, Reed DA, Arora VM. Patient safety, resident education and resident well‐being following implementation of the 2003 ACGME duty hour rules. J Gen Intern Med. 2011;26(8):907–919.
  9. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298(9):975–983.
  10. Rosen AK, Loveland SA, Romano PS, et al. Effects of resident duty hour reform on surgical and procedural patient safety indicators among hospitalized Veterans Health Administration and Medicare patients. Med Care. 2009;47(7):723–731.
  11. Schuh LA, Khan MA, Harle H, et al. Pilot trial of IOM duty hour recommendations in neurology residency programs. Neurology. 2011;77(9):883–887.
  12. McCoy CP, Halvorsen AJ, Loftus CG, et al. Effect of 16‐hour duty periods of patient care and resident education. Mayo Clin Proc. 2011;86:192–196.
  13. Sen S, Kranzler HR, Didwania AK, et al. Effects of the 2011 duty hour reforms on interns and their patients: a prospective longitudinal cohort study. JAMA Intern Med. 2013;173(8):657–662.
  14. Desai SV, Feldman L, Brown L, et al. Effect of the 2011 vs 2003 duty hour regulation—compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff. JAMA Intern Med. 2013;173(8):649–655.
  15. Drolet BC, Christopher DA, Fischer SA. Residents' response to duty‐hour regulations—a follow‐up national survey. N Engl J Med. 2012;366:e35.
  16. Drolet BS, Sangisetty S, Tracy TF, Cioffi WG. Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations. JAMA Surg. 2013;148(5):427–433.
  17. Drolet BC, Khokhar MT, Fischer SA. The 2011 duty hour requirements—a survey of residency program directors. N Engl J Med. 2013;368:694–697.
  18. Theobald CN, Stover DG, Choma NN, et al. The effect of reducing maximum shift lengths to 16 hours on internal medicine interns' educational opportunities. Acad Med. 2013;88(4):512–518.
  19. Nuckols TK, Escarce JJ. Residency work‐hours reform. A cost analysis including preventable adverse events. J Gen Intern Med. 2005;20(10):873–878.
  20. Nuckols TK, Bhattacharya J, Wolman DM, Ulmer C, Escarce JJ. Cost implications of reduced work hours and workloads for resident physicians. N Engl J Med. 2009;360:2202–2215.
  21. Horwitz LI, Kosiborod M, Lin Z, Krumholz HM. Changes in outcomes for internal medicine inpatients after work‐hour regulations. Ann Intern Med. 2007;147:97–103.
  22. .Maryland Health Services Cost Review Commission. Complications: Maryland Hospital Acquired Conditions. Available at: http://www.hscrc.state.md.us/init_qi_MHAC.cfm. Accessed May 23, 2013.
  23. Averill R, Goldfield N, Hughes J, et al. What are APR‐DRGs? An introduction to severity of illness and risk of mortality adjustment methodology. 3M Health Information Systems. Available at: http://solutions.3m.com/3MContentRetrievalAPI/BlobServlet?locale=it_IT44(4):1049–1060.
  24. Ross JS, Wang R, Long JB, Gross CP, Ma X. Impact of the 2008 US Preventive Services Task Force Recommendation to discontinue prostate cancer screening among male Medicare beneficiaries. Arch Intern Med. 2012;172(20):1601–1603.
  25. Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns' work hour on serious medical errors in intensive care units. N Engl J Med. 2004;351(18):1838–1848.
  26. Levine AC, Adusumilli J, Landrigan CP. Effects of reducing or eliminating resident work shifts over 16 hours: a systematic review. Sleep. 2010;33(8):1043–1053.
  27. Bhavsar J, Montgomery D, Li J, et al. Impact of duty hours restrictions on quality of care and clinical outcomes. Am J Med. 2007;120(11):968–974.
  28. Schumacher DJ, Slovein SR, Riebschleger MP, Englander R, Hicks P, Carraccio C. Beyond counting hours: the importance of supervision, professionalism, transitions in care, and workload in residency training. Acad Med. 2012;87(7):883–888.
  29. Tessing S, Amendt A, Jennings J, Thomson J, Auger KA, Gonzalez del Rey JA. One possible future for resident hours: interns' perspective on a one‐month trial of the Institute of Medicine recommended duty hour limits. J Grad Med Educ. 2009;1(2):185–187.
  30. Blum AB, Shea S, Czeisler CA, Landrigan CP, Leape L. Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety. Nature Sci Sleep. 2001;3:47–85.
  31. Bricker DA, Markert RJ. Night float teaching and learning: perceptions of residents and faculty. J Grad Med Educ. 2010;2(2):236–241.
  32. Institute of Medicine. Resident duty hours: enhancing sleep, supervision, and safety. Report brief. Washington, DC: National Academies; 2008. Available at: http://www.iom.edu/∼/media/Files/Report Files/2008/Resident‐Duty‐Hours/residency hours revised for web.pdf. Accessed May 23, 2013.
References
  1. Accreditation Council for Graduate Medical Education. Common program requirements effective: July 1, 2011. Available at: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramResources/Common_Program_Requirements_07012011[1].pdf. Accessed February 10, 2014.
  2. Nasca TJ, Day SH, Amis ES. The new recommendations on duty hours from the ACGME Task Force. N Engl J Med. 2010;363:e3.
  3. Landrigan CP, Barger LK, Cade BE, Ayas NT, Czeisler CA. Interns' compliance with Accreditation Council for Graduate Medical Education work‐hour limits. JAMA. 2006;296(9):1063–1070.
  4. Fletcher KE, Underwood W, Davis SQ, Mangulkar RS, McMahon LF, Saint S. Effects of work hour reduction on residents' lives: a systematic review. JAMA. 2005;294(9):1088–1100.
  5. Landrigan CP, Fahrenkopf AM, Lewin D, et al. Effects of the ACGME duty hour limits on sleep, work hours, and safety. Pediatrics. 2008;122(2):250–258.
  6. Volpp KG, Small DS, Romano PS. Teaching hospital five‐year mortality trends in the wake of duty hour reforms. J Gen Intern Med. 2013;28(8):1048–1055.
  7. Philibert I, Nasca T, Brigham T, Shapiro J. Duty hour limits and patient care and resident outcomes: can high‐quality studies offer insight into complex relationships? Ann Rev Med. 2013;64:467–483.
  8. Fletcher KE, Reed DA, Arora VM. Patient safety, resident education and resident well‐being following implementation of the 2003 ACGME duty hour rules. J Gen Intern Med. 2011;26(8):907–919.
  9. Volpp KG, Rosen AK, Rosenbaum PR, et al. Mortality among hospitalized Medicare beneficiaries in the first 2 years following ACGME resident duty hour reform. JAMA. 2007;298(9):975–983.
  10. Rosen AK, Loveland SA, Romano PS, et al. Effects of resident duty hour reform on surgical and procedural patient safety indicators among hospitalized Veterans Health Administration and Medicare patients. Med Care. 2009;47(7):723–731.
  11. Schuh LA, Khan MA, Harle H, et al. Pilot trial of IOM duty hour recommendations in neurology residency programs. Neurology. 2011;77(9):883–887.
  12. McCoy CP, Halvorsen AJ, Loftus CG, et al. Effect of 16‐hour duty periods of patient care and resident education. Mayo Clin Proc. 2011;86:192–196.
  13. Sen S, Kranzler HR, Didwania AK, et al. Effects of the 2011 duty hour reforms on interns and their patients: a prospective longitudinal cohort study. JAMA Intern Med. 2013;173(8):657–662.
  14. Desai SV, Feldman L, Brown L, et al. Effect of the 2011 vs 2003 duty hour regulation—compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff. JAMA Intern Med. 2013;173(8):649–655.
  15. Drolet BC, Christopher DA, Fischer SA. Residents' response to duty‐hour regulations—a follow‐up national survey. N Engl J Med. 2012;366:e35.
  16. Drolet BS, Sangisetty S, Tracy TF, Cioffi WG. Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations. JAMA Surg. 2013;148(5):427–433.
  17. Drolet BC, Khokhar MT, Fischer SA. The 2011 duty hour requirements—a survey of residency program directors. N Engl J Med. 2013;368:694–697.
  18. Theobald CN, Stover DG, Choma NN, et al. The effect of reducing maximum shift lengths to 16 hours on internal medicine interns' educational opportunities. Acad Med. 2013;88(4):512–518.
  19. Nuckols TK, Escarce JJ. Residency work‐hours reform. A cost analysis including preventable adverse events. J Gen Intern Med. 2005;20(10):873–878.
  20. Nuckols TK, Bhattacharya J, Wolman DM, Ulmer C, Escarce JJ. Cost implications of reduced work hours and workloads for resident physicians. N Engl J Med. 2009;360:2202–2215.
  21. Horwitz LI, Kosiborod M, Lin Z, Krumholz HM. Changes in outcomes for internal medicine inpatients after work‐hour regulations. Ann Intern Med. 2007;147:97–103.
  22. .Maryland Health Services Cost Review Commission. Complications: Maryland Hospital Acquired Conditions. Available at: http://www.hscrc.state.md.us/init_qi_MHAC.cfm. Accessed May 23, 2013.
  23. Averill R, Goldfield N, Hughes J, et al. What are APR‐DRGs? An introduction to severity of illness and risk of mortality adjustment methodology. 3M Health Information Systems. Available at: http://solutions.3m.com/3MContentRetrievalAPI/BlobServlet?locale=it_IT44(4):1049–1060.
  24. Ross JS, Wang R, Long JB, Gross CP, Ma X. Impact of the 2008 US Preventive Services Task Force Recommendation to discontinue prostate cancer screening among male Medicare beneficiaries. Arch Intern Med. 2012;172(20):1601–1603.
  25. Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns' work hour on serious medical errors in intensive care units. N Engl J Med. 2004;351(18):1838–1848.
  26. Levine AC, Adusumilli J, Landrigan CP. Effects of reducing or eliminating resident work shifts over 16 hours: a systematic review. Sleep. 2010;33(8):1043–1053.
  27. Bhavsar J, Montgomery D, Li J, et al. Impact of duty hours restrictions on quality of care and clinical outcomes. Am J Med. 2007;120(11):968–974.
  28. Schumacher DJ, Slovein SR, Riebschleger MP, Englander R, Hicks P, Carraccio C. Beyond counting hours: the importance of supervision, professionalism, transitions in care, and workload in residency training. Acad Med. 2012;87(7):883–888.
  29. Tessing S, Amendt A, Jennings J, Thomson J, Auger KA, Gonzalez del Rey JA. One possible future for resident hours: interns' perspective on a one‐month trial of the Institute of Medicine recommended duty hour limits. J Grad Med Educ. 2009;1(2):185–187.
  30. Blum AB, Shea S, Czeisler CA, Landrigan CP, Leape L. Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety. Nature Sci Sleep. 2001;3:47–85.
  31. Bricker DA, Markert RJ. Night float teaching and learning: perceptions of residents and faculty. J Grad Med Educ. 2010;2(2):236–241.
  32. Institute of Medicine. Resident duty hours: enhancing sleep, supervision, and safety. Report brief. Washington, DC: National Academies; 2008. Available at: http://www.iom.edu/∼/media/Files/Report Files/2008/Resident‐Duty‐Hours/residency hours revised for web.pdf. Accessed May 23, 2013.
Issue
Journal of Hospital Medicine - 9(6)
Issue
Journal of Hospital Medicine - 9(6)
Page Number
347-352
Page Number
347-352
Publications
Journal of Hospital Medicine
Publications
Journal of Hospital Medicine
Article Type
Article
Display Headline
Inpatient safety outcomes following the 2011 residency work‐hour reform
Display Headline
Inpatient safety outcomes following the 2011 residency work‐hour reform
Sections
Original Research
Article Source

© 2014 Society of Hospital Medicine

Disallow All Ads
Corresponding Author
Lauren Block, MD, MPH
Correspondence Location
Address for correspondence and reprint requests: Lauren Block, MD, Hofstra North Shore‐LIJ School of Medicine, 2001 Marcus Ave, Suite S160, Lake Success NY 11042; Telephone: 516‐519‐5600; Fax: 516‐519‐5601; E‐mail: lblock2@nshs.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files
Subscribe to Jenna Swann, MS