Samuel R. Kaufman, MA

Deep venous thrombosis (DVT) and pulmonary embolism (PE), collectively referred to as venous thromboembolism (VTE), are common events in hospitalized patients and result in significant morbidity and mortality. Often silent and frequently unexpected, VTE is preventable. Accordingly, the American College of Chest Physicians recommends that pharmacologic prophylaxis be given to acutely ill medical patients admitted to the hospital with congestive heart failure or severe respiratory disease, or to patients who are confined to bed who have additional risk factors, such as cancer or previous VTE.1 Three recent meta‐analyses24 demonstrated significant reductions in VTE in general medicine patients with pharmacologic prophylaxis. Recently the National Quality Forum advocated that hospitals evaluate each patient upon admission and regularly thereafter, for the risk of developing DVT/VTE and utilize clinically appropriate methods to prevent DVT/VTE.5

Despite recommendations for prophylaxis, multiple studies demonstrate utilization in <50% of at‐risk general medical patients.68 Physicians' lack of awareness may partially explain this underutilization, but other likely factors include physicians' questions about the clinical importance of the outcome (eg, some studies have shown reductions primarily in asymptomatic distal DVT), doubt regarding the best form of prophylaxis (ie, unfractionated heparin [UFH] vs. low molecular weight heparin [LMWH]), uncertainty regarding optimal dosing regimens, and comparable uncertainty regarding which patients have sufficiently high risk for VTE to outweigh the risks of anticoagulation.

We undertook the current meta‐analysis to address questions about thromboembolism prevention in general medicine patients. Does pharmacologic prophylaxis prevent clinically relevant events? Is LMWH or UFH preferable in terms of either efficacy or safety?

MATERIALS AND METHODS

RESULTS

Study Identification and Selection

The computerized literature search resulted in 5284 articles. Three additional citations were found by review of bibliographies. No additional trials were identified from reviews of abstracts from national meetings. Representatives from the 3 pharmaceutical companies reported no knowledge of additional published or unpublished data. Of the 5287 studies identified by the search, 14 studies met all eligibility criteria (Figure 1).

Figure 1

UFH or LMWH vs. Control

DVT

Across 7 trials comparing either UFH or LMWH to control, heparin products significantly decreased the risk of all DVT (RR = 0.55; 95% CI: 0.36‐0.83) (Figure 2A). When stratified by methodological quality, 5 trials16, 2225 with Jadad scores of 3 or higher showed an RR reduction of 0.53 (95% CI: 0.38‐0.72) in reducing all DVT. All of the higher‐quality trials compared LMWH to placebo. Across 4 trials that reported data for symptomatic DVT there was a nonsignificant reduction in RR compared with placebo (RR = 0.73; 95% CI: 0.45‐1.16) (Figure 2B). Only 2 trials24, 25 (both LMWH trials) reported results for proximal DVT and demonstrated significant benefit of prophylaxis with a pooled RR of 0.46 (95% CI: 0.31‐0.69) (Figure 2C).

Figure 2

PE

Across 7 trials comparing either UFH or LMWH to control, heparin products significantly decreased the risk of PE (RR = 0.70; 95% CI: 0.53‐0.93) (Figure 3A). The 5 trials16, 2225 with Jadad scores of 3 or greater showed a similar relative risk reduction, but the result was no longer statistically significant (RR = 0.56; 95% CI: 0.31‐1.02). Two of the trials16, 21 relied solely on the results of autopsy to diagnose PE, which may have given rise to chance differences in detection due to generally low autopsy rates. Eliminating these 2 studies from the analysis resulted in loss of statistical significance for the reduction in risk for PE (RR = 0.48; 95% CI: 0.20‐1.15).

Figure 3

Death

Seven trials16, 2025 comparing either UFH or LMWH to control examined the impact of pharmacologic prophylaxis on death and found no significant difference between treated and untreated patients across all trials (RR = 0.92; 95% CI: 0.82‐1.03) and those limited to studies with Jadad scores of 3 or higher (RR = 0.97; 95% CI: 0.80‐1.17).

Complications

We evaluated adverse events of heparin products used for prophylaxis and whether there were differences between UFH and LMWH. Reporting of complications was not uniform from study to study, making pooling more difficult. However, we were able to abstract data on any bleeding, major bleeding, and thrombocytopenia from several studies. In 5 studies15, 16, 2325 of either UFH or LMWH vs. control, a significantly increased risk of any bleeding (RR = 1.54; 95% CI: 1.15‐2.06) (Figure 4A) was found. When only major bleeding was evaluated, no statistically significant difference was noted (RR = 1.20; 95% CI: 0.55‐2.58) (Figure 4B). In 4 trials16, 22, 24, 25 the occurrence of thrombocytopenia was not significantly different when comparing UFH or LMWH to control (RR = 0.92; 95% CI: 0.46‐1.86).

Figure 4

When LMWH was compared to UFH in 4 trials,14, 17, 18, 27 a nonsignificant trend toward a decrease in any bleeding was found in the LMWH group (RR = 0.72; 95% CI: 0.44‐1.16) (Figure 5A). A similar trend was seen favoring LMWH in rates of major bleeding (RR = 0.57; 95% CI: 0.25‐1.32) (Figure 5B). Neither trend was statistically significant. Three trials comparing LMWH to UFH reported on thrombocytopenia17, 18, 27 with no significant difference noted (RR = 0.52; 95% CI: 0.06‐4.18).

Figure 5

DISCUSSION

When compared to control, LMWH or UFH decreased the risk of all DVT by 45% (RR = 0.55; 95% CI: 0.36‐0.83) and proximal DVT by 54% (RR = 0.46; 95% CI: 0.31‐0.69). PE was also decreased by 30% (RR = 0.70; 95% CI: 0.53‐0.93). Of note, when prophylaxis was compared with placebo all of the high‐quality studies showing a benefit were done using LMWH. The benefits of prophylaxis occurred at the cost of a 54% increased overall risk of bleeding (RR = 1.54; 95% CI 1.15‐2.06). However, the risk of major bleeding was not significantly increased. We did not find a mortality benefit to pharmacologic thromboembolism prophylaxis.

When comparing UFH to LMWH, we noted no difference in all DVT, symptomatic DVT, proximal DVT, PE, or death. While there was a trend toward less bleeding with LMWH, this was not statistically significant.

Taken in aggregate, our findings are in agreement with previous published meta‐analyses reporting net benefit for thromboembolism prophylaxis in medical patients.24, 22, 28, 29 Our meta‐analysis has several methodological strengths over the prior studies, including a comprehensive search of both the published and unpublished literature and assessment of the relationship between methodological quality of included trials and reported benefit. In contrast to previous reviews, our analysis highlights several limitations of the current evidence.

First, many of the studies are older, with predicted lengths of stay of greater than 1 week. The 8‐13‐day range of treatment duration we found in this study is longer than the average length of stay in today's hospitals. Second, there is variability in the diagnostic tests used to diagnose DVT, as well as variation in the definition of DVT among studies. Studies using fibrinogen uptake scanning reported rates of DVT as high as 26%15 while studies using venography reported DVT rates of almost 15% in the placebo arm.24 These rates are higher than most physicians' routine practice. One reason for this discrepancy is most studies did not distinguish below‐the‐knee DVT from more clinically relevant above‐the‐knee DVT. Systematic reviews of medical and surgical patients have found rates of proximal propagation from 0% to 29% in untreated patients.30, 31 Though controversial, below‐the‐knee DVT is believed less morbid than proximal DVT or symptomatic DVT. We addressed this by focusing specifically on clinically relevant endpoints of proximal and symptomatic DVT. When we restricted our analysis to proximal DVT we found a 54% RR reduction in 2 pooled trials of LMWH compared to placebo. In pooled analyses symptomatic DVT was not affected by prophylaxis. When compared head‐to‐head there were no differences between LMWH and UFH for proximal DVT or symptomatic DVT.

When considering PE, the utilization of autopsy as the sole diagnostic method in 2 large trials16, 21 is particularly problematic. In the trial by Garlund,21 the mortality rate was 5.4%, with an autopsy rate of 60.1%. Similarly, in the trial by Mahe et al.,16 the mortality rate was 10%, with an autopsy rate of 49%. Given the low absolute number of deaths and substantial proportion of decedents without autopsy, the potential for chance to produce an imbalance in detection of PE is high in these studies. When we excluded these 2 trials, we found that PE was no longer reduced to a statistically significant degree by prophylaxis. Loss of significance for PE in 2 sensitivity analyses (when excluding studies of lower quality, or using autopsy as a sole diagnostic study) is problematic and calls into question the true benefit of prophylaxis for prevention of PE.

Another limitation of the current literature centers on the variability of dosing used. We pooled trials of UFH whether given BID or TID. Given the small number of trials we did not do sensitivity analyses by dosage. A recent meta‐analysis3 found both doses are efficacious, while a recent review article32 suggested superiority of TID dosing. We believe the available literature does not clearly address this issue. Regarding comparisons of LMWH to UFH, dosing variability was also noted. The trial by Bergmann and Neuhart27 used enoxaparin 20 mg per day and found similar efficacy to UFH BID, while the Samama et al.24 trial found enoxaparin 20 mg per day no more efficacious than placebo. While the literature does not clearly define a best dose, we believe enoxaparin doses lower than 40 mg daily do not reflect the standard of care.

An additional limitation of the literature is publication bias. We assessed the possibility of publication bias by a variety of means. We did find statistical evidence of publication bias for the outcome of PE when prophylaxis was compared to control. Importantly, two meta‐analyses2, 4 on thromboembolism prophylaxis for general medicine patients suggested publication bias is present and our finding supports this conclusion. While no test for publication bias is foolproof, the best protection against publication bias, which we pursued in our study, consists of a thorough search for unpublished studies, including a search of conference proceedings, contact with experts in the field, and manufacturers of LMWH.

A final limitation of the current literature centers on risk assessment. All of the trials in this meta‐analysis included patients with an elevated level of risk. Unfortunately, risk was not clearly defined in many studies, and there was no minimum level of risk between trials. While immobility, age, and length of stay were reported for most studies, other risk factors such as personal history of thromboembolism and malignancy were not uniformly reported. Based on our analysis we are not confident our results can be extrapolated to all general medicine patients.

In conclusion, we found good evidence that pharmacologic prophylaxis significantly decreases the risk of all DVT and proximal DVT in at‐risk general medical patients. However, only LMWH was shown to prevent proximal DVT. We found inconclusive evidence that prophylaxis prevents PE. When compared directly we did not find clear superiority between UFH and LMWH, though several limitations of the current literature hamper decision‐making. Given the lower cost, it may seem justified to use UFH. However, there are other practical issues, such as the fact that LMWH is given once daily, and so potentially preferred by patients and more efficient for nurses. All of these results pertain to patients with elevated risk. While we did not find significant safety concerns with prophylaxis we do not know if these results can be extrapolated to lower‐risk patients. We believe that recommending widespread prophylaxis of all general medicine patients requires additional evidence about appropriate patient selection.

Deep venous thrombosis (DVT) and pulmonary embolism (PE), collectively referred to as venous thromboembolism (VTE), are common events in hospitalized patients and result in significant morbidity and mortality. Often silent and frequently unexpected, VTE is preventable. Accordingly, the American College of Chest Physicians recommends that pharmacologic prophylaxis be given to acutely ill medical patients admitted to the hospital with congestive heart failure or severe respiratory disease, or to patients who are confined to bed who have additional risk factors, such as cancer or previous VTE.1 Three recent meta‐analyses24 demonstrated significant reductions in VTE in general medicine patients with pharmacologic prophylaxis. Recently the National Quality Forum advocated that hospitals evaluate each patient upon admission and regularly thereafter, for the risk of developing DVT/VTE and utilize clinically appropriate methods to prevent DVT/VTE.5

Despite recommendations for prophylaxis, multiple studies demonstrate utilization in <50% of at‐risk general medical patients.68 Physicians' lack of awareness may partially explain this underutilization, but other likely factors include physicians' questions about the clinical importance of the outcome (eg, some studies have shown reductions primarily in asymptomatic distal DVT), doubt regarding the best form of prophylaxis (ie, unfractionated heparin [UFH] vs. low molecular weight heparin [LMWH]), uncertainty regarding optimal dosing regimens, and comparable uncertainty regarding which patients have sufficiently high risk for VTE to outweigh the risks of anticoagulation.

We undertook the current meta‐analysis to address questions about thromboembolism prevention in general medicine patients. Does pharmacologic prophylaxis prevent clinically relevant events? Is LMWH or UFH preferable in terms of either efficacy or safety?

MATERIALS AND METHODS

RESULTS

Study Identification and Selection

The computerized literature search resulted in 5284 articles. Three additional citations were found by review of bibliographies. No additional trials were identified from reviews of abstracts from national meetings. Representatives from the 3 pharmaceutical companies reported no knowledge of additional published or unpublished data. Of the 5287 studies identified by the search, 14 studies met all eligibility criteria (Figure 1).

Figure 1

UFH or LMWH vs. Control

DVT

Across 7 trials comparing either UFH or LMWH to control, heparin products significantly decreased the risk of all DVT (RR = 0.55; 95% CI: 0.36‐0.83) (Figure 2A). When stratified by methodological quality, 5 trials16, 2225 with Jadad scores of 3 or higher showed an RR reduction of 0.53 (95% CI: 0.38‐0.72) in reducing all DVT. All of the higher‐quality trials compared LMWH to placebo. Across 4 trials that reported data for symptomatic DVT there was a nonsignificant reduction in RR compared with placebo (RR = 0.73; 95% CI: 0.45‐1.16) (Figure 2B). Only 2 trials24, 25 (both LMWH trials) reported results for proximal DVT and demonstrated significant benefit of prophylaxis with a pooled RR of 0.46 (95% CI: 0.31‐0.69) (Figure 2C).

Figure 2

PE

Across 7 trials comparing either UFH or LMWH to control, heparin products significantly decreased the risk of PE (RR = 0.70; 95% CI: 0.53‐0.93) (Figure 3A). The 5 trials16, 2225 with Jadad scores of 3 or greater showed a similar relative risk reduction, but the result was no longer statistically significant (RR = 0.56; 95% CI: 0.31‐1.02). Two of the trials16, 21 relied solely on the results of autopsy to diagnose PE, which may have given rise to chance differences in detection due to generally low autopsy rates. Eliminating these 2 studies from the analysis resulted in loss of statistical significance for the reduction in risk for PE (RR = 0.48; 95% CI: 0.20‐1.15).

Figure 3

Death

Seven trials16, 2025 comparing either UFH or LMWH to control examined the impact of pharmacologic prophylaxis on death and found no significant difference between treated and untreated patients across all trials (RR = 0.92; 95% CI: 0.82‐1.03) and those limited to studies with Jadad scores of 3 or higher (RR = 0.97; 95% CI: 0.80‐1.17).

Complications

We evaluated adverse events of heparin products used for prophylaxis and whether there were differences between UFH and LMWH. Reporting of complications was not uniform from study to study, making pooling more difficult. However, we were able to abstract data on any bleeding, major bleeding, and thrombocytopenia from several studies. In 5 studies15, 16, 2325 of either UFH or LMWH vs. control, a significantly increased risk of any bleeding (RR = 1.54; 95% CI: 1.15‐2.06) (Figure 4A) was found. When only major bleeding was evaluated, no statistically significant difference was noted (RR = 1.20; 95% CI: 0.55‐2.58) (Figure 4B). In 4 trials16, 22, 24, 25 the occurrence of thrombocytopenia was not significantly different when comparing UFH or LMWH to control (RR = 0.92; 95% CI: 0.46‐1.86).

Figure 4

When LMWH was compared to UFH in 4 trials,14, 17, 18, 27 a nonsignificant trend toward a decrease in any bleeding was found in the LMWH group (RR = 0.72; 95% CI: 0.44‐1.16) (Figure 5A). A similar trend was seen favoring LMWH in rates of major bleeding (RR = 0.57; 95% CI: 0.25‐1.32) (Figure 5B). Neither trend was statistically significant. Three trials comparing LMWH to UFH reported on thrombocytopenia17, 18, 27 with no significant difference noted (RR = 0.52; 95% CI: 0.06‐4.18).

Figure 5

DISCUSSION

When compared to control, LMWH or UFH decreased the risk of all DVT by 45% (RR = 0.55; 95% CI: 0.36‐0.83) and proximal DVT by 54% (RR = 0.46; 95% CI: 0.31‐0.69). PE was also decreased by 30% (RR = 0.70; 95% CI: 0.53‐0.93). Of note, when prophylaxis was compared with placebo all of the high‐quality studies showing a benefit were done using LMWH. The benefits of prophylaxis occurred at the cost of a 54% increased overall risk of bleeding (RR = 1.54; 95% CI 1.15‐2.06). However, the risk of major bleeding was not significantly increased. We did not find a mortality benefit to pharmacologic thromboembolism prophylaxis.

When comparing UFH to LMWH, we noted no difference in all DVT, symptomatic DVT, proximal DVT, PE, or death. While there was a trend toward less bleeding with LMWH, this was not statistically significant.

Taken in aggregate, our findings are in agreement with previous published meta‐analyses reporting net benefit for thromboembolism prophylaxis in medical patients.24, 22, 28, 29 Our meta‐analysis has several methodological strengths over the prior studies, including a comprehensive search of both the published and unpublished literature and assessment of the relationship between methodological quality of included trials and reported benefit. In contrast to previous reviews, our analysis highlights several limitations of the current evidence.

First, many of the studies are older, with predicted lengths of stay of greater than 1 week. The 8‐13‐day range of treatment duration we found in this study is longer than the average length of stay in today's hospitals. Second, there is variability in the diagnostic tests used to diagnose DVT, as well as variation in the definition of DVT among studies. Studies using fibrinogen uptake scanning reported rates of DVT as high as 26%15 while studies using venography reported DVT rates of almost 15% in the placebo arm.24 These rates are higher than most physicians' routine practice. One reason for this discrepancy is most studies did not distinguish below‐the‐knee DVT from more clinically relevant above‐the‐knee DVT. Systematic reviews of medical and surgical patients have found rates of proximal propagation from 0% to 29% in untreated patients.30, 31 Though controversial, below‐the‐knee DVT is believed less morbid than proximal DVT or symptomatic DVT. We addressed this by focusing specifically on clinically relevant endpoints of proximal and symptomatic DVT. When we restricted our analysis to proximal DVT we found a 54% RR reduction in 2 pooled trials of LMWH compared to placebo. In pooled analyses symptomatic DVT was not affected by prophylaxis. When compared head‐to‐head there were no differences between LMWH and UFH for proximal DVT or symptomatic DVT.

When considering PE, the utilization of autopsy as the sole diagnostic method in 2 large trials16, 21 is particularly problematic. In the trial by Garlund,21 the mortality rate was 5.4%, with an autopsy rate of 60.1%. Similarly, in the trial by Mahe et al.,16 the mortality rate was 10%, with an autopsy rate of 49%. Given the low absolute number of deaths and substantial proportion of decedents without autopsy, the potential for chance to produce an imbalance in detection of PE is high in these studies. When we excluded these 2 trials, we found that PE was no longer reduced to a statistically significant degree by prophylaxis. Loss of significance for PE in 2 sensitivity analyses (when excluding studies of lower quality, or using autopsy as a sole diagnostic study) is problematic and calls into question the true benefit of prophylaxis for prevention of PE.

Another limitation of the current literature centers on the variability of dosing used. We pooled trials of UFH whether given BID or TID. Given the small number of trials we did not do sensitivity analyses by dosage. A recent meta‐analysis3 found both doses are efficacious, while a recent review article32 suggested superiority of TID dosing. We believe the available literature does not clearly address this issue. Regarding comparisons of LMWH to UFH, dosing variability was also noted. The trial by Bergmann and Neuhart27 used enoxaparin 20 mg per day and found similar efficacy to UFH BID, while the Samama et al.24 trial found enoxaparin 20 mg per day no more efficacious than placebo. While the literature does not clearly define a best dose, we believe enoxaparin doses lower than 40 mg daily do not reflect the standard of care.

An additional limitation of the literature is publication bias. We assessed the possibility of publication bias by a variety of means. We did find statistical evidence of publication bias for the outcome of PE when prophylaxis was compared to control. Importantly, two meta‐analyses2, 4 on thromboembolism prophylaxis for general medicine patients suggested publication bias is present and our finding supports this conclusion. While no test for publication bias is foolproof, the best protection against publication bias, which we pursued in our study, consists of a thorough search for unpublished studies, including a search of conference proceedings, contact with experts in the field, and manufacturers of LMWH.

A final limitation of the current literature centers on risk assessment. All of the trials in this meta‐analysis included patients with an elevated level of risk. Unfortunately, risk was not clearly defined in many studies, and there was no minimum level of risk between trials. While immobility, age, and length of stay were reported for most studies, other risk factors such as personal history of thromboembolism and malignancy were not uniformly reported. Based on our analysis we are not confident our results can be extrapolated to all general medicine patients.

In conclusion, we found good evidence that pharmacologic prophylaxis significantly decreases the risk of all DVT and proximal DVT in at‐risk general medical patients. However, only LMWH was shown to prevent proximal DVT. We found inconclusive evidence that prophylaxis prevents PE. When compared directly we did not find clear superiority between UFH and LMWH, though several limitations of the current literature hamper decision‐making. Given the lower cost, it may seem justified to use UFH. However, there are other practical issues, such as the fact that LMWH is given once daily, and so potentially preferred by patients and more efficient for nurses. All of these results pertain to patients with elevated risk. While we did not find significant safety concerns with prophylaxis we do not know if these results can be extrapolated to lower‐risk patients. We believe that recommending widespread prophylaxis of all general medicine patients requires additional evidence about appropriate patient selection.

Deep venous thrombosis (DVT) and pulmonary embolism (PE), collectively referred to as venous thromboembolism (VTE), are common events in hospitalized patients and result in significant morbidity and mortality. Often silent and frequently unexpected, VTE is preventable. Accordingly, the American College of Chest Physicians recommends that pharmacologic prophylaxis be given to acutely ill medical patients admitted to the hospital with congestive heart failure or severe respiratory disease, or to patients who are confined to bed who have additional risk factors, such as cancer or previous VTE.1 Three recent meta‐analyses24 demonstrated significant reductions in VTE in general medicine patients with pharmacologic prophylaxis. Recently the National Quality Forum advocated that hospitals evaluate each patient upon admission and regularly thereafter, for the risk of developing DVT/VTE and utilize clinically appropriate methods to prevent DVT/VTE.5

Despite recommendations for prophylaxis, multiple studies demonstrate utilization in <50% of at‐risk general medical patients.68 Physicians' lack of awareness may partially explain this underutilization, but other likely factors include physicians' questions about the clinical importance of the outcome (eg, some studies have shown reductions primarily in asymptomatic distal DVT), doubt regarding the best form of prophylaxis (ie, unfractionated heparin [UFH] vs. low molecular weight heparin [LMWH]), uncertainty regarding optimal dosing regimens, and comparable uncertainty regarding which patients have sufficiently high risk for VTE to outweigh the risks of anticoagulation.

We undertook the current meta‐analysis to address questions about thromboembolism prevention in general medicine patients. Does pharmacologic prophylaxis prevent clinically relevant events? Is LMWH or UFH preferable in terms of either efficacy or safety?

MATERIALS AND METHODS

RESULTS

Study Identification and Selection

The computerized literature search resulted in 5284 articles. Three additional citations were found by review of bibliographies. No additional trials were identified from reviews of abstracts from national meetings. Representatives from the 3 pharmaceutical companies reported no knowledge of additional published or unpublished data. Of the 5287 studies identified by the search, 14 studies met all eligibility criteria (Figure 1).

Figure 1

UFH or LMWH vs. Control

DVT

Across 7 trials comparing either UFH or LMWH to control, heparin products significantly decreased the risk of all DVT (RR = 0.55; 95% CI: 0.36‐0.83) (Figure 2A). When stratified by methodological quality, 5 trials16, 2225 with Jadad scores of 3 or higher showed an RR reduction of 0.53 (95% CI: 0.38‐0.72) in reducing all DVT. All of the higher‐quality trials compared LMWH to placebo. Across 4 trials that reported data for symptomatic DVT there was a nonsignificant reduction in RR compared with placebo (RR = 0.73; 95% CI: 0.45‐1.16) (Figure 2B). Only 2 trials24, 25 (both LMWH trials) reported results for proximal DVT and demonstrated significant benefit of prophylaxis with a pooled RR of 0.46 (95% CI: 0.31‐0.69) (Figure 2C).

Figure 2

PE

Across 7 trials comparing either UFH or LMWH to control, heparin products significantly decreased the risk of PE (RR = 0.70; 95% CI: 0.53‐0.93) (Figure 3A). The 5 trials16, 2225 with Jadad scores of 3 or greater showed a similar relative risk reduction, but the result was no longer statistically significant (RR = 0.56; 95% CI: 0.31‐1.02). Two of the trials16, 21 relied solely on the results of autopsy to diagnose PE, which may have given rise to chance differences in detection due to generally low autopsy rates. Eliminating these 2 studies from the analysis resulted in loss of statistical significance for the reduction in risk for PE (RR = 0.48; 95% CI: 0.20‐1.15).

Figure 3

Death

Seven trials16, 2025 comparing either UFH or LMWH to control examined the impact of pharmacologic prophylaxis on death and found no significant difference between treated and untreated patients across all trials (RR = 0.92; 95% CI: 0.82‐1.03) and those limited to studies with Jadad scores of 3 or higher (RR = 0.97; 95% CI: 0.80‐1.17).

Complications

We evaluated adverse events of heparin products used for prophylaxis and whether there were differences between UFH and LMWH. Reporting of complications was not uniform from study to study, making pooling more difficult. However, we were able to abstract data on any bleeding, major bleeding, and thrombocytopenia from several studies. In 5 studies15, 16, 2325 of either UFH or LMWH vs. control, a significantly increased risk of any bleeding (RR = 1.54; 95% CI: 1.15‐2.06) (Figure 4A) was found. When only major bleeding was evaluated, no statistically significant difference was noted (RR = 1.20; 95% CI: 0.55‐2.58) (Figure 4B). In 4 trials16, 22, 24, 25 the occurrence of thrombocytopenia was not significantly different when comparing UFH or LMWH to control (RR = 0.92; 95% CI: 0.46‐1.86).

Figure 4

When LMWH was compared to UFH in 4 trials,14, 17, 18, 27 a nonsignificant trend toward a decrease in any bleeding was found in the LMWH group (RR = 0.72; 95% CI: 0.44‐1.16) (Figure 5A). A similar trend was seen favoring LMWH in rates of major bleeding (RR = 0.57; 95% CI: 0.25‐1.32) (Figure 5B). Neither trend was statistically significant. Three trials comparing LMWH to UFH reported on thrombocytopenia17, 18, 27 with no significant difference noted (RR = 0.52; 95% CI: 0.06‐4.18).

Figure 5

DISCUSSION

When compared to control, LMWH or UFH decreased the risk of all DVT by 45% (RR = 0.55; 95% CI: 0.36‐0.83) and proximal DVT by 54% (RR = 0.46; 95% CI: 0.31‐0.69). PE was also decreased by 30% (RR = 0.70; 95% CI: 0.53‐0.93). Of note, when prophylaxis was compared with placebo all of the high‐quality studies showing a benefit were done using LMWH. The benefits of prophylaxis occurred at the cost of a 54% increased overall risk of bleeding (RR = 1.54; 95% CI 1.15‐2.06). However, the risk of major bleeding was not significantly increased. We did not find a mortality benefit to pharmacologic thromboembolism prophylaxis.

When comparing UFH to LMWH, we noted no difference in all DVT, symptomatic DVT, proximal DVT, PE, or death. While there was a trend toward less bleeding with LMWH, this was not statistically significant.

Taken in aggregate, our findings are in agreement with previous published meta‐analyses reporting net benefit for thromboembolism prophylaxis in medical patients.24, 22, 28, 29 Our meta‐analysis has several methodological strengths over the prior studies, including a comprehensive search of both the published and unpublished literature and assessment of the relationship between methodological quality of included trials and reported benefit. In contrast to previous reviews, our analysis highlights several limitations of the current evidence.

First, many of the studies are older, with predicted lengths of stay of greater than 1 week. The 8‐13‐day range of treatment duration we found in this study is longer than the average length of stay in today's hospitals. Second, there is variability in the diagnostic tests used to diagnose DVT, as well as variation in the definition of DVT among studies. Studies using fibrinogen uptake scanning reported rates of DVT as high as 26%15 while studies using venography reported DVT rates of almost 15% in the placebo arm.24 These rates are higher than most physicians' routine practice. One reason for this discrepancy is most studies did not distinguish below‐the‐knee DVT from more clinically relevant above‐the‐knee DVT. Systematic reviews of medical and surgical patients have found rates of proximal propagation from 0% to 29% in untreated patients.30, 31 Though controversial, below‐the‐knee DVT is believed less morbid than proximal DVT or symptomatic DVT. We addressed this by focusing specifically on clinically relevant endpoints of proximal and symptomatic DVT. When we restricted our analysis to proximal DVT we found a 54% RR reduction in 2 pooled trials of LMWH compared to placebo. In pooled analyses symptomatic DVT was not affected by prophylaxis. When compared head‐to‐head there were no differences between LMWH and UFH for proximal DVT or symptomatic DVT.

When considering PE, the utilization of autopsy as the sole diagnostic method in 2 large trials16, 21 is particularly problematic. In the trial by Garlund,21 the mortality rate was 5.4%, with an autopsy rate of 60.1%. Similarly, in the trial by Mahe et al.,16 the mortality rate was 10%, with an autopsy rate of 49%. Given the low absolute number of deaths and substantial proportion of decedents without autopsy, the potential for chance to produce an imbalance in detection of PE is high in these studies. When we excluded these 2 trials, we found that PE was no longer reduced to a statistically significant degree by prophylaxis. Loss of significance for PE in 2 sensitivity analyses (when excluding studies of lower quality, or using autopsy as a sole diagnostic study) is problematic and calls into question the true benefit of prophylaxis for prevention of PE.

Another limitation of the current literature centers on the variability of dosing used. We pooled trials of UFH whether given BID or TID. Given the small number of trials we did not do sensitivity analyses by dosage. A recent meta‐analysis3 found both doses are efficacious, while a recent review article32 suggested superiority of TID dosing. We believe the available literature does not clearly address this issue. Regarding comparisons of LMWH to UFH, dosing variability was also noted. The trial by Bergmann and Neuhart27 used enoxaparin 20 mg per day and found similar efficacy to UFH BID, while the Samama et al.24 trial found enoxaparin 20 mg per day no more efficacious than placebo. While the literature does not clearly define a best dose, we believe enoxaparin doses lower than 40 mg daily do not reflect the standard of care.

An additional limitation of the literature is publication bias. We assessed the possibility of publication bias by a variety of means. We did find statistical evidence of publication bias for the outcome of PE when prophylaxis was compared to control. Importantly, two meta‐analyses2, 4 on thromboembolism prophylaxis for general medicine patients suggested publication bias is present and our finding supports this conclusion. While no test for publication bias is foolproof, the best protection against publication bias, which we pursued in our study, consists of a thorough search for unpublished studies, including a search of conference proceedings, contact with experts in the field, and manufacturers of LMWH.

A final limitation of the current literature centers on risk assessment. All of the trials in this meta‐analysis included patients with an elevated level of risk. Unfortunately, risk was not clearly defined in many studies, and there was no minimum level of risk between trials. While immobility, age, and length of stay were reported for most studies, other risk factors such as personal history of thromboembolism and malignancy were not uniformly reported. Based on our analysis we are not confident our results can be extrapolated to all general medicine patients.

In conclusion, we found good evidence that pharmacologic prophylaxis significantly decreases the risk of all DVT and proximal DVT in at‐risk general medical patients. However, only LMWH was shown to prevent proximal DVT. We found inconclusive evidence that prophylaxis prevents PE. When compared directly we did not find clear superiority between UFH and LMWH, though several limitations of the current literature hamper decision‐making. Given the lower cost, it may seem justified to use UFH. However, there are other practical issues, such as the fact that LMWH is given once daily, and so potentially preferred by patients and more efficient for nurses. All of these results pertain to patients with elevated risk. While we did not find significant safety concerns with prophylaxis we do not know if these results can be extrapolated to lower‐risk patients. We believe that recommending widespread prophylaxis of all general medicine patients requires additional evidence about appropriate patient selection.

Dramatic changes in the organization, financing, and delivery of hospital care that began a decade ago continue to accelerate. One of the most important changes has been the emergence of hospitalists as providers of inpatient care.1 Hospitalists are physicians, usually general internists, whose clinical focus is the hospitalized patient. As patient illnesses have become more severe and complex, physicians have found it difficult to balance inpatient and outpatient care and have focused on one of the two.25 It is estimated that there are currently 15,000 practicing hospitalists nationally, and projections suggest that this number may exceed 30,000 by 2010, which is equal to the number of cardiologists currently practicing in the United States.6 A 2003 survey from the American Hospital Association showed that more than 30% of the nation's 4900 community hospitals have hospital medicine groups.7 Furthermore, more than 70% of the nation's largest hospitals (>500 beds) and 66% of major teaching hospitals use hospitalists.7

The transition to a hospitalist model generates multiple new research questions about the best approach to caring for the hospitalized patient. Additionally, hospitalists may spawn new areas of clinical research by tackling clinical issues that formerly lacked a large number of specialist investigators. Examples include implementation‐based studies,8, 9 inpatient safety practices,1012 quasi‐experimental studies focusing on common inpatient issues,13, 14 and the evaluation of new methods for reducing resource utilization within various inpatient care delivery structures.15, 16

Similarly, if future clinical trials are to be carried out in real‐world settings, by necessity these will require the participation of hospitalists. Clinical research performed by hospitalists and hospital medicine programs, however, remains underdeveloped. Although this has been attributed to several variables, including the youth of the field, a paucity of fellowship‐trained hospitalist researchers, and a lack of a hospitalist‐oriented national funding source, we also believe that additional barriers exist which could be overcome if hospitalists actively partnered with specialists to perform hospital‐based clinical and translational research.

Hospitalists lack clinical expertise in many clinical issues. In both academic and nonacademic settings, the diagnostic approach, individual treatment decisions, and follow‐up of complex patients occur with frequent consultation of specialists. Specialists often provide a deeper understanding of both the pathophysiologic concepts and scientific principles underlying important clinical questions and are more likely to have had fellowship training that included clinical research experience. Specialists also have more access to extramural funding for disease‐based investigation, and thus their involvement in hospital‐based clinical research would likely enhance funding opportunities, improve project feasibility, and increase dissemination of the results. A successful clinical research program will therefore be one that combines specialists and hospitalists working collaboratively to determine the best way to care for inpatients. With that in mind, we created the University of Michigan SpecialistHospitalist Allied Research Program (SHARP).

METHODS

DISCUSSION AND NEXT STEPS

SHARP is a novel clinical research program partnering hospitalists with specialists. Its current focus targets single‐institution studies that generate pilot data leading to larger projects. The ultimate goal is to develop the ability to do larger multicenter investigator‐initiated projects. The SHARP program will also have the ability to perform observational studies to identify predictors and risk factors and the ability to carry out implementation studies that show how best to translate results from published articles to direct patient care.

A specialisthospitalist collaboration overcomes barriers that we feel may impede hospital medicine research at an academic medical center. For a similar program to succeed at other institutions, key components from our program will have to be replicated. First, senior, fellowship‐trained researchers are required to mentor junior investigators (who may or may not have additional fellowship training), help guide project selection, oversee grant and manuscript submissions, and troubleshoot problems that arise in the course of any clinical research project. In our institution, this comes from within our hospitalist program and from our specialist collaborators. In institutions lacking hospitalists with research experience, this guidance could come from within a division of general medicine, internal medicine specialty divisions, internal medicine department leadership, or even noninternal medicine departments (eg, emergency medicine, neurology, and surgery) that have traditionally been involved in clinical research programs.

A second key component that must be considered is funding. An initial investment is necessary to fund key personnel dedicated to getting projects started on the right track, collecting pilot data, and ensuring project completion and dissemination of the results. The positive margin generated by our hospitalist program facilitated the initial investment. In the absence of a positive margin, resources could come directly from the hospital, the medical school, the department of internal medicine, or perhaps a foundation. The case would need to be made that an initial short‐term investment would enhance the academic standing of the institution, enhance the careers of young investigators, and over time lead to a self‐sustaining program through investigator‐initiated grants and extramural funding. In addition to experienced leadership and funding, we created an oversight committee, but we feel that this is not a critical component. A potential concern with a program that partners with specialists might be that research topics become too disease‐specific or specialty‐oriented. We specifically created the oversight committee to protect against this possibility, and other institutions might need similar safeguards.

Our next step includes leveraging existing hospitalist collaboratives that reach beyond academic medical centers to expand further the reach of SHARP. Ultimately, any new therapy, clinical tool, diagnostic paradigm, or implementation strategy that is developed or evaluated bythe SHARP program would need to be tested in a real‐world setting to assess external validity. With support from the Blue Cross Blue Shield of Michigan Foundation, we have created a multihospital patient safety consortium, the Hospitalists as Emerging Leaders in Patient Safety Consortium, which includes academic, government, urban, rural, teaching, and nonteaching hospitals.23 Although the initial focus is patient safety, our goal for the consortium is to develop it into a multihospital clinical research program that could take pilot projects developed by SHARP and test them in real‐world settings. We believe that full‐scale multihospital studies based on SHARP pilot data will be very attractive to external funding agencies and will help SHARP become financially self‐sufficient after the initial 3‐year start‐up.

Hospital medicine research is desperately needed.24, 25 Unfortunately, the clinical research capabilities of most hospital medicine programs are quite underdeveloped. We believe that partnering hospitalists with specialists can facilitate collaborative research to identify the best way to care for inpatients. If successful, we believe that variations of this model can be replicated at other institutions and will be a critical factor in jumpstarting hospital medicine clinical research.

Dramatic changes in the organization, financing, and delivery of hospital care that began a decade ago continue to accelerate. One of the most important changes has been the emergence of hospitalists as providers of inpatient care.1 Hospitalists are physicians, usually general internists, whose clinical focus is the hospitalized patient. As patient illnesses have become more severe and complex, physicians have found it difficult to balance inpatient and outpatient care and have focused on one of the two.25 It is estimated that there are currently 15,000 practicing hospitalists nationally, and projections suggest that this number may exceed 30,000 by 2010, which is equal to the number of cardiologists currently practicing in the United States.6 A 2003 survey from the American Hospital Association showed that more than 30% of the nation's 4900 community hospitals have hospital medicine groups.7 Furthermore, more than 70% of the nation's largest hospitals (>500 beds) and 66% of major teaching hospitals use hospitalists.7

The transition to a hospitalist model generates multiple new research questions about the best approach to caring for the hospitalized patient. Additionally, hospitalists may spawn new areas of clinical research by tackling clinical issues that formerly lacked a large number of specialist investigators. Examples include implementation‐based studies,8, 9 inpatient safety practices,1012 quasi‐experimental studies focusing on common inpatient issues,13, 14 and the evaluation of new methods for reducing resource utilization within various inpatient care delivery structures.15, 16

Similarly, if future clinical trials are to be carried out in real‐world settings, by necessity these will require the participation of hospitalists. Clinical research performed by hospitalists and hospital medicine programs, however, remains underdeveloped. Although this has been attributed to several variables, including the youth of the field, a paucity of fellowship‐trained hospitalist researchers, and a lack of a hospitalist‐oriented national funding source, we also believe that additional barriers exist which could be overcome if hospitalists actively partnered with specialists to perform hospital‐based clinical and translational research.

Hospitalists lack clinical expertise in many clinical issues. In both academic and nonacademic settings, the diagnostic approach, individual treatment decisions, and follow‐up of complex patients occur with frequent consultation of specialists. Specialists often provide a deeper understanding of both the pathophysiologic concepts and scientific principles underlying important clinical questions and are more likely to have had fellowship training that included clinical research experience. Specialists also have more access to extramural funding for disease‐based investigation, and thus their involvement in hospital‐based clinical research would likely enhance funding opportunities, improve project feasibility, and increase dissemination of the results. A successful clinical research program will therefore be one that combines specialists and hospitalists working collaboratively to determine the best way to care for inpatients. With that in mind, we created the University of Michigan SpecialistHospitalist Allied Research Program (SHARP).

METHODS

DISCUSSION AND NEXT STEPS

SHARP is a novel clinical research program partnering hospitalists with specialists. Its current focus targets single‐institution studies that generate pilot data leading to larger projects. The ultimate goal is to develop the ability to do larger multicenter investigator‐initiated projects. The SHARP program will also have the ability to perform observational studies to identify predictors and risk factors and the ability to carry out implementation studies that show how best to translate results from published articles to direct patient care.

A specialisthospitalist collaboration overcomes barriers that we feel may impede hospital medicine research at an academic medical center. For a similar program to succeed at other institutions, key components from our program will have to be replicated. First, senior, fellowship‐trained researchers are required to mentor junior investigators (who may or may not have additional fellowship training), help guide project selection, oversee grant and manuscript submissions, and troubleshoot problems that arise in the course of any clinical research project. In our institution, this comes from within our hospitalist program and from our specialist collaborators. In institutions lacking hospitalists with research experience, this guidance could come from within a division of general medicine, internal medicine specialty divisions, internal medicine department leadership, or even noninternal medicine departments (eg, emergency medicine, neurology, and surgery) that have traditionally been involved in clinical research programs.

A second key component that must be considered is funding. An initial investment is necessary to fund key personnel dedicated to getting projects started on the right track, collecting pilot data, and ensuring project completion and dissemination of the results. The positive margin generated by our hospitalist program facilitated the initial investment. In the absence of a positive margin, resources could come directly from the hospital, the medical school, the department of internal medicine, or perhaps a foundation. The case would need to be made that an initial short‐term investment would enhance the academic standing of the institution, enhance the careers of young investigators, and over time lead to a self‐sustaining program through investigator‐initiated grants and extramural funding. In addition to experienced leadership and funding, we created an oversight committee, but we feel that this is not a critical component. A potential concern with a program that partners with specialists might be that research topics become too disease‐specific or specialty‐oriented. We specifically created the oversight committee to protect against this possibility, and other institutions might need similar safeguards.

Our next step includes leveraging existing hospitalist collaboratives that reach beyond academic medical centers to expand further the reach of SHARP. Ultimately, any new therapy, clinical tool, diagnostic paradigm, or implementation strategy that is developed or evaluated bythe SHARP program would need to be tested in a real‐world setting to assess external validity. With support from the Blue Cross Blue Shield of Michigan Foundation, we have created a multihospital patient safety consortium, the Hospitalists as Emerging Leaders in Patient Safety Consortium, which includes academic, government, urban, rural, teaching, and nonteaching hospitals.23 Although the initial focus is patient safety, our goal for the consortium is to develop it into a multihospital clinical research program that could take pilot projects developed by SHARP and test them in real‐world settings. We believe that full‐scale multihospital studies based on SHARP pilot data will be very attractive to external funding agencies and will help SHARP become financially self‐sufficient after the initial 3‐year start‐up.

Hospital medicine research is desperately needed.24, 25 Unfortunately, the clinical research capabilities of most hospital medicine programs are quite underdeveloped. We believe that partnering hospitalists with specialists can facilitate collaborative research to identify the best way to care for inpatients. If successful, we believe that variations of this model can be replicated at other institutions and will be a critical factor in jumpstarting hospital medicine clinical research.

Dramatic changes in the organization, financing, and delivery of hospital care that began a decade ago continue to accelerate. One of the most important changes has been the emergence of hospitalists as providers of inpatient care.1 Hospitalists are physicians, usually general internists, whose clinical focus is the hospitalized patient. As patient illnesses have become more severe and complex, physicians have found it difficult to balance inpatient and outpatient care and have focused on one of the two.25 It is estimated that there are currently 15,000 practicing hospitalists nationally, and projections suggest that this number may exceed 30,000 by 2010, which is equal to the number of cardiologists currently practicing in the United States.6 A 2003 survey from the American Hospital Association showed that more than 30% of the nation's 4900 community hospitals have hospital medicine groups.7 Furthermore, more than 70% of the nation's largest hospitals (>500 beds) and 66% of major teaching hospitals use hospitalists.7

The transition to a hospitalist model generates multiple new research questions about the best approach to caring for the hospitalized patient. Additionally, hospitalists may spawn new areas of clinical research by tackling clinical issues that formerly lacked a large number of specialist investigators. Examples include implementation‐based studies,8, 9 inpatient safety practices,1012 quasi‐experimental studies focusing on common inpatient issues,13, 14 and the evaluation of new methods for reducing resource utilization within various inpatient care delivery structures.15, 16

Similarly, if future clinical trials are to be carried out in real‐world settings, by necessity these will require the participation of hospitalists. Clinical research performed by hospitalists and hospital medicine programs, however, remains underdeveloped. Although this has been attributed to several variables, including the youth of the field, a paucity of fellowship‐trained hospitalist researchers, and a lack of a hospitalist‐oriented national funding source, we also believe that additional barriers exist which could be overcome if hospitalists actively partnered with specialists to perform hospital‐based clinical and translational research.

Hospitalists lack clinical expertise in many clinical issues. In both academic and nonacademic settings, the diagnostic approach, individual treatment decisions, and follow‐up of complex patients occur with frequent consultation of specialists. Specialists often provide a deeper understanding of both the pathophysiologic concepts and scientific principles underlying important clinical questions and are more likely to have had fellowship training that included clinical research experience. Specialists also have more access to extramural funding for disease‐based investigation, and thus their involvement in hospital‐based clinical research would likely enhance funding opportunities, improve project feasibility, and increase dissemination of the results. A successful clinical research program will therefore be one that combines specialists and hospitalists working collaboratively to determine the best way to care for inpatients. With that in mind, we created the University of Michigan SpecialistHospitalist Allied Research Program (SHARP).

METHODS

DISCUSSION AND NEXT STEPS

SHARP is a novel clinical research program partnering hospitalists with specialists. Its current focus targets single‐institution studies that generate pilot data leading to larger projects. The ultimate goal is to develop the ability to do larger multicenter investigator‐initiated projects. The SHARP program will also have the ability to perform observational studies to identify predictors and risk factors and the ability to carry out implementation studies that show how best to translate results from published articles to direct patient care.

A specialisthospitalist collaboration overcomes barriers that we feel may impede hospital medicine research at an academic medical center. For a similar program to succeed at other institutions, key components from our program will have to be replicated. First, senior, fellowship‐trained researchers are required to mentor junior investigators (who may or may not have additional fellowship training), help guide project selection, oversee grant and manuscript submissions, and troubleshoot problems that arise in the course of any clinical research project. In our institution, this comes from within our hospitalist program and from our specialist collaborators. In institutions lacking hospitalists with research experience, this guidance could come from within a division of general medicine, internal medicine specialty divisions, internal medicine department leadership, or even noninternal medicine departments (eg, emergency medicine, neurology, and surgery) that have traditionally been involved in clinical research programs.

A second key component that must be considered is funding. An initial investment is necessary to fund key personnel dedicated to getting projects started on the right track, collecting pilot data, and ensuring project completion and dissemination of the results. The positive margin generated by our hospitalist program facilitated the initial investment. In the absence of a positive margin, resources could come directly from the hospital, the medical school, the department of internal medicine, or perhaps a foundation. The case would need to be made that an initial short‐term investment would enhance the academic standing of the institution, enhance the careers of young investigators, and over time lead to a self‐sustaining program through investigator‐initiated grants and extramural funding. In addition to experienced leadership and funding, we created an oversight committee, but we feel that this is not a critical component. A potential concern with a program that partners with specialists might be that research topics become too disease‐specific or specialty‐oriented. We specifically created the oversight committee to protect against this possibility, and other institutions might need similar safeguards.

Our next step includes leveraging existing hospitalist collaboratives that reach beyond academic medical centers to expand further the reach of SHARP. Ultimately, any new therapy, clinical tool, diagnostic paradigm, or implementation strategy that is developed or evaluated bythe SHARP program would need to be tested in a real‐world setting to assess external validity. With support from the Blue Cross Blue Shield of Michigan Foundation, we have created a multihospital patient safety consortium, the Hospitalists as Emerging Leaders in Patient Safety Consortium, which includes academic, government, urban, rural, teaching, and nonteaching hospitals.23 Although the initial focus is patient safety, our goal for the consortium is to develop it into a multihospital clinical research program that could take pilot projects developed by SHARP and test them in real‐world settings. We believe that full‐scale multihospital studies based on SHARP pilot data will be very attractive to external funding agencies and will help SHARP become financially self‐sufficient after the initial 3‐year start‐up.

Hospital medicine research is desperately needed.24, 25 Unfortunately, the clinical research capabilities of most hospital medicine programs are quite underdeveloped. We believe that partnering hospitalists with specialists can facilitate collaborative research to identify the best way to care for inpatients. If successful, we believe that variations of this model can be replicated at other institutions and will be a critical factor in jumpstarting hospital medicine clinical research.

Patient safety can be understood in terms of the Swiss cheese model of systems accidents. This model implies that many holes must align before an adverse event occurs.1 The limitations on work hours instituted by the Accreditation Council for Graduate Medical Education (ACGME)2 sought to close one hole by reducing fatigue in residents. As programs comply with these regulations, new interventions are being implemented to limit resident hours. This has resulted in more handoffs of care and therefore less continuity. The ultimate result may be to increase patient care errors by opening up new holes, the opposite of the stated goal of this reform.

Some residency programs have reported on their experience with hour reductions, giving insight into residents' perceptions on the benefits and drawbacks of such interventions. Residents have reported concern about continuity of care after such interventions.37 However, some residents believed they provided better patient care after the interventions to reduce hours.8, 9 Few studies have actually documented changes in the incidence of adverse events or errors as a result of work hour limitations.10 One study conducted prior to implementation of the ACGME work hour rules demonstrated more complications in internal medicine patients after New York's Code 405 (a state regulation that limited resident work hours, similar to the ACGME rules) was implemented.11 In contrast, another study showed that errors committed by interns were reduced with scheduling changes that resulted in shorter shifts and reduced hours.12

Because residents are on the front lines of patient care, they are uniquely positioned to provide insight into the impact of the work hour rules on patient safety. We conducted this study to more fully understand the effect of the ACGME work hour limitations and other possible factors on patient care errors from the perspectives of internal medicine residents.

METHODS

Design

We conducted 4 focus groups in total (Appendix 1). The first 3 focus groups followed the same focus group guide, developed after a literature review. Focus groups 1 and 2 were conducted at the university‐based program. Focus group 3 was conducted at the community teaching hospitalaffiliated program. The first 3 focus groups were analyzed before the fourth focus group was conducted. A new focus group guide was developed for the fourth focus group to further explore themes identified in the first 3 focus groups (Fig. 1 and Appendix 2). The fourth focus group was conducted at the program affiliated with a freestanding medical college. All focus groups were audiotaped and transcribed verbatim. Each lasted approximately 90‐120 minutes.

Figure 1

RESULTS

Twenty‐eight residents participated. Some of these residents had experience in the prework hour era, and some did not. Average age was 28 years (range 26‐33 years); 18 were women, and 11 were interns (Table 2). The focus groups ranged in size from 5 to 9. A sample of the codes and their definitions can be found in Table 3.

The Model

The model (Fig. 2) illustrates resident‐perceived contributors to patient care mistakes related to the ACGME work hour rules. These contributors are in the center circle. They include fatigue, inexperience, sign‐out, not knowing their own patients well enough, entropy (which we defined as the amount of chaos in the system), and workload. They are not listed in order of importance. The boxes outside the circle are consequences of the ACGME work hour rules and their perceived impact on the contributors to patient care mistakes. At the top are the intended consequences, that is the specific goals of the ACGME: less resident time in the hospital (ie, reduced hours) and improved well‐being.17 At the bottom are the unintended consequences: more patient care discontinuity and compliance with the work hour rules becoming a goal equally important to providing high‐quality patient care. Of these 4 consequences, only improved well‐being was viewed by the residents as decreasing patient care mistakes. The other consequences were cited by residents as sometimes increasing patient care errors. Because of the complexity of the model, several factors not directly related to resident work hours were identified in the analysis but are not shown in the model. They include faculty involvement and team work (usually positive influences), nurses and information technology (could be positive or negative), and late‐night/early‐morning hours (negative).

Figure 2

The quotations below illustrate the relationships between the consequences of the work hour rules, resident‐perceived contributors to patient care mistakes, and actual patient care.

DISCUSSION

We conducted this study to better understand why internal medicine residents thought patient care mistakes occurred; we were particularly interested in how they perceived the impact of certain aspects of the ACGME work hour rules on patient care mistakes. Designing systems that achieve compliance with the work hour rules while minimizing patient risk can best be accomplished by fully understanding why errors occur.

Our study revealed that in the opinion of some interns and residents, the work hour rules had consequences for patient care. Like any intervention, this one had both intended and unintended consequences.18 The ACGME has stated that improvement in residents' quality of life was an intended consequence,17 and the participants in our study reported that this had occurred. Despite uncertainty about the overall impact on patient outcomes, residents were glad to have more time away from the hospital.

Our respondents reported that not knowing patients well was a factor that contributed to patient care errors. It is intuitive that working fewer hours often results in more handoffs of care,19 a situation characterized by not knowing patients well. However, residents also identified not getting to know their own patients well as a factor that led to patient care mistakes because of (1) incomplete knowledge of a patient's status, (2) delays in diagnosis, and (3) errors in management. They also described feelings of professional disappointment and frustration at not being able to perform certain aspects of patient care (eg, family meetings) because of the hour limits and the inflexibility of the rules. As we strive to redefine professionalism in the setting of reduced work hours,20 this phenomenon should be addressed.

Sign‐out was identified as another contributor to patient care errors. The effectiveness of sign‐outs is a concern across medicine, and the Joint Commission on Accreditation of Healthcare Organizations made sign‐out procedures one of its priority areas in 2006.21 Much has been written about resident sign‐out, emphasizing the relationship between poor‐quality sign‐outs and patient safety.19, 22 However, barriers to effective sign‐out processes persist,23 even though standardized sign‐out strategies have been described.24, 25 Even in a rigorous study of work hours and patient safety, the computerized sign‐out template for the residents was rarely used.12 Cross‐coverage, or the patient care that occurs after sign‐out is complete, has also been linked to a greater likelihood of adverse events.26

Several factors not related to resident work hours were noted to often mitigate patient care mistakes. Physician teamwork, nursing, information technology (eg, computerized medical records), and faculty supervision were the most prominent. For example, the information technology available at the VA hospitals often helped to facilitate patient care, but it also provided an overwhelming amount of information to sift through. It was clear that the influence of some of these factors varied from institution to institution, reflecting the cultures of different programs.

Our results are consistent with those reported from previous studies. Striking a balance between preventing resident fatigue and preserving continuity of care has been debated since the ACGME announced changes to resident work hour limits.27 Resident quality of life generally improves and fatigue decreases with work hour limits in place,28 but patient safety remains a concern.10 Our findings corroborate the benefits of improved resident well‐being and the persistent concerns about patient safety, identified in a recently published study at a different institution.29 However, our findings expand on those reported in the literature by offering additional empirical evidence, albeit qualitative, about the way that residents see the relationships among the consequences of work hour rules, resident‐reported contributors to patient care mistakes, and the mistakes themselves.

Our study should be interpreted in the context of several limitations. First, the use of qualitative methods did not allow us to generalize or quantify our findings. However, we purposely included 3 diverse institutions with differing responses to the work hour rules to enhance the external validity of our findings. Second, the last focus group was conducted a year after the first 3; by that point, the work hour rules had been in place for 20 months. We believe that this was both a strength and a limitation because it allowed us to gain a perspective after some of the initial growing pains were over. This time lag also allowed for analysis of the first 3 transcripts so we could revise the focus group guide and ultimately determine that thematic saturation had occurred. In addition, few of our questions were phrased to evaluate the ACGME rules; instead, they asked about links among discontinuity, scheduling, fatigue, and patient care. We therefore believe that even residents who were not in the programs before the work hour rules began were still able to knowledgeably participate in the conversation. One question directly referable to the ACGME rules asked residents to reflect on problems arising from them. This could have led residents to only reflect on the problems associated with the rules. However, in all 4 focus groups, residents commented on the positive impact of improved well‐being resulting from the work hour rules. This led us to believe the respondents felt they could voice their favorable feelings as well as their unfavorable feelings about the rules. An additional limitation is that the agreement between coders was only 45%. It is important to realize that assessing coding agreement in qualitative work is quite difficult because it is often difficult to assign a single code to a section of text. When the coders discussed a disagreement, it was almost always the case that the difference was subtle and that the coding of either investigator would made sense for that text. Finally, our results are based on the participation of 28 residents. To be certain we were not representing the opinions of only a few people, we presented iterations of this model to faculty and resident groups for their feedback. Importantly, the residents offered no substantial changes or criticisms of the model.

Limitations notwithstanding, we believe our findings have important policy implications. First, despite work hours successfully being reduced, residents perceived no decrease in the amount of work they did. This resulted in higher workload and more entropy, suggesting that residency programs may need to carefully evaluate the patient care responsibility carried by residents. Second, continued effort to educate residents to provide effective sign‐out is needed. As one participant pointed out, residency offers a unique opportunity to learn to manage discontinuity in a controlled setting. Another educational opportunity is the chance to teach physician teamwork. Participants believed that effective teamwork could ameliorate some of the discontinuity in patient care. This teamwork training should include faculty as well, although further work is needed to define how faculty can best add to patient continuity while still fostering resident autonomy. Finally, the impact of work hour rules on the professional development of residents should be further explored.

In conclusion, we have proposed a model to explain the major resident‐reported contributors to patient care mistakes with respect to resident work hour rules. Our results help to clarify the next steps needed: testing the proposed relationships between the factors and patient care mistakes and rigorously evaluating solutions that minimize the impact of these factors. Returning to the Swiss cheese framework for describing systems accidents, our results suggest that although resident work hour reductions may have sufficiently filled the hole caused by resident fatigue, other gaps may have actually widened as a result of the systems put into place to achieve compliance. Continued vigilance is therefore necessary to both identify the additional holes likely to appear and, more importantly, effectively close those holes before patient harm occurs.

FOCUS GROUP GUIDE (4TH FOCUS GROUP)

The IOM definition of a medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.

• 1

  Please describe the call structure at each institution where you do ward months (eg, non‐ICU months).

• 2

  What are some settings at the hospitals where you work where you have seen medical errors, mistakes, or adverse outcomes?

• 3

  How do you think that other nurses influence the occurrence of medical errors, mistakes, or adverse outcomes?

   

  • a

    Clerks?

  • b

    Other ancillary staff?

  • 4

    How would you describe the responsibilities of a cross‐covering resident or intern?

  • 5

    How do you think continuity of care impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

     

    • a

      What role do sign‐outs have?

    • 6

      How do you think that fatigue impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

    • 7

      How do you think that technology such as computerized physician order entry impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

       

      • a

        Electronic medical records?

      • b

        Palm pilots?

      • c

        Is there such a thing as too much information?

      • 8

        How do you think that experience (or inexperience) impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

      • 9

        Please describe how attendings supervise you when you are on a ward team. How do you think that attending supervision impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

         

        • a

          What about resident supervision of interns?

        • b

          What is the ideal role of an attending on a team?

        • c

          Can you think of a time when having attending input changed the plans or the course of a patient in a major way, good, bad, or neutral?

        • 10

          How do you think that time of day impacts patient care in terms of in terms of medical errors, mistakes, or adverse outcomes?

           

          • a

            How comfortable do you feel calling for help at night? What makes you more or less likely to do it (personal attributes of person to be called, situation, etc.)?

          • 11

            What do you think is an ideal workload? (eg, How many complex patients are typical of your hospitals?) Does that vary from the VA to St. Joe's to Froedtert? How many patients should be admitted in 1 night by an intern? How many should an intern have ongoing responsibility for? Is there such a thing as too few patients?

          • 12

            If one of your family members were to admitted to your hospital at night with a life‐threatening condition, which situation would you prefer for their care (all other things being equal): admission by night float with handoff to a new but well‐rested resident or admission by a resident who then continues to care for that family member the next day but has been awake for 24 hours, admitting and cross‐covering other patients? Why?

          • 13

            What do you think was the intent of the ACGME rules? Do you think that those goals have been accomplished? Why or why not? How have they affected you as residents? How do you think that the ACGME work hour rules have influenced patient care?

           

Patient safety can be understood in terms of the Swiss cheese model of systems accidents. This model implies that many holes must align before an adverse event occurs.1 The limitations on work hours instituted by the Accreditation Council for Graduate Medical Education (ACGME)2 sought to close one hole by reducing fatigue in residents. As programs comply with these regulations, new interventions are being implemented to limit resident hours. This has resulted in more handoffs of care and therefore less continuity. The ultimate result may be to increase patient care errors by opening up new holes, the opposite of the stated goal of this reform.

Some residency programs have reported on their experience with hour reductions, giving insight into residents' perceptions on the benefits and drawbacks of such interventions. Residents have reported concern about continuity of care after such interventions.37 However, some residents believed they provided better patient care after the interventions to reduce hours.8, 9 Few studies have actually documented changes in the incidence of adverse events or errors as a result of work hour limitations.10 One study conducted prior to implementation of the ACGME work hour rules demonstrated more complications in internal medicine patients after New York's Code 405 (a state regulation that limited resident work hours, similar to the ACGME rules) was implemented.11 In contrast, another study showed that errors committed by interns were reduced with scheduling changes that resulted in shorter shifts and reduced hours.12

Because residents are on the front lines of patient care, they are uniquely positioned to provide insight into the impact of the work hour rules on patient safety. We conducted this study to more fully understand the effect of the ACGME work hour limitations and other possible factors on patient care errors from the perspectives of internal medicine residents.

METHODS

Design

We conducted 4 focus groups in total (Appendix 1). The first 3 focus groups followed the same focus group guide, developed after a literature review. Focus groups 1 and 2 were conducted at the university‐based program. Focus group 3 was conducted at the community teaching hospitalaffiliated program. The first 3 focus groups were analyzed before the fourth focus group was conducted. A new focus group guide was developed for the fourth focus group to further explore themes identified in the first 3 focus groups (Fig. 1 and Appendix 2). The fourth focus group was conducted at the program affiliated with a freestanding medical college. All focus groups were audiotaped and transcribed verbatim. Each lasted approximately 90‐120 minutes.

Figure 1

RESULTS

Twenty‐eight residents participated. Some of these residents had experience in the prework hour era, and some did not. Average age was 28 years (range 26‐33 years); 18 were women, and 11 were interns (Table 2). The focus groups ranged in size from 5 to 9. A sample of the codes and their definitions can be found in Table 3.

The Model

The model (Fig. 2) illustrates resident‐perceived contributors to patient care mistakes related to the ACGME work hour rules. These contributors are in the center circle. They include fatigue, inexperience, sign‐out, not knowing their own patients well enough, entropy (which we defined as the amount of chaos in the system), and workload. They are not listed in order of importance. The boxes outside the circle are consequences of the ACGME work hour rules and their perceived impact on the contributors to patient care mistakes. At the top are the intended consequences, that is the specific goals of the ACGME: less resident time in the hospital (ie, reduced hours) and improved well‐being.17 At the bottom are the unintended consequences: more patient care discontinuity and compliance with the work hour rules becoming a goal equally important to providing high‐quality patient care. Of these 4 consequences, only improved well‐being was viewed by the residents as decreasing patient care mistakes. The other consequences were cited by residents as sometimes increasing patient care errors. Because of the complexity of the model, several factors not directly related to resident work hours were identified in the analysis but are not shown in the model. They include faculty involvement and team work (usually positive influences), nurses and information technology (could be positive or negative), and late‐night/early‐morning hours (negative).

Figure 2

The quotations below illustrate the relationships between the consequences of the work hour rules, resident‐perceived contributors to patient care mistakes, and actual patient care.

DISCUSSION

We conducted this study to better understand why internal medicine residents thought patient care mistakes occurred; we were particularly interested in how they perceived the impact of certain aspects of the ACGME work hour rules on patient care mistakes. Designing systems that achieve compliance with the work hour rules while minimizing patient risk can best be accomplished by fully understanding why errors occur.

Our study revealed that in the opinion of some interns and residents, the work hour rules had consequences for patient care. Like any intervention, this one had both intended and unintended consequences.18 The ACGME has stated that improvement in residents' quality of life was an intended consequence,17 and the participants in our study reported that this had occurred. Despite uncertainty about the overall impact on patient outcomes, residents were glad to have more time away from the hospital.

Our respondents reported that not knowing patients well was a factor that contributed to patient care errors. It is intuitive that working fewer hours often results in more handoffs of care,19 a situation characterized by not knowing patients well. However, residents also identified not getting to know their own patients well as a factor that led to patient care mistakes because of (1) incomplete knowledge of a patient's status, (2) delays in diagnosis, and (3) errors in management. They also described feelings of professional disappointment and frustration at not being able to perform certain aspects of patient care (eg, family meetings) because of the hour limits and the inflexibility of the rules. As we strive to redefine professionalism in the setting of reduced work hours,20 this phenomenon should be addressed.

Sign‐out was identified as another contributor to patient care errors. The effectiveness of sign‐outs is a concern across medicine, and the Joint Commission on Accreditation of Healthcare Organizations made sign‐out procedures one of its priority areas in 2006.21 Much has been written about resident sign‐out, emphasizing the relationship between poor‐quality sign‐outs and patient safety.19, 22 However, barriers to effective sign‐out processes persist,23 even though standardized sign‐out strategies have been described.24, 25 Even in a rigorous study of work hours and patient safety, the computerized sign‐out template for the residents was rarely used.12 Cross‐coverage, or the patient care that occurs after sign‐out is complete, has also been linked to a greater likelihood of adverse events.26

Several factors not related to resident work hours were noted to often mitigate patient care mistakes. Physician teamwork, nursing, information technology (eg, computerized medical records), and faculty supervision were the most prominent. For example, the information technology available at the VA hospitals often helped to facilitate patient care, but it also provided an overwhelming amount of information to sift through. It was clear that the influence of some of these factors varied from institution to institution, reflecting the cultures of different programs.

Our results are consistent with those reported from previous studies. Striking a balance between preventing resident fatigue and preserving continuity of care has been debated since the ACGME announced changes to resident work hour limits.27 Resident quality of life generally improves and fatigue decreases with work hour limits in place,28 but patient safety remains a concern.10 Our findings corroborate the benefits of improved resident well‐being and the persistent concerns about patient safety, identified in a recently published study at a different institution.29 However, our findings expand on those reported in the literature by offering additional empirical evidence, albeit qualitative, about the way that residents see the relationships among the consequences of work hour rules, resident‐reported contributors to patient care mistakes, and the mistakes themselves.

Our study should be interpreted in the context of several limitations. First, the use of qualitative methods did not allow us to generalize or quantify our findings. However, we purposely included 3 diverse institutions with differing responses to the work hour rules to enhance the external validity of our findings. Second, the last focus group was conducted a year after the first 3; by that point, the work hour rules had been in place for 20 months. We believe that this was both a strength and a limitation because it allowed us to gain a perspective after some of the initial growing pains were over. This time lag also allowed for analysis of the first 3 transcripts so we could revise the focus group guide and ultimately determine that thematic saturation had occurred. In addition, few of our questions were phrased to evaluate the ACGME rules; instead, they asked about links among discontinuity, scheduling, fatigue, and patient care. We therefore believe that even residents who were not in the programs before the work hour rules began were still able to knowledgeably participate in the conversation. One question directly referable to the ACGME rules asked residents to reflect on problems arising from them. This could have led residents to only reflect on the problems associated with the rules. However, in all 4 focus groups, residents commented on the positive impact of improved well‐being resulting from the work hour rules. This led us to believe the respondents felt they could voice their favorable feelings as well as their unfavorable feelings about the rules. An additional limitation is that the agreement between coders was only 45%. It is important to realize that assessing coding agreement in qualitative work is quite difficult because it is often difficult to assign a single code to a section of text. When the coders discussed a disagreement, it was almost always the case that the difference was subtle and that the coding of either investigator would made sense for that text. Finally, our results are based on the participation of 28 residents. To be certain we were not representing the opinions of only a few people, we presented iterations of this model to faculty and resident groups for their feedback. Importantly, the residents offered no substantial changes or criticisms of the model.

Limitations notwithstanding, we believe our findings have important policy implications. First, despite work hours successfully being reduced, residents perceived no decrease in the amount of work they did. This resulted in higher workload and more entropy, suggesting that residency programs may need to carefully evaluate the patient care responsibility carried by residents. Second, continued effort to educate residents to provide effective sign‐out is needed. As one participant pointed out, residency offers a unique opportunity to learn to manage discontinuity in a controlled setting. Another educational opportunity is the chance to teach physician teamwork. Participants believed that effective teamwork could ameliorate some of the discontinuity in patient care. This teamwork training should include faculty as well, although further work is needed to define how faculty can best add to patient continuity while still fostering resident autonomy. Finally, the impact of work hour rules on the professional development of residents should be further explored.

In conclusion, we have proposed a model to explain the major resident‐reported contributors to patient care mistakes with respect to resident work hour rules. Our results help to clarify the next steps needed: testing the proposed relationships between the factors and patient care mistakes and rigorously evaluating solutions that minimize the impact of these factors. Returning to the Swiss cheese framework for describing systems accidents, our results suggest that although resident work hour reductions may have sufficiently filled the hole caused by resident fatigue, other gaps may have actually widened as a result of the systems put into place to achieve compliance. Continued vigilance is therefore necessary to both identify the additional holes likely to appear and, more importantly, effectively close those holes before patient harm occurs.

FOCUS GROUP GUIDE (4TH FOCUS GROUP)

The IOM definition of a medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.

• 1

  Please describe the call structure at each institution where you do ward months (eg, non‐ICU months).

• 2

  What are some settings at the hospitals where you work where you have seen medical errors, mistakes, or adverse outcomes?

• 3

  How do you think that other nurses influence the occurrence of medical errors, mistakes, or adverse outcomes?

   

  • a

    Clerks?

  • b

    Other ancillary staff?

  • 4

    How would you describe the responsibilities of a cross‐covering resident or intern?

  • 5

    How do you think continuity of care impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

     

    • a

      What role do sign‐outs have?

    • 6

      How do you think that fatigue impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

    • 7

      How do you think that technology such as computerized physician order entry impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

       

      • a

        Electronic medical records?

      • b

        Palm pilots?

      • c

        Is there such a thing as too much information?

      • 8

        How do you think that experience (or inexperience) impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

      • 9

        Please describe how attendings supervise you when you are on a ward team. How do you think that attending supervision impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

         

        • a

          What about resident supervision of interns?

        • b

          What is the ideal role of an attending on a team?

        • c

          Can you think of a time when having attending input changed the plans or the course of a patient in a major way, good, bad, or neutral?

        • 10

          How do you think that time of day impacts patient care in terms of in terms of medical errors, mistakes, or adverse outcomes?

           

          • a

            How comfortable do you feel calling for help at night? What makes you more or less likely to do it (personal attributes of person to be called, situation, etc.)?

          • 11

            What do you think is an ideal workload? (eg, How many complex patients are typical of your hospitals?) Does that vary from the VA to St. Joe's to Froedtert? How many patients should be admitted in 1 night by an intern? How many should an intern have ongoing responsibility for? Is there such a thing as too few patients?

          • 12

            If one of your family members were to admitted to your hospital at night with a life‐threatening condition, which situation would you prefer for their care (all other things being equal): admission by night float with handoff to a new but well‐rested resident or admission by a resident who then continues to care for that family member the next day but has been awake for 24 hours, admitting and cross‐covering other patients? Why?

          • 13

            What do you think was the intent of the ACGME rules? Do you think that those goals have been accomplished? Why or why not? How have they affected you as residents? How do you think that the ACGME work hour rules have influenced patient care?

           

Patient safety can be understood in terms of the Swiss cheese model of systems accidents. This model implies that many holes must align before an adverse event occurs.1 The limitations on work hours instituted by the Accreditation Council for Graduate Medical Education (ACGME)2 sought to close one hole by reducing fatigue in residents. As programs comply with these regulations, new interventions are being implemented to limit resident hours. This has resulted in more handoffs of care and therefore less continuity. The ultimate result may be to increase patient care errors by opening up new holes, the opposite of the stated goal of this reform.

Some residency programs have reported on their experience with hour reductions, giving insight into residents' perceptions on the benefits and drawbacks of such interventions. Residents have reported concern about continuity of care after such interventions.37 However, some residents believed they provided better patient care after the interventions to reduce hours.8, 9 Few studies have actually documented changes in the incidence of adverse events or errors as a result of work hour limitations.10 One study conducted prior to implementation of the ACGME work hour rules demonstrated more complications in internal medicine patients after New York's Code 405 (a state regulation that limited resident work hours, similar to the ACGME rules) was implemented.11 In contrast, another study showed that errors committed by interns were reduced with scheduling changes that resulted in shorter shifts and reduced hours.12

Because residents are on the front lines of patient care, they are uniquely positioned to provide insight into the impact of the work hour rules on patient safety. We conducted this study to more fully understand the effect of the ACGME work hour limitations and other possible factors on patient care errors from the perspectives of internal medicine residents.

METHODS

Design

We conducted 4 focus groups in total (Appendix 1). The first 3 focus groups followed the same focus group guide, developed after a literature review. Focus groups 1 and 2 were conducted at the university‐based program. Focus group 3 was conducted at the community teaching hospitalaffiliated program. The first 3 focus groups were analyzed before the fourth focus group was conducted. A new focus group guide was developed for the fourth focus group to further explore themes identified in the first 3 focus groups (Fig. 1 and Appendix 2). The fourth focus group was conducted at the program affiliated with a freestanding medical college. All focus groups were audiotaped and transcribed verbatim. Each lasted approximately 90‐120 minutes.

Figure 1

RESULTS

Twenty‐eight residents participated. Some of these residents had experience in the prework hour era, and some did not. Average age was 28 years (range 26‐33 years); 18 were women, and 11 were interns (Table 2). The focus groups ranged in size from 5 to 9. A sample of the codes and their definitions can be found in Table 3.

The Model

The model (Fig. 2) illustrates resident‐perceived contributors to patient care mistakes related to the ACGME work hour rules. These contributors are in the center circle. They include fatigue, inexperience, sign‐out, not knowing their own patients well enough, entropy (which we defined as the amount of chaos in the system), and workload. They are not listed in order of importance. The boxes outside the circle are consequences of the ACGME work hour rules and their perceived impact on the contributors to patient care mistakes. At the top are the intended consequences, that is the specific goals of the ACGME: less resident time in the hospital (ie, reduced hours) and improved well‐being.17 At the bottom are the unintended consequences: more patient care discontinuity and compliance with the work hour rules becoming a goal equally important to providing high‐quality patient care. Of these 4 consequences, only improved well‐being was viewed by the residents as decreasing patient care mistakes. The other consequences were cited by residents as sometimes increasing patient care errors. Because of the complexity of the model, several factors not directly related to resident work hours were identified in the analysis but are not shown in the model. They include faculty involvement and team work (usually positive influences), nurses and information technology (could be positive or negative), and late‐night/early‐morning hours (negative).

Figure 2

The quotations below illustrate the relationships between the consequences of the work hour rules, resident‐perceived contributors to patient care mistakes, and actual patient care.

DISCUSSION

We conducted this study to better understand why internal medicine residents thought patient care mistakes occurred; we were particularly interested in how they perceived the impact of certain aspects of the ACGME work hour rules on patient care mistakes. Designing systems that achieve compliance with the work hour rules while minimizing patient risk can best be accomplished by fully understanding why errors occur.

Our study revealed that in the opinion of some interns and residents, the work hour rules had consequences for patient care. Like any intervention, this one had both intended and unintended consequences.18 The ACGME has stated that improvement in residents' quality of life was an intended consequence,17 and the participants in our study reported that this had occurred. Despite uncertainty about the overall impact on patient outcomes, residents were glad to have more time away from the hospital.

Our respondents reported that not knowing patients well was a factor that contributed to patient care errors. It is intuitive that working fewer hours often results in more handoffs of care,19 a situation characterized by not knowing patients well. However, residents also identified not getting to know their own patients well as a factor that led to patient care mistakes because of (1) incomplete knowledge of a patient's status, (2) delays in diagnosis, and (3) errors in management. They also described feelings of professional disappointment and frustration at not being able to perform certain aspects of patient care (eg, family meetings) because of the hour limits and the inflexibility of the rules. As we strive to redefine professionalism in the setting of reduced work hours,20 this phenomenon should be addressed.

Sign‐out was identified as another contributor to patient care errors. The effectiveness of sign‐outs is a concern across medicine, and the Joint Commission on Accreditation of Healthcare Organizations made sign‐out procedures one of its priority areas in 2006.21 Much has been written about resident sign‐out, emphasizing the relationship between poor‐quality sign‐outs and patient safety.19, 22 However, barriers to effective sign‐out processes persist,23 even though standardized sign‐out strategies have been described.24, 25 Even in a rigorous study of work hours and patient safety, the computerized sign‐out template for the residents was rarely used.12 Cross‐coverage, or the patient care that occurs after sign‐out is complete, has also been linked to a greater likelihood of adverse events.26

Several factors not related to resident work hours were noted to often mitigate patient care mistakes. Physician teamwork, nursing, information technology (eg, computerized medical records), and faculty supervision were the most prominent. For example, the information technology available at the VA hospitals often helped to facilitate patient care, but it also provided an overwhelming amount of information to sift through. It was clear that the influence of some of these factors varied from institution to institution, reflecting the cultures of different programs.

Our results are consistent with those reported from previous studies. Striking a balance between preventing resident fatigue and preserving continuity of care has been debated since the ACGME announced changes to resident work hour limits.27 Resident quality of life generally improves and fatigue decreases with work hour limits in place,28 but patient safety remains a concern.10 Our findings corroborate the benefits of improved resident well‐being and the persistent concerns about patient safety, identified in a recently published study at a different institution.29 However, our findings expand on those reported in the literature by offering additional empirical evidence, albeit qualitative, about the way that residents see the relationships among the consequences of work hour rules, resident‐reported contributors to patient care mistakes, and the mistakes themselves.

Our study should be interpreted in the context of several limitations. First, the use of qualitative methods did not allow us to generalize or quantify our findings. However, we purposely included 3 diverse institutions with differing responses to the work hour rules to enhance the external validity of our findings. Second, the last focus group was conducted a year after the first 3; by that point, the work hour rules had been in place for 20 months. We believe that this was both a strength and a limitation because it allowed us to gain a perspective after some of the initial growing pains were over. This time lag also allowed for analysis of the first 3 transcripts so we could revise the focus group guide and ultimately determine that thematic saturation had occurred. In addition, few of our questions were phrased to evaluate the ACGME rules; instead, they asked about links among discontinuity, scheduling, fatigue, and patient care. We therefore believe that even residents who were not in the programs before the work hour rules began were still able to knowledgeably participate in the conversation. One question directly referable to the ACGME rules asked residents to reflect on problems arising from them. This could have led residents to only reflect on the problems associated with the rules. However, in all 4 focus groups, residents commented on the positive impact of improved well‐being resulting from the work hour rules. This led us to believe the respondents felt they could voice their favorable feelings as well as their unfavorable feelings about the rules. An additional limitation is that the agreement between coders was only 45%. It is important to realize that assessing coding agreement in qualitative work is quite difficult because it is often difficult to assign a single code to a section of text. When the coders discussed a disagreement, it was almost always the case that the difference was subtle and that the coding of either investigator would made sense for that text. Finally, our results are based on the participation of 28 residents. To be certain we were not representing the opinions of only a few people, we presented iterations of this model to faculty and resident groups for their feedback. Importantly, the residents offered no substantial changes or criticisms of the model.

Limitations notwithstanding, we believe our findings have important policy implications. First, despite work hours successfully being reduced, residents perceived no decrease in the amount of work they did. This resulted in higher workload and more entropy, suggesting that residency programs may need to carefully evaluate the patient care responsibility carried by residents. Second, continued effort to educate residents to provide effective sign‐out is needed. As one participant pointed out, residency offers a unique opportunity to learn to manage discontinuity in a controlled setting. Another educational opportunity is the chance to teach physician teamwork. Participants believed that effective teamwork could ameliorate some of the discontinuity in patient care. This teamwork training should include faculty as well, although further work is needed to define how faculty can best add to patient continuity while still fostering resident autonomy. Finally, the impact of work hour rules on the professional development of residents should be further explored.

In conclusion, we have proposed a model to explain the major resident‐reported contributors to patient care mistakes with respect to resident work hour rules. Our results help to clarify the next steps needed: testing the proposed relationships between the factors and patient care mistakes and rigorously evaluating solutions that minimize the impact of these factors. Returning to the Swiss cheese framework for describing systems accidents, our results suggest that although resident work hour reductions may have sufficiently filled the hole caused by resident fatigue, other gaps may have actually widened as a result of the systems put into place to achieve compliance. Continued vigilance is therefore necessary to both identify the additional holes likely to appear and, more importantly, effectively close those holes before patient harm occurs.

FOCUS GROUP GUIDE (4TH FOCUS GROUP)

The IOM definition of a medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim.

• 1

  Please describe the call structure at each institution where you do ward months (eg, non‐ICU months).

• 2

  What are some settings at the hospitals where you work where you have seen medical errors, mistakes, or adverse outcomes?

• 3

  How do you think that other nurses influence the occurrence of medical errors, mistakes, or adverse outcomes?

   

  • a

    Clerks?

  • b

    Other ancillary staff?

  • 4

    How would you describe the responsibilities of a cross‐covering resident or intern?

  • 5

    How do you think continuity of care impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

     

    • a

      What role do sign‐outs have?

    • 6

      How do you think that fatigue impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

    • 7

      How do you think that technology such as computerized physician order entry impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

       

      • a

        Electronic medical records?

      • b

        Palm pilots?

      • c

        Is there such a thing as too much information?

      • 8

        How do you think that experience (or inexperience) impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

      • 9

        Please describe how attendings supervise you when you are on a ward team. How do you think that attending supervision impacts patient care in terms of medical errors, mistakes, or adverse outcomes?

         

        • a

          What about resident supervision of interns?

        • b

          What is the ideal role of an attending on a team?

        • c

          Can you think of a time when having attending input changed the plans or the course of a patient in a major way, good, bad, or neutral?

        • 10

          How do you think that time of day impacts patient care in terms of in terms of medical errors, mistakes, or adverse outcomes?

           

          • a

            How comfortable do you feel calling for help at night? What makes you more or less likely to do it (personal attributes of person to be called, situation, etc.)?

          • 11

            What do you think is an ideal workload? (eg, How many complex patients are typical of your hospitals?) Does that vary from the VA to St. Joe's to Froedtert? How many patients should be admitted in 1 night by an intern? How many should an intern have ongoing responsibility for? Is there such a thing as too few patients?

          • 12

            If one of your family members were to admitted to your hospital at night with a life‐threatening condition, which situation would you prefer for their care (all other things being equal): admission by night float with handoff to a new but well‐rested resident or admission by a resident who then continues to care for that family member the next day but has been awake for 24 hours, admitting and cross‐covering other patients? Why?

          • 13

            What do you think was the intent of the ACGME rules? Do you think that those goals have been accomplished? Why or why not? How have they affected you as residents? How do you think that the ACGME work hour rules have influenced patient care?

           

Cross‐cover is defined as an on‐call physician managing acute problems such as chest pain, dyspnea, and hypoxemia for patients primarily cared for by another physician. Cross‐cover problems are commonly encountered with hospitalized patients, and inappropriate evaluation and management can result in misdiagnosis. Residents in many internal medicine residency programs receive only informal instruction about how to manage cross‐cover problems, usually from senior medical residents. Unfortunately, instruction is often provided while a patient is experiencing a problem, a frequent occurrence in the chaotic circumstances of a stressful learning environment. Furthermore, the knowledge base, experience, and teaching skills of senior residents vary substantially, and typically senior residents receive no formal instruction to guide them in how or what to teach more junior residents. If formal instruction is provided to residents, it is typically through often poorly attended didactic lectures that have been shown to be an ineffective forum for acquiring skills or changing physician behavior.15

Although previous studies did find that educational interventions can improve confidence and increase knowledge about various aspects of residency training, many of these studies were not randomized,68 or they involved complex interventions requiring a significant amount of resident and teaching staff time.911 The few randomized studies that used simple educational interventions focused on outpatient education, but most of a resident's time is spent in an inpatient setting.1213

Therefore, we designed a simple, randomized educational intervention consisting of 2 formal small‐group, case‐based discussion sessions addressing 1 cross‐cover situation: a hospitalized patient with acute dyspnea. We hypothesized that the addition of small‐group sessions would improve intern knowledge about and confidence in managing acute dyspnea above that gained from a combination of informal education and formal but lecture‐based education.

METHODS

Thirty‐eight internal medicine residents in their first year of postgraduate training (interns) at the University of Michigan were approached to participate in the study. Twenty‐six interns signed informed consent forms and were randomized using a random number generator to receive either the standard education (the control group) or the standard education plus the educational intervention (the intervention group). The standard education was informal teaching by senior medical residents on the wards and a 1‐hour lecture on Approach to the Patient with Acute Dyspnea, taught by an attending physician from the Department of Pulmonary and Critical Care Medicine. The educational intervention included the standard education as well as 2 small‐group, case‐based interactive sessions on acute dyspnea management. Both sessions were developed and taught by the first author (T.M.R.), a third‐year resident in internal medicine. A senior resident taught the sessions to try to make the information more relevant and practical and to make asking questions less intimidating. The first session, which lasted 50 minutes, discussed cases of bronchospasm, pulmonary edema, and pulmonary embolism as causes of acute dyspnea. It addressed several concepts: knowing when and how quickly to evaluate a dyspneic patient, formulating a differential diagnosis, appropriately evaluating acute dyspnea, providing empiric therapy, and recognizing indications for intubation. The second small‐group session occurred approximately 1 month after the first session and lasted 30 minutes. In this session key concepts learned during the first session were reviewed, and a case of ventricular tachycardia presenting as acute dyspnea was discussed. In an effort to increase attendance, free food and drink were provided at each session, and participants were sent reminders via e‐mail and the paging system prior to each session.

All study participants completed pre‐ and postintervention surveys that assessed their knowledge of acute dyspnea management and their confidence in managing patients with this condition. The pretests were conducted just before the first small‐group session was held. The post‐tests were conducted 4 months later. Knowledge was assessed by the score on the 45‐point test, which contained both open‐ and closed‐ended questions derived from 10 case‐based items. The number of points that a question was worth varied depending on how many elements made up a correct answer. For example, one question asked, What tests (if any) do you plan to order immediately after you examine the patient? As 3 tests should have been obtained (EKG, CXR, and ABG), this item had a maximum score of 3 points. Confidence was assessed by averaging 17 items scored on a 5‐point Likert scale (from strongly agree to strongly disagree). The items measured the physician's confidence in managing various aspects of the dyspneic patient (eg, confidence in knowing when to intubate a patient, when to obtain an ABG/CXR/EKG, and when to transfer a patient to the ICU). Data were analyzed using repeated‐measures analysis of variance. Primary analysis was based on the intention‐to‐treat principle, with alpha set to .05 (2‐sided). A secondary, per‐protocol analysis was also performed. In this analysis, study participants who attended both small‐group sessions (ie, completed the entire intervention) were compared with the control group. The protocol was approved by the institutional review board at the University of Michigan Health System.

RESULTS

All participants completed the study. Overall, only 3 of the 26 interns attended the lecture on Approach to the Patient with Acute Dyspnea. Fourteen of the 16 interns assigned to the intervention group attended 1 of the 2 small‐group sessions (11 attended the first session, and 10 attended the second session). Seven interns attended both sessions. The study period was 4 months. Both the intervention and control groups reported managing a similar number of patients with acute dyspnea, both prior to the study (mean of 5.9 in the intervention group and 7.4 in the control group, P = .51) and at the end of study (mean 10.6 in the intervention group and 10.2 in the control group, P = .91). There was no significant difference in the total number of completed inpatient months (mean of 4.9 in the intervention group and 4.7 in the control group, P =. 32) or in the number of inpatient months completed prior to the start of the study (mean of 2 in the intervention group and 2.4 in the control group, P = .15).

Confidence

Subjects in both the intervention and control groups showed increased confidence over time. The mean score of the intervention group increased from 3.77 to 4.57 (a 21.2% increase) and that of the control group increased from 3.74 to 4.28 (a 14.4% increase). Although the trend over time was highly significant for both groups (P < .001), the effect of the intervention was not significant (P = .19). However, the power to detect a difference between the groups was low (0.25). In the per‐protocol analysis, there was no significant difference between the groups (P = .26; see Fig. 1).

Figure 1

Knowledge

In the primary analysis, results for knowledge were similar to those obtained for the confidence outcome. In the intervention group, the mean score increased from 35.6 to 38.3 (a 7.6% increase); in the control group, the mean increased from 36.2 to 38.2 (a 5.5% increase). Scores ranged from 31 to 42. Again, the trend for both groups was significant (P < .01), but the effect of the intervention was not significant (P = .65). The power to detect a difference between groups was again low (0.07). In the per‐protocol analysis a trend toward significance was seen, with mean scores increasing from 34.6 to 40.0, a 15.6% increase (P = .067; see Fig. 2).

Figure 2

DISCUSSION

Our randomized controlled trial found that intern confidence and knowledge about acute dyspnea management both increased significantly over time; however, no significant differences between the intervention and control groups were observed. The complete intervention was not administered to the vast majority of those in the intervention group, however, likely skewing results toward the null. As suggested by the per‐protocol analysis, there was a trend toward a significant increase in the knowledge of the interns who had received the entire intervention. This is similar to results found in a randomized study by Schroy et al., which demonstrated a significant increase in resident knowledge of colorectal cancer screening after an educational intervention that used an interactive, case‐based seminar.13

Our study had several strengths. First, we employed the most robust design to detect efficacy, a randomized controlled study design. Second, we had complete follow‐up because all participants finished the study. Finally, our intervention is easily reproducible.

Our findings should also be considered within the context of several limitations. Despite the use of a random number generator, the control and intervention groups were unequal in number, which may have affected the results, particularly with such a small sample size.

Second, the intervention did not occur until 3 months after the start of each participant's internship. The intention was to implement the intervention at the start of internship, but institutional review board approval did not occur for an additional 3 months. This late timing might have been unfortunate because interns may already have had an established management plan for acute dyspnea, making their behavior more difficult to alter, even with additional education.

Third, because we were unaware of available test instruments to assess resident knowledge of acute dyspnea in the hospitalized patient, we needed to create our own. Unfortunately, the instrument yielded only a small variance in test scores, which may have made it difficult to detect an effect on scores if present.

Fourth, attendance at each session was suboptimal, and thus the complete intervention was not administered to the vast majority of those in the intervention group. Because the first small‐group session was the main teaching session, interns who only attended the second session were exposed to just one case discussion and only a review, rather than a full formal discussion, of the material presented during the first session. Therefore, it is not known if the intervention really had no effect or if no differences were detected simply because the complete intervention was not received. The trend toward significance observed in the per‐protocol analysis suggests that compliance with the intervention may be the key to improving knowledge.

Given the small differences observed in this study, future interventions ideally should use a more sensitive testing instrument, a larger sample, and a more powerful intervention that occurs early in training. Future efforts should also be designed to improve attendance at educational interventions. In the setting of reduced resident work hours and increased demands on resident time, this will prove to be a true challenge for all educators and residency programs.

Cross‐cover is defined as an on‐call physician managing acute problems such as chest pain, dyspnea, and hypoxemia for patients primarily cared for by another physician. Cross‐cover problems are commonly encountered with hospitalized patients, and inappropriate evaluation and management can result in misdiagnosis. Residents in many internal medicine residency programs receive only informal instruction about how to manage cross‐cover problems, usually from senior medical residents. Unfortunately, instruction is often provided while a patient is experiencing a problem, a frequent occurrence in the chaotic circumstances of a stressful learning environment. Furthermore, the knowledge base, experience, and teaching skills of senior residents vary substantially, and typically senior residents receive no formal instruction to guide them in how or what to teach more junior residents. If formal instruction is provided to residents, it is typically through often poorly attended didactic lectures that have been shown to be an ineffective forum for acquiring skills or changing physician behavior.15

Although previous studies did find that educational interventions can improve confidence and increase knowledge about various aspects of residency training, many of these studies were not randomized,68 or they involved complex interventions requiring a significant amount of resident and teaching staff time.911 The few randomized studies that used simple educational interventions focused on outpatient education, but most of a resident's time is spent in an inpatient setting.1213

Therefore, we designed a simple, randomized educational intervention consisting of 2 formal small‐group, case‐based discussion sessions addressing 1 cross‐cover situation: a hospitalized patient with acute dyspnea. We hypothesized that the addition of small‐group sessions would improve intern knowledge about and confidence in managing acute dyspnea above that gained from a combination of informal education and formal but lecture‐based education.

METHODS

Thirty‐eight internal medicine residents in their first year of postgraduate training (interns) at the University of Michigan were approached to participate in the study. Twenty‐six interns signed informed consent forms and were randomized using a random number generator to receive either the standard education (the control group) or the standard education plus the educational intervention (the intervention group). The standard education was informal teaching by senior medical residents on the wards and a 1‐hour lecture on Approach to the Patient with Acute Dyspnea, taught by an attending physician from the Department of Pulmonary and Critical Care Medicine. The educational intervention included the standard education as well as 2 small‐group, case‐based interactive sessions on acute dyspnea management. Both sessions were developed and taught by the first author (T.M.R.), a third‐year resident in internal medicine. A senior resident taught the sessions to try to make the information more relevant and practical and to make asking questions less intimidating. The first session, which lasted 50 minutes, discussed cases of bronchospasm, pulmonary edema, and pulmonary embolism as causes of acute dyspnea. It addressed several concepts: knowing when and how quickly to evaluate a dyspneic patient, formulating a differential diagnosis, appropriately evaluating acute dyspnea, providing empiric therapy, and recognizing indications for intubation. The second small‐group session occurred approximately 1 month after the first session and lasted 30 minutes. In this session key concepts learned during the first session were reviewed, and a case of ventricular tachycardia presenting as acute dyspnea was discussed. In an effort to increase attendance, free food and drink were provided at each session, and participants were sent reminders via e‐mail and the paging system prior to each session.

All study participants completed pre‐ and postintervention surveys that assessed their knowledge of acute dyspnea management and their confidence in managing patients with this condition. The pretests were conducted just before the first small‐group session was held. The post‐tests were conducted 4 months later. Knowledge was assessed by the score on the 45‐point test, which contained both open‐ and closed‐ended questions derived from 10 case‐based items. The number of points that a question was worth varied depending on how many elements made up a correct answer. For example, one question asked, What tests (if any) do you plan to order immediately after you examine the patient? As 3 tests should have been obtained (EKG, CXR, and ABG), this item had a maximum score of 3 points. Confidence was assessed by averaging 17 items scored on a 5‐point Likert scale (from strongly agree to strongly disagree). The items measured the physician's confidence in managing various aspects of the dyspneic patient (eg, confidence in knowing when to intubate a patient, when to obtain an ABG/CXR/EKG, and when to transfer a patient to the ICU). Data were analyzed using repeated‐measures analysis of variance. Primary analysis was based on the intention‐to‐treat principle, with alpha set to .05 (2‐sided). A secondary, per‐protocol analysis was also performed. In this analysis, study participants who attended both small‐group sessions (ie, completed the entire intervention) were compared with the control group. The protocol was approved by the institutional review board at the University of Michigan Health System.

RESULTS

All participants completed the study. Overall, only 3 of the 26 interns attended the lecture on Approach to the Patient with Acute Dyspnea. Fourteen of the 16 interns assigned to the intervention group attended 1 of the 2 small‐group sessions (11 attended the first session, and 10 attended the second session). Seven interns attended both sessions. The study period was 4 months. Both the intervention and control groups reported managing a similar number of patients with acute dyspnea, both prior to the study (mean of 5.9 in the intervention group and 7.4 in the control group, P = .51) and at the end of study (mean 10.6 in the intervention group and 10.2 in the control group, P = .91). There was no significant difference in the total number of completed inpatient months (mean of 4.9 in the intervention group and 4.7 in the control group, P =. 32) or in the number of inpatient months completed prior to the start of the study (mean of 2 in the intervention group and 2.4 in the control group, P = .15).

Confidence

Subjects in both the intervention and control groups showed increased confidence over time. The mean score of the intervention group increased from 3.77 to 4.57 (a 21.2% increase) and that of the control group increased from 3.74 to 4.28 (a 14.4% increase). Although the trend over time was highly significant for both groups (P < .001), the effect of the intervention was not significant (P = .19). However, the power to detect a difference between the groups was low (0.25). In the per‐protocol analysis, there was no significant difference between the groups (P = .26; see Fig. 1).

Figure 1

Knowledge

In the primary analysis, results for knowledge were similar to those obtained for the confidence outcome. In the intervention group, the mean score increased from 35.6 to 38.3 (a 7.6% increase); in the control group, the mean increased from 36.2 to 38.2 (a 5.5% increase). Scores ranged from 31 to 42. Again, the trend for both groups was significant (P < .01), but the effect of the intervention was not significant (P = .65). The power to detect a difference between groups was again low (0.07). In the per‐protocol analysis a trend toward significance was seen, with mean scores increasing from 34.6 to 40.0, a 15.6% increase (P = .067; see Fig. 2).

Figure 2

DISCUSSION

Our randomized controlled trial found that intern confidence and knowledge about acute dyspnea management both increased significantly over time; however, no significant differences between the intervention and control groups were observed. The complete intervention was not administered to the vast majority of those in the intervention group, however, likely skewing results toward the null. As suggested by the per‐protocol analysis, there was a trend toward a significant increase in the knowledge of the interns who had received the entire intervention. This is similar to results found in a randomized study by Schroy et al., which demonstrated a significant increase in resident knowledge of colorectal cancer screening after an educational intervention that used an interactive, case‐based seminar.13

Our study had several strengths. First, we employed the most robust design to detect efficacy, a randomized controlled study design. Second, we had complete follow‐up because all participants finished the study. Finally, our intervention is easily reproducible.

Our findings should also be considered within the context of several limitations. Despite the use of a random number generator, the control and intervention groups were unequal in number, which may have affected the results, particularly with such a small sample size.

Second, the intervention did not occur until 3 months after the start of each participant's internship. The intention was to implement the intervention at the start of internship, but institutional review board approval did not occur for an additional 3 months. This late timing might have been unfortunate because interns may already have had an established management plan for acute dyspnea, making their behavior more difficult to alter, even with additional education.

Third, because we were unaware of available test instruments to assess resident knowledge of acute dyspnea in the hospitalized patient, we needed to create our own. Unfortunately, the instrument yielded only a small variance in test scores, which may have made it difficult to detect an effect on scores if present.

Fourth, attendance at each session was suboptimal, and thus the complete intervention was not administered to the vast majority of those in the intervention group. Because the first small‐group session was the main teaching session, interns who only attended the second session were exposed to just one case discussion and only a review, rather than a full formal discussion, of the material presented during the first session. Therefore, it is not known if the intervention really had no effect or if no differences were detected simply because the complete intervention was not received. The trend toward significance observed in the per‐protocol analysis suggests that compliance with the intervention may be the key to improving knowledge.

Given the small differences observed in this study, future interventions ideally should use a more sensitive testing instrument, a larger sample, and a more powerful intervention that occurs early in training. Future efforts should also be designed to improve attendance at educational interventions. In the setting of reduced resident work hours and increased demands on resident time, this will prove to be a true challenge for all educators and residency programs.

Cross‐cover is defined as an on‐call physician managing acute problems such as chest pain, dyspnea, and hypoxemia for patients primarily cared for by another physician. Cross‐cover problems are commonly encountered with hospitalized patients, and inappropriate evaluation and management can result in misdiagnosis. Residents in many internal medicine residency programs receive only informal instruction about how to manage cross‐cover problems, usually from senior medical residents. Unfortunately, instruction is often provided while a patient is experiencing a problem, a frequent occurrence in the chaotic circumstances of a stressful learning environment. Furthermore, the knowledge base, experience, and teaching skills of senior residents vary substantially, and typically senior residents receive no formal instruction to guide them in how or what to teach more junior residents. If formal instruction is provided to residents, it is typically through often poorly attended didactic lectures that have been shown to be an ineffective forum for acquiring skills or changing physician behavior.15

Although previous studies did find that educational interventions can improve confidence and increase knowledge about various aspects of residency training, many of these studies were not randomized,68 or they involved complex interventions requiring a significant amount of resident and teaching staff time.911 The few randomized studies that used simple educational interventions focused on outpatient education, but most of a resident's time is spent in an inpatient setting.1213

Therefore, we designed a simple, randomized educational intervention consisting of 2 formal small‐group, case‐based discussion sessions addressing 1 cross‐cover situation: a hospitalized patient with acute dyspnea. We hypothesized that the addition of small‐group sessions would improve intern knowledge about and confidence in managing acute dyspnea above that gained from a combination of informal education and formal but lecture‐based education.

METHODS

Thirty‐eight internal medicine residents in their first year of postgraduate training (interns) at the University of Michigan were approached to participate in the study. Twenty‐six interns signed informed consent forms and were randomized using a random number generator to receive either the standard education (the control group) or the standard education plus the educational intervention (the intervention group). The standard education was informal teaching by senior medical residents on the wards and a 1‐hour lecture on Approach to the Patient with Acute Dyspnea, taught by an attending physician from the Department of Pulmonary and Critical Care Medicine. The educational intervention included the standard education as well as 2 small‐group, case‐based interactive sessions on acute dyspnea management. Both sessions were developed and taught by the first author (T.M.R.), a third‐year resident in internal medicine. A senior resident taught the sessions to try to make the information more relevant and practical and to make asking questions less intimidating. The first session, which lasted 50 minutes, discussed cases of bronchospasm, pulmonary edema, and pulmonary embolism as causes of acute dyspnea. It addressed several concepts: knowing when and how quickly to evaluate a dyspneic patient, formulating a differential diagnosis, appropriately evaluating acute dyspnea, providing empiric therapy, and recognizing indications for intubation. The second small‐group session occurred approximately 1 month after the first session and lasted 30 minutes. In this session key concepts learned during the first session were reviewed, and a case of ventricular tachycardia presenting as acute dyspnea was discussed. In an effort to increase attendance, free food and drink were provided at each session, and participants were sent reminders via e‐mail and the paging system prior to each session.

All study participants completed pre‐ and postintervention surveys that assessed their knowledge of acute dyspnea management and their confidence in managing patients with this condition. The pretests were conducted just before the first small‐group session was held. The post‐tests were conducted 4 months later. Knowledge was assessed by the score on the 45‐point test, which contained both open‐ and closed‐ended questions derived from 10 case‐based items. The number of points that a question was worth varied depending on how many elements made up a correct answer. For example, one question asked, What tests (if any) do you plan to order immediately after you examine the patient? As 3 tests should have been obtained (EKG, CXR, and ABG), this item had a maximum score of 3 points. Confidence was assessed by averaging 17 items scored on a 5‐point Likert scale (from strongly agree to strongly disagree). The items measured the physician's confidence in managing various aspects of the dyspneic patient (eg, confidence in knowing when to intubate a patient, when to obtain an ABG/CXR/EKG, and when to transfer a patient to the ICU). Data were analyzed using repeated‐measures analysis of variance. Primary analysis was based on the intention‐to‐treat principle, with alpha set to .05 (2‐sided). A secondary, per‐protocol analysis was also performed. In this analysis, study participants who attended both small‐group sessions (ie, completed the entire intervention) were compared with the control group. The protocol was approved by the institutional review board at the University of Michigan Health System.

RESULTS

All participants completed the study. Overall, only 3 of the 26 interns attended the lecture on Approach to the Patient with Acute Dyspnea. Fourteen of the 16 interns assigned to the intervention group attended 1 of the 2 small‐group sessions (11 attended the first session, and 10 attended the second session). Seven interns attended both sessions. The study period was 4 months. Both the intervention and control groups reported managing a similar number of patients with acute dyspnea, both prior to the study (mean of 5.9 in the intervention group and 7.4 in the control group, P = .51) and at the end of study (mean 10.6 in the intervention group and 10.2 in the control group, P = .91). There was no significant difference in the total number of completed inpatient months (mean of 4.9 in the intervention group and 4.7 in the control group, P =. 32) or in the number of inpatient months completed prior to the start of the study (mean of 2 in the intervention group and 2.4 in the control group, P = .15).

Confidence

Subjects in both the intervention and control groups showed increased confidence over time. The mean score of the intervention group increased from 3.77 to 4.57 (a 21.2% increase) and that of the control group increased from 3.74 to 4.28 (a 14.4% increase). Although the trend over time was highly significant for both groups (P < .001), the effect of the intervention was not significant (P = .19). However, the power to detect a difference between the groups was low (0.25). In the per‐protocol analysis, there was no significant difference between the groups (P = .26; see Fig. 1).

Figure 1

Knowledge

In the primary analysis, results for knowledge were similar to those obtained for the confidence outcome. In the intervention group, the mean score increased from 35.6 to 38.3 (a 7.6% increase); in the control group, the mean increased from 36.2 to 38.2 (a 5.5% increase). Scores ranged from 31 to 42. Again, the trend for both groups was significant (P < .01), but the effect of the intervention was not significant (P = .65). The power to detect a difference between groups was again low (0.07). In the per‐protocol analysis a trend toward significance was seen, with mean scores increasing from 34.6 to 40.0, a 15.6% increase (P = .067; see Fig. 2).

Figure 2

DISCUSSION

Our randomized controlled trial found that intern confidence and knowledge about acute dyspnea management both increased significantly over time; however, no significant differences between the intervention and control groups were observed. The complete intervention was not administered to the vast majority of those in the intervention group, however, likely skewing results toward the null. As suggested by the per‐protocol analysis, there was a trend toward a significant increase in the knowledge of the interns who had received the entire intervention. This is similar to results found in a randomized study by Schroy et al., which demonstrated a significant increase in resident knowledge of colorectal cancer screening after an educational intervention that used an interactive, case‐based seminar.13

Our study had several strengths. First, we employed the most robust design to detect efficacy, a randomized controlled study design. Second, we had complete follow‐up because all participants finished the study. Finally, our intervention is easily reproducible.

Our findings should also be considered within the context of several limitations. Despite the use of a random number generator, the control and intervention groups were unequal in number, which may have affected the results, particularly with such a small sample size.

Second, the intervention did not occur until 3 months after the start of each participant's internship. The intention was to implement the intervention at the start of internship, but institutional review board approval did not occur for an additional 3 months. This late timing might have been unfortunate because interns may already have had an established management plan for acute dyspnea, making their behavior more difficult to alter, even with additional education.

Third, because we were unaware of available test instruments to assess resident knowledge of acute dyspnea in the hospitalized patient, we needed to create our own. Unfortunately, the instrument yielded only a small variance in test scores, which may have made it difficult to detect an effect on scores if present.

Fourth, attendance at each session was suboptimal, and thus the complete intervention was not administered to the vast majority of those in the intervention group. Because the first small‐group session was the main teaching session, interns who only attended the second session were exposed to just one case discussion and only a review, rather than a full formal discussion, of the material presented during the first session. Therefore, it is not known if the intervention really had no effect or if no differences were detected simply because the complete intervention was not received. The trend toward significance observed in the per‐protocol analysis suggests that compliance with the intervention may be the key to improving knowledge.

Given the small differences observed in this study, future interventions ideally should use a more sensitive testing instrument, a larger sample, and a more powerful intervention that occurs early in training. Future efforts should also be designed to improve attendance at educational interventions. In the setting of reduced resident work hours and increased demands on resident time, this will prove to be a true challenge for all educators and residency programs.