Design and Setting

This was a prospective, single‐center, randomized, controlled trial conducted at a community hospital in Connecticut.

Subjects

Male and female hospitalized patients at least 18 years old in the general medicine in‐patient services at a community hospital in Connecticut were assessed for eligibility to participate in this study if they had been put in restraints by the health care team independent of the study for one of these acute conditions: (1) delirium from any cause, including drug or alcohol withdrawal, or other medical conditions resulting in acute delirium; (2) confusional state from any cause; (3) agitation and disruptive behavior requiring restraints; (4) psychosis, hallucinations, or delusions requiring acute intervention (such as medication, restraints, or sitter); or (5) suicidality. Once in restraints, patients were screened for eligibility to participate in this study.

Exclusion criteria included: (1) need for acute respiratory or hemodynamic support or cardiac or septic shock; (2) terminal illness; (3) documented history of claustrophobia; (4) refusal by the family to give consent; (5) hospital stay < 24 hours; and (6) need for intravenous vasopressors, intubation, or ventilatory support. We also excluded patients who had been in restraints for more than 48 hours prior to potential study enrollment. Because safe enclosure would be a redundant system of restraint for patients requiring more than 1 limb in restraints, those patients were also excluded from the study. Figure 1 shows participation flow.

Figure 1

A number of screened patients were excluded for not being appropriate candidates for the safe enclosure. Of these, 20% required more than 1 limb in restraints, 13% required restraint not for agitation but for IV or catheter protection only, 10% were in critical care or on ventilators, and 13% were not appropriate for various other reasons including claustrophobia. The remaining excluded patients may have been eligible but either were preparing for discharge on screening (26%), were in restraints for more than 48 hours on screening (11%), stayed in the hospital less than 24 hours (4%), or had previously been a study participant (3%).

A stratified permuted block randomization was used to control for age (65 vs. >65 years) and sex (male vs. female) to ensure equal representation in both study arms. The study was approved by the institutional review board of the study site, and written informed consent was obtained by the study coordinator from patients' families. Because eligible patients suffered from acute delirium or agitation, most were not sufficiently cognizant to participate in the consent process. As a result, consent was largely obtained from patients' family members. Although the intent of this trial was to recruit 60 patients over an 18‐month period, the study was closed at 49 after 2 years because of slow recruitment and a lack of remaining funds.

Intervention

The safe enclosure, also known as a net bed or safety net, is an alternative to standard restraints. It consists of: (1) a metal frame that sits on the floor completely enclosing a standard hospital bed and (2) a nylon net canopy that encloses the patient and the mattress. We used the SOMA Safe Enclosure (Vivax Medical Corporation, Torrington, CT, www.vivaxmedical.com) in the intervention group and standard hospital restraints (Posey vests, 2‐point or 4‐point soft or hard restraints) in the control group.

Procedures

Patients were enrolled in this study from April 2003 to February 2005. Once a patient had been placed in restraints by a physician, the nurse in charge alerted the study investigators by beeper of a potential subject, who was then screened based on the above eligibility criteria. We also actively screened restraint log sheets maintained by the nursing staff on most weekdays to monitor new patients who may have been put on restraints. Subjects were randomized to remain in standard restraints or be transferred to the safe enclosure. The randomization scheme was generated using software available at www.randomization.com, and separate, opaque envelopes containing patient assignments were opened sequentially as patients were enrolled. Blinding was not possible because of the visible nature of the intervention. For all patients, standard hospital policies and procedures regarding restraint use were followed. Discontinuation of restraints in both groups was at the discretion of the medical staff, independent of the study. Subjects in the safe enclosure group could have 1 limb in an additional restraint when needed to protect an IV line.

Modification of the use of restraints is common in hospital settings, as patient needs fluctuate. In the 2 study groups, restraint crossover did occur at the discretion of the attending physician or nurse. Four patients in the safe enclosure group required either additional restraints or had the safe enclosure removed and alternate restraints applied. Similarly, 4 patients in the standard restraint group were either given the safe enclosure or had additional standard restraints applied. Under the principle of intent to treat, patients remained in their original randomized group for the purpose of analyses. This approach provided the most conservative analyses by keeping the sickest patients in the intervention group, thus guarding against type I error.

Measurements

Baseline data obtained at enrollment included: (1) demographic information, (2) clinical information, and (3) restraint information (ie, time of restraint application, clinical indication for use, type of restraint used, ordering physician, and alternative treatment tried before decision to start restraints).

Primary outcomes included: (1) perception survey scores of family members, physicians, and nurses regarding patient comfort, acceptability, and safety of the restraint device; and (2) patient agitation scores. A preplanned subgroup analysis separated nurses into 2 categories (primary and secondary). The admitting nurse was designated the primary nurse; all other nurses were considered secondary nurses. This analysis was performed to examine any differences in perception resulting from a nurse's level of patient involvement.

Family and provider perceptions were assessed with a self‐administered survey containing 11 items each measured on a 10‐point scale (from 1 = viewed negatively to 10 = viewed positively; maximum score 110 points). Surveyed physicians, nurses, and relatives were asked to rate: (1) patient comfort, (2) accessibility to patient, (3) ease of communication with patient, (4) how calm patient was, (5) perceived safety of patient, (6) patient's feeding convenience, (7) ease of bedpan use, (8) impact on recovery time; (9) how humane and ethical the restraint was, (10) recommendation for use on other patients, and (11) how demanding or difficult caring for the patient was.

Agitation was measured using 2 distinct methods: the Alcohol Withdrawal Assessment Form (AWAF)5 and the Agitated Behavior Scale (ABS).6, 7 Both techniques have been widely used to assess delirium of hospitalized patients. The AWAF measures agitation by analyzing key physiologic indicators such as blood pressure and heart rate on a 0‐ to 22‐point scale. The ABS is a 14‐item scale that measures specific behaviors related to agitation (eg, distractibility, uncooperativeness, and restlessness). Each behavior is rated on a 4‐point Likert scale (0‐3); total score ranges from 0 to 42. Each scale was completed once per 8‐hour shift by the nurse on duty.

Secondary outcomes consisted of total length of stay, duration of restraint use, time from application of restraints until time of discharge, and time from admission until time of application of restraints. Length of stay was calculated as the number of days from the time the patient was admitted to the time the patient was discharged. Time from admission until time of restraint application, duration of restraint use, and time from restraint application until discharge were assessed in minutes. These measurements were based on written restraint order forms and nursing progress reports. Hospital protocol regarding restraint requires hospital staff members to document the application, removal, and adjustment of restraints.

Additional outcomes measured included total amount of medications used to treat agitation and number of injuries incurred. Total amount of medication administered was determined with an equivalence system for different drugs used to treat agitation or delirium.811 Medications were separated into 4 groups: antianxiety medications, antidepressants, antipsychotics, and opioid analgesics. Total amount included both regularly administered and as‐needed dosages of medication. We identified injuries through reports of a subject's primary nurse and by review of medical records.

Data Analyses

Sample size was calculated using a 2‐sample t test formula based on the primary outcome. The study was designed to detect an absolute difference in points of 10% (total absolute score difference of 11 per survey or a total difference of 33). The 2‐sided alpha was initially set at 0.05 and the power at 80%, with an estimated standard deviation of 20. The alpha level was Bonferroni‐adjusted for up to 6 additional comparisons, with each significance level of 0.0071 (z = 2.70).

To assess differences in patient characteristics between the standard restraints and safe enclosure groups, we used the Student t test for continuous variables and Fisher's 2‐sided exact test for categorical variables. Differences in family and staff perceptions of the restraint mechanisms were measured using the Student t test with Satterthwaite's method for calculating variance. However, to account for questions marked not applicable by the responder, weighted scores, defined as the total score divided by the percentage of questions answered, were calculated.

Differences in agitation scores (ABS and AWAF) were analyzed using 2 strategies. First, the ABS and AWAF scores 24 hours after study enrollment were compared across groups using the Student t test with Satterthwaite's method for calculating variance. Then, separate comparisons of the ABS and AWAF scores 48 and 72 hours after enrollment were conducted using the Student t test with a pooled variance. Next, 2 longitudinal analyses were performed using a mixed‐effects (fixed and random) model. These analyses modeled change in the ABS or AWAF scores over (1) the first 3 days and (2) the first 6 days of hospitalization as a function of being restrained with the safe enclosure or being restrained with the hospital's standard restraint systems. For these comparisons, the model included not only the main effects of type of restraint and time, but also the interaction between type of restraint and time and the covariates sex, age, and initial ABS or AWAF score. For these models, a backward elimination procedure was undertaken using a significance level of = 0.05 in order to determine the most parsimonious model.

To determine if the total length of subject stay in the hospital was different between groups, the Student t test was used with Satterthwaite's method for calculating variance. Differences in time from admission until time of restraint application, duration of restraint use, and time from application of restraints until time of discharge were analyzed with the Student t test with pooled variances.

To compare the amount of medication used, equivalent dosage conversions were used for each of the 4 medication categories (antianxiety medications, antidepressants, antipsychotics, and opioid analgesics). To determine if the amounts of these 4 categories of medications differed between groups, the Student t test was used. Last, to determine if there was a difference in the number of patient injuries between groups, Fisher's 2‐sided exact test was used.

Study Population

Of the 49 subjects enrolled in the study, 20 were randomized to the safe enclosure and 29 to standard restraints. This imbalance was likely a result of the premature termination of the study, which in turn was a result of slow recruitment. Table 1 shows selected baseline characteristics of the enrolled subjects. There were no significant differences between the 2 groups in sex, age, patient diagnoses, reason for restraint, or type of medication. However, the subjects randomized to the safe enclosure were less likely to have hypertension than those randomized to standard restraints (36.8% vs. 72.4%, P = .019).

Primary Outcomes

The rates of response to the perception survey were: relatives/next of kin, 90%; physicians, 90%; primary nurses, 100%; and secondary nurses, 78%. Family members and physicians viewed the safe enclosure significantly more positively than they viewed standard restraints (P < .0001 and P < .0001, respectively; Table 2). There was a trend toward more positive perceptions of the safe enclosure among nurses; however, this trend did not achieve statistical significance (P = .0836). The subgroup analysis of nurses (primary vs. secondary) revealed that secondary nurses viewed the safe enclosure more positively (P = .023). Primary nurses tended to view the safe enclosure more positively than the standard restraints, but the association was not significant (P = .1313).

There were no statistically significant differences between the 2 randomized groups in ABS or AWAF scores 24, 48, or 72 hours after restraint application (Table 2). In addition, there were no statistically significant differences during the study between the groups in the rates of change in ABS or AWAF score . This was the case when looking at the first 3 days of hospitalization as well as the first 6 days (data not shown). All results were also calculated after adjusting for length of stay; this covariate did not affect any of the results.

Table 3 details the results for each perception survey question. Perceived comfort, calmness, and safety of patients were rated higher in the safe enclosure group by physicians, relatives, and all nurses. With the exception of perceived accessibility to patients, relatives rated the safe enclosure higher than standard restraints on all other perception measures. Table 4 illustrates the differences in the responses of primary and secondary nurse to each perception survey question. Primary and secondary nurses viewed the safety of the safe enclosure significantly more positively than they did the standard restraints.

Secondary Outcomes

There was a trend toward shorter total length of stay, time from admission until restraint application, duration of restraint use, and time from restraint application until discharge among subjects restrained by the safe enclosure compared with those restrained with standard restraints. However, these unadjusted differences were not statistically significant. We examined secondary outcomes after adjusting for 2 covariates, age and sex. Age but not sex affected the results. We found that subjects in the intervention group younger than 80 years of age had a shorter length of stay for 2 of the 4 related outcomes: time of admittance to time of discharge (P = .0199) and time of restraint to time of discharge (P = .0274). Time of admission to time of restraint application and duration of restraint did not differ between groups. The former outcome was not expected to differ between groups.

Additional Outcomes

There were no differences between groups in the amounts of 3 of the 4 types of medications used to treat agitation or delirium (ie, antianxiety medications, antipsychotic medications, opioid analgesic medications). The proportion of patients on these medications did not differ by group (P = .59). Only 5% of patients in standard restraints were on antidepressants, and about 5% were on opioids in each group. There was only 1 minor patient injury recorded during the study. This minor abrasion was to a patient assigned to the standard restraint group. No injuries were reported in the safe enclosure group.

Design and Setting

This was a prospective, single‐center, randomized, controlled trial conducted at a community hospital in Connecticut.

Subjects

Male and female hospitalized patients at least 18 years old in the general medicine in‐patient services at a community hospital in Connecticut were assessed for eligibility to participate in this study if they had been put in restraints by the health care team independent of the study for one of these acute conditions: (1) delirium from any cause, including drug or alcohol withdrawal, or other medical conditions resulting in acute delirium; (2) confusional state from any cause; (3) agitation and disruptive behavior requiring restraints; (4) psychosis, hallucinations, or delusions requiring acute intervention (such as medication, restraints, or sitter); or (5) suicidality. Once in restraints, patients were screened for eligibility to participate in this study.

Exclusion criteria included: (1) need for acute respiratory or hemodynamic support or cardiac or septic shock; (2) terminal illness; (3) documented history of claustrophobia; (4) refusal by the family to give consent; (5) hospital stay < 24 hours; and (6) need for intravenous vasopressors, intubation, or ventilatory support. We also excluded patients who had been in restraints for more than 48 hours prior to potential study enrollment. Because safe enclosure would be a redundant system of restraint for patients requiring more than 1 limb in restraints, those patients were also excluded from the study. Figure 1 shows participation flow.

Figure 1

A number of screened patients were excluded for not being appropriate candidates for the safe enclosure. Of these, 20% required more than 1 limb in restraints, 13% required restraint not for agitation but for IV or catheter protection only, 10% were in critical care or on ventilators, and 13% were not appropriate for various other reasons including claustrophobia. The remaining excluded patients may have been eligible but either were preparing for discharge on screening (26%), were in restraints for more than 48 hours on screening (11%), stayed in the hospital less than 24 hours (4%), or had previously been a study participant (3%).

A stratified permuted block randomization was used to control for age (65 vs. >65 years) and sex (male vs. female) to ensure equal representation in both study arms. The study was approved by the institutional review board of the study site, and written informed consent was obtained by the study coordinator from patients' families. Because eligible patients suffered from acute delirium or agitation, most were not sufficiently cognizant to participate in the consent process. As a result, consent was largely obtained from patients' family members. Although the intent of this trial was to recruit 60 patients over an 18‐month period, the study was closed at 49 after 2 years because of slow recruitment and a lack of remaining funds.

Intervention

The safe enclosure, also known as a net bed or safety net, is an alternative to standard restraints. It consists of: (1) a metal frame that sits on the floor completely enclosing a standard hospital bed and (2) a nylon net canopy that encloses the patient and the mattress. We used the SOMA Safe Enclosure (Vivax Medical Corporation, Torrington, CT, www.vivaxmedical.com) in the intervention group and standard hospital restraints (Posey vests, 2‐point or 4‐point soft or hard restraints) in the control group.

Procedures

Patients were enrolled in this study from April 2003 to February 2005. Once a patient had been placed in restraints by a physician, the nurse in charge alerted the study investigators by beeper of a potential subject, who was then screened based on the above eligibility criteria. We also actively screened restraint log sheets maintained by the nursing staff on most weekdays to monitor new patients who may have been put on restraints. Subjects were randomized to remain in standard restraints or be transferred to the safe enclosure. The randomization scheme was generated using software available at www.randomization.com, and separate, opaque envelopes containing patient assignments were opened sequentially as patients were enrolled. Blinding was not possible because of the visible nature of the intervention. For all patients, standard hospital policies and procedures regarding restraint use were followed. Discontinuation of restraints in both groups was at the discretion of the medical staff, independent of the study. Subjects in the safe enclosure group could have 1 limb in an additional restraint when needed to protect an IV line.

Modification of the use of restraints is common in hospital settings, as patient needs fluctuate. In the 2 study groups, restraint crossover did occur at the discretion of the attending physician or nurse. Four patients in the safe enclosure group required either additional restraints or had the safe enclosure removed and alternate restraints applied. Similarly, 4 patients in the standard restraint group were either given the safe enclosure or had additional standard restraints applied. Under the principle of intent to treat, patients remained in their original randomized group for the purpose of analyses. This approach provided the most conservative analyses by keeping the sickest patients in the intervention group, thus guarding against type I error.

Measurements

Baseline data obtained at enrollment included: (1) demographic information, (2) clinical information, and (3) restraint information (ie, time of restraint application, clinical indication for use, type of restraint used, ordering physician, and alternative treatment tried before decision to start restraints).

Primary outcomes included: (1) perception survey scores of family members, physicians, and nurses regarding patient comfort, acceptability, and safety of the restraint device; and (2) patient agitation scores. A preplanned subgroup analysis separated nurses into 2 categories (primary and secondary). The admitting nurse was designated the primary nurse; all other nurses were considered secondary nurses. This analysis was performed to examine any differences in perception resulting from a nurse's level of patient involvement.

Family and provider perceptions were assessed with a self‐administered survey containing 11 items each measured on a 10‐point scale (from 1 = viewed negatively to 10 = viewed positively; maximum score 110 points). Surveyed physicians, nurses, and relatives were asked to rate: (1) patient comfort, (2) accessibility to patient, (3) ease of communication with patient, (4) how calm patient was, (5) perceived safety of patient, (6) patient's feeding convenience, (7) ease of bedpan use, (8) impact on recovery time; (9) how humane and ethical the restraint was, (10) recommendation for use on other patients, and (11) how demanding or difficult caring for the patient was.

Agitation was measured using 2 distinct methods: the Alcohol Withdrawal Assessment Form (AWAF)5 and the Agitated Behavior Scale (ABS).6, 7 Both techniques have been widely used to assess delirium of hospitalized patients. The AWAF measures agitation by analyzing key physiologic indicators such as blood pressure and heart rate on a 0‐ to 22‐point scale. The ABS is a 14‐item scale that measures specific behaviors related to agitation (eg, distractibility, uncooperativeness, and restlessness). Each behavior is rated on a 4‐point Likert scale (0‐3); total score ranges from 0 to 42. Each scale was completed once per 8‐hour shift by the nurse on duty.

Secondary outcomes consisted of total length of stay, duration of restraint use, time from application of restraints until time of discharge, and time from admission until time of application of restraints. Length of stay was calculated as the number of days from the time the patient was admitted to the time the patient was discharged. Time from admission until time of restraint application, duration of restraint use, and time from restraint application until discharge were assessed in minutes. These measurements were based on written restraint order forms and nursing progress reports. Hospital protocol regarding restraint requires hospital staff members to document the application, removal, and adjustment of restraints.

Additional outcomes measured included total amount of medications used to treat agitation and number of injuries incurred. Total amount of medication administered was determined with an equivalence system for different drugs used to treat agitation or delirium.811 Medications were separated into 4 groups: antianxiety medications, antidepressants, antipsychotics, and opioid analgesics. Total amount included both regularly administered and as‐needed dosages of medication. We identified injuries through reports of a subject's primary nurse and by review of medical records.

Data Analyses

Sample size was calculated using a 2‐sample t test formula based on the primary outcome. The study was designed to detect an absolute difference in points of 10% (total absolute score difference of 11 per survey or a total difference of 33). The 2‐sided alpha was initially set at 0.05 and the power at 80%, with an estimated standard deviation of 20. The alpha level was Bonferroni‐adjusted for up to 6 additional comparisons, with each significance level of 0.0071 (z = 2.70).

To assess differences in patient characteristics between the standard restraints and safe enclosure groups, we used the Student t test for continuous variables and Fisher's 2‐sided exact test for categorical variables. Differences in family and staff perceptions of the restraint mechanisms were measured using the Student t test with Satterthwaite's method for calculating variance. However, to account for questions marked not applicable by the responder, weighted scores, defined as the total score divided by the percentage of questions answered, were calculated.

Differences in agitation scores (ABS and AWAF) were analyzed using 2 strategies. First, the ABS and AWAF scores 24 hours after study enrollment were compared across groups using the Student t test with Satterthwaite's method for calculating variance. Then, separate comparisons of the ABS and AWAF scores 48 and 72 hours after enrollment were conducted using the Student t test with a pooled variance. Next, 2 longitudinal analyses were performed using a mixed‐effects (fixed and random) model. These analyses modeled change in the ABS or AWAF scores over (1) the first 3 days and (2) the first 6 days of hospitalization as a function of being restrained with the safe enclosure or being restrained with the hospital's standard restraint systems. For these comparisons, the model included not only the main effects of type of restraint and time, but also the interaction between type of restraint and time and the covariates sex, age, and initial ABS or AWAF score. For these models, a backward elimination procedure was undertaken using a significance level of = 0.05 in order to determine the most parsimonious model.

To determine if the total length of subject stay in the hospital was different between groups, the Student t test was used with Satterthwaite's method for calculating variance. Differences in time from admission until time of restraint application, duration of restraint use, and time from application of restraints until time of discharge were analyzed with the Student t test with pooled variances.

To compare the amount of medication used, equivalent dosage conversions were used for each of the 4 medication categories (antianxiety medications, antidepressants, antipsychotics, and opioid analgesics). To determine if the amounts of these 4 categories of medications differed between groups, the Student t test was used. Last, to determine if there was a difference in the number of patient injuries between groups, Fisher's 2‐sided exact test was used.

Study Population

Of the 49 subjects enrolled in the study, 20 were randomized to the safe enclosure and 29 to standard restraints. This imbalance was likely a result of the premature termination of the study, which in turn was a result of slow recruitment. Table 1 shows selected baseline characteristics of the enrolled subjects. There were no significant differences between the 2 groups in sex, age, patient diagnoses, reason for restraint, or type of medication. However, the subjects randomized to the safe enclosure were less likely to have hypertension than those randomized to standard restraints (36.8% vs. 72.4%, P = .019).

Primary Outcomes

The rates of response to the perception survey were: relatives/next of kin, 90%; physicians, 90%; primary nurses, 100%; and secondary nurses, 78%. Family members and physicians viewed the safe enclosure significantly more positively than they viewed standard restraints (P < .0001 and P < .0001, respectively; Table 2). There was a trend toward more positive perceptions of the safe enclosure among nurses; however, this trend did not achieve statistical significance (P = .0836). The subgroup analysis of nurses (primary vs. secondary) revealed that secondary nurses viewed the safe enclosure more positively (P = .023). Primary nurses tended to view the safe enclosure more positively than the standard restraints, but the association was not significant (P = .1313).

There were no statistically significant differences between the 2 randomized groups in ABS or AWAF scores 24, 48, or 72 hours after restraint application (Table 2). In addition, there were no statistically significant differences during the study between the groups in the rates of change in ABS or AWAF score . This was the case when looking at the first 3 days of hospitalization as well as the first 6 days (data not shown). All results were also calculated after adjusting for length of stay; this covariate did not affect any of the results.

Table 3 details the results for each perception survey question. Perceived comfort, calmness, and safety of patients were rated higher in the safe enclosure group by physicians, relatives, and all nurses. With the exception of perceived accessibility to patients, relatives rated the safe enclosure higher than standard restraints on all other perception measures. Table 4 illustrates the differences in the responses of primary and secondary nurse to each perception survey question. Primary and secondary nurses viewed the safety of the safe enclosure significantly more positively than they did the standard restraints.

Secondary Outcomes

There was a trend toward shorter total length of stay, time from admission until restraint application, duration of restraint use, and time from restraint application until discharge among subjects restrained by the safe enclosure compared with those restrained with standard restraints. However, these unadjusted differences were not statistically significant. We examined secondary outcomes after adjusting for 2 covariates, age and sex. Age but not sex affected the results. We found that subjects in the intervention group younger than 80 years of age had a shorter length of stay for 2 of the 4 related outcomes: time of admittance to time of discharge (P = .0199) and time of restraint to time of discharge (P = .0274). Time of admission to time of restraint application and duration of restraint did not differ between groups. The former outcome was not expected to differ between groups.

Additional Outcomes

There were no differences between groups in the amounts of 3 of the 4 types of medications used to treat agitation or delirium (ie, antianxiety medications, antipsychotic medications, opioid analgesic medications). The proportion of patients on these medications did not differ by group (P = .59). Only 5% of patients in standard restraints were on antidepressants, and about 5% were on opioids in each group. There was only 1 minor patient injury recorded during the study. This minor abrasion was to a patient assigned to the standard restraint group. No injuries were reported in the safe enclosure group.

Design and Setting

This was a prospective, single‐center, randomized, controlled trial conducted at a community hospital in Connecticut.

Subjects

Male and female hospitalized patients at least 18 years old in the general medicine in‐patient services at a community hospital in Connecticut were assessed for eligibility to participate in this study if they had been put in restraints by the health care team independent of the study for one of these acute conditions: (1) delirium from any cause, including drug or alcohol withdrawal, or other medical conditions resulting in acute delirium; (2) confusional state from any cause; (3) agitation and disruptive behavior requiring restraints; (4) psychosis, hallucinations, or delusions requiring acute intervention (such as medication, restraints, or sitter); or (5) suicidality. Once in restraints, patients were screened for eligibility to participate in this study.

Exclusion criteria included: (1) need for acute respiratory or hemodynamic support or cardiac or septic shock; (2) terminal illness; (3) documented history of claustrophobia; (4) refusal by the family to give consent; (5) hospital stay < 24 hours; and (6) need for intravenous vasopressors, intubation, or ventilatory support. We also excluded patients who had been in restraints for more than 48 hours prior to potential study enrollment. Because safe enclosure would be a redundant system of restraint for patients requiring more than 1 limb in restraints, those patients were also excluded from the study. Figure 1 shows participation flow.

Figure 1

A number of screened patients were excluded for not being appropriate candidates for the safe enclosure. Of these, 20% required more than 1 limb in restraints, 13% required restraint not for agitation but for IV or catheter protection only, 10% were in critical care or on ventilators, and 13% were not appropriate for various other reasons including claustrophobia. The remaining excluded patients may have been eligible but either were preparing for discharge on screening (26%), were in restraints for more than 48 hours on screening (11%), stayed in the hospital less than 24 hours (4%), or had previously been a study participant (3%).

A stratified permuted block randomization was used to control for age (65 vs. >65 years) and sex (male vs. female) to ensure equal representation in both study arms. The study was approved by the institutional review board of the study site, and written informed consent was obtained by the study coordinator from patients' families. Because eligible patients suffered from acute delirium or agitation, most were not sufficiently cognizant to participate in the consent process. As a result, consent was largely obtained from patients' family members. Although the intent of this trial was to recruit 60 patients over an 18‐month period, the study was closed at 49 after 2 years because of slow recruitment and a lack of remaining funds.

Intervention

The safe enclosure, also known as a net bed or safety net, is an alternative to standard restraints. It consists of: (1) a metal frame that sits on the floor completely enclosing a standard hospital bed and (2) a nylon net canopy that encloses the patient and the mattress. We used the SOMA Safe Enclosure (Vivax Medical Corporation, Torrington, CT, www.vivaxmedical.com) in the intervention group and standard hospital restraints (Posey vests, 2‐point or 4‐point soft or hard restraints) in the control group.

Procedures

Patients were enrolled in this study from April 2003 to February 2005. Once a patient had been placed in restraints by a physician, the nurse in charge alerted the study investigators by beeper of a potential subject, who was then screened based on the above eligibility criteria. We also actively screened restraint log sheets maintained by the nursing staff on most weekdays to monitor new patients who may have been put on restraints. Subjects were randomized to remain in standard restraints or be transferred to the safe enclosure. The randomization scheme was generated using software available at www.randomization.com, and separate, opaque envelopes containing patient assignments were opened sequentially as patients were enrolled. Blinding was not possible because of the visible nature of the intervention. For all patients, standard hospital policies and procedures regarding restraint use were followed. Discontinuation of restraints in both groups was at the discretion of the medical staff, independent of the study. Subjects in the safe enclosure group could have 1 limb in an additional restraint when needed to protect an IV line.

Modification of the use of restraints is common in hospital settings, as patient needs fluctuate. In the 2 study groups, restraint crossover did occur at the discretion of the attending physician or nurse. Four patients in the safe enclosure group required either additional restraints or had the safe enclosure removed and alternate restraints applied. Similarly, 4 patients in the standard restraint group were either given the safe enclosure or had additional standard restraints applied. Under the principle of intent to treat, patients remained in their original randomized group for the purpose of analyses. This approach provided the most conservative analyses by keeping the sickest patients in the intervention group, thus guarding against type I error.

Measurements

Baseline data obtained at enrollment included: (1) demographic information, (2) clinical information, and (3) restraint information (ie, time of restraint application, clinical indication for use, type of restraint used, ordering physician, and alternative treatment tried before decision to start restraints).

Primary outcomes included: (1) perception survey scores of family members, physicians, and nurses regarding patient comfort, acceptability, and safety of the restraint device; and (2) patient agitation scores. A preplanned subgroup analysis separated nurses into 2 categories (primary and secondary). The admitting nurse was designated the primary nurse; all other nurses were considered secondary nurses. This analysis was performed to examine any differences in perception resulting from a nurse's level of patient involvement.

Family and provider perceptions were assessed with a self‐administered survey containing 11 items each measured on a 10‐point scale (from 1 = viewed negatively to 10 = viewed positively; maximum score 110 points). Surveyed physicians, nurses, and relatives were asked to rate: (1) patient comfort, (2) accessibility to patient, (3) ease of communication with patient, (4) how calm patient was, (5) perceived safety of patient, (6) patient's feeding convenience, (7) ease of bedpan use, (8) impact on recovery time; (9) how humane and ethical the restraint was, (10) recommendation for use on other patients, and (11) how demanding or difficult caring for the patient was.

Agitation was measured using 2 distinct methods: the Alcohol Withdrawal Assessment Form (AWAF)5 and the Agitated Behavior Scale (ABS).6, 7 Both techniques have been widely used to assess delirium of hospitalized patients. The AWAF measures agitation by analyzing key physiologic indicators such as blood pressure and heart rate on a 0‐ to 22‐point scale. The ABS is a 14‐item scale that measures specific behaviors related to agitation (eg, distractibility, uncooperativeness, and restlessness). Each behavior is rated on a 4‐point Likert scale (0‐3); total score ranges from 0 to 42. Each scale was completed once per 8‐hour shift by the nurse on duty.

Secondary outcomes consisted of total length of stay, duration of restraint use, time from application of restraints until time of discharge, and time from admission until time of application of restraints. Length of stay was calculated as the number of days from the time the patient was admitted to the time the patient was discharged. Time from admission until time of restraint application, duration of restraint use, and time from restraint application until discharge were assessed in minutes. These measurements were based on written restraint order forms and nursing progress reports. Hospital protocol regarding restraint requires hospital staff members to document the application, removal, and adjustment of restraints.

Additional outcomes measured included total amount of medications used to treat agitation and number of injuries incurred. Total amount of medication administered was determined with an equivalence system for different drugs used to treat agitation or delirium.811 Medications were separated into 4 groups: antianxiety medications, antidepressants, antipsychotics, and opioid analgesics. Total amount included both regularly administered and as‐needed dosages of medication. We identified injuries through reports of a subject's primary nurse and by review of medical records.

Data Analyses

Sample size was calculated using a 2‐sample t test formula based on the primary outcome. The study was designed to detect an absolute difference in points of 10% (total absolute score difference of 11 per survey or a total difference of 33). The 2‐sided alpha was initially set at 0.05 and the power at 80%, with an estimated standard deviation of 20. The alpha level was Bonferroni‐adjusted for up to 6 additional comparisons, with each significance level of 0.0071 (z = 2.70).

To assess differences in patient characteristics between the standard restraints and safe enclosure groups, we used the Student t test for continuous variables and Fisher's 2‐sided exact test for categorical variables. Differences in family and staff perceptions of the restraint mechanisms were measured using the Student t test with Satterthwaite's method for calculating variance. However, to account for questions marked not applicable by the responder, weighted scores, defined as the total score divided by the percentage of questions answered, were calculated.

Differences in agitation scores (ABS and AWAF) were analyzed using 2 strategies. First, the ABS and AWAF scores 24 hours after study enrollment were compared across groups using the Student t test with Satterthwaite's method for calculating variance. Then, separate comparisons of the ABS and AWAF scores 48 and 72 hours after enrollment were conducted using the Student t test with a pooled variance. Next, 2 longitudinal analyses were performed using a mixed‐effects (fixed and random) model. These analyses modeled change in the ABS or AWAF scores over (1) the first 3 days and (2) the first 6 days of hospitalization as a function of being restrained with the safe enclosure or being restrained with the hospital's standard restraint systems. For these comparisons, the model included not only the main effects of type of restraint and time, but also the interaction between type of restraint and time and the covariates sex, age, and initial ABS or AWAF score. For these models, a backward elimination procedure was undertaken using a significance level of = 0.05 in order to determine the most parsimonious model.

To determine if the total length of subject stay in the hospital was different between groups, the Student t test was used with Satterthwaite's method for calculating variance. Differences in time from admission until time of restraint application, duration of restraint use, and time from application of restraints until time of discharge were analyzed with the Student t test with pooled variances.

To compare the amount of medication used, equivalent dosage conversions were used for each of the 4 medication categories (antianxiety medications, antidepressants, antipsychotics, and opioid analgesics). To determine if the amounts of these 4 categories of medications differed between groups, the Student t test was used. Last, to determine if there was a difference in the number of patient injuries between groups, Fisher's 2‐sided exact test was used.

Study Population

Of the 49 subjects enrolled in the study, 20 were randomized to the safe enclosure and 29 to standard restraints. This imbalance was likely a result of the premature termination of the study, which in turn was a result of slow recruitment. Table 1 shows selected baseline characteristics of the enrolled subjects. There were no significant differences between the 2 groups in sex, age, patient diagnoses, reason for restraint, or type of medication. However, the subjects randomized to the safe enclosure were less likely to have hypertension than those randomized to standard restraints (36.8% vs. 72.4%, P = .019).

Primary Outcomes

The rates of response to the perception survey were: relatives/next of kin, 90%; physicians, 90%; primary nurses, 100%; and secondary nurses, 78%. Family members and physicians viewed the safe enclosure significantly more positively than they viewed standard restraints (P < .0001 and P < .0001, respectively; Table 2). There was a trend toward more positive perceptions of the safe enclosure among nurses; however, this trend did not achieve statistical significance (P = .0836). The subgroup analysis of nurses (primary vs. secondary) revealed that secondary nurses viewed the safe enclosure more positively (P = .023). Primary nurses tended to view the safe enclosure more positively than the standard restraints, but the association was not significant (P = .1313).

There were no statistically significant differences between the 2 randomized groups in ABS or AWAF scores 24, 48, or 72 hours after restraint application (Table 2). In addition, there were no statistically significant differences during the study between the groups in the rates of change in ABS or AWAF score . This was the case when looking at the first 3 days of hospitalization as well as the first 6 days (data not shown). All results were also calculated after adjusting for length of stay; this covariate did not affect any of the results.

Table 3 details the results for each perception survey question. Perceived comfort, calmness, and safety of patients were rated higher in the safe enclosure group by physicians, relatives, and all nurses. With the exception of perceived accessibility to patients, relatives rated the safe enclosure higher than standard restraints on all other perception measures. Table 4 illustrates the differences in the responses of primary and secondary nurse to each perception survey question. Primary and secondary nurses viewed the safety of the safe enclosure significantly more positively than they did the standard restraints.

Secondary Outcomes

There was a trend toward shorter total length of stay, time from admission until restraint application, duration of restraint use, and time from restraint application until discharge among subjects restrained by the safe enclosure compared with those restrained with standard restraints. However, these unadjusted differences were not statistically significant. We examined secondary outcomes after adjusting for 2 covariates, age and sex. Age but not sex affected the results. We found that subjects in the intervention group younger than 80 years of age had a shorter length of stay for 2 of the 4 related outcomes: time of admittance to time of discharge (P = .0199) and time of restraint to time of discharge (P = .0274). Time of admission to time of restraint application and duration of restraint did not differ between groups. The former outcome was not expected to differ between groups.

Additional Outcomes

There were no differences between groups in the amounts of 3 of the 4 types of medications used to treat agitation or delirium (ie, antianxiety medications, antipsychotic medications, opioid analgesic medications). The proportion of patients on these medications did not differ by group (P = .59). Only 5% of patients in standard restraints were on antidepressants, and about 5% were on opioids in each group. There was only 1 minor patient injury recorded during the study. This minor abrasion was to a patient assigned to the standard restraint group. No injuries were reported in the safe enclosure group.

Provider Perspective

The responsibility of breaking bad news to patients weighs heavily on clinicians. As in this case, most providers' first experience with breaking bad news occurs with patients they have known for only a few hours or days. Even for the more experienced, this part of the job is rated as at least moderately stressful. Notably, most also feel that this stress lasts beyond the encounter, despite their perceived ability to manage their own stress during these situations.4 Additional training on clinicians' own coping skills may alleviate some of the emotional burden.

Provider's awareness and management of distress may enhance ability to provide comfort to patients or to specifically address their needs. Medical providers may try to suppress personal thoughts and feelings in these situations, but they bring emotional attachments to almost all encounters with patients.5 Emotional preparation by the provider is an important step prior to delivering bad news. Self‐reflection helps to identify personal emotions of sadness, anger, fear, or guilt and will help the provider not to disengage from the delivery of bad news.6 It is normal to have strong feelings, especially in difficult situations. Encouraging and validating these emotions personally will lead to a more therapeutic presence during a patient's time of need.7

Clinicians' perceptions of their interactions with patients when discussing bad news are probably more strongly influenced by the content of the discussion rather than the process itself. When asked to analyze their own videotaped consultations, doctors thought performance was worse when discussing palliative therapy than when discussing curative therapy.8

Traditionally, greatest emphasis has been placed on the acquisition and assessment of medical knowledge in medical training, and thus the focus on content is understandable. But more recently, efforts have been made at all levels of medical education to shift this focus toward encompassing many other competencies including professionalism and communication skills, which should translate into equal emphasis on the quality of these interactions.

As many hospitalists work closely with trainees, they are in the ideal position to serve as mentors and role models for communication. The case discussed in this article provides an example of a missed teaching opportunity. Ideally, the attending would have gone through the steps of preparation with the resident prior to the meeting, reviewed one or several of the suggested approaches discussed below, and observed the conversation and provided immediate feedback and a forum for processing afterward. It is especially helpful when first developing this skill to be familiar with helpful phrases to open the conversation, clarify patient preferences for communication, and convey empathy. It is also helpful to be aware of phrases that should be avoided (Table 1).

Family/Patient Perspective

As patient preferences for receiving bad news vary widely, it is these preferences that should determine the approach to the delivery, content, and context in which the news will be received. Some patients want information, and some do not; this needs to be clarified before beginning the discussion. The amount of detail should be negotiated in advance as well. As suggested by Back et al., soliciting patient preference prior to a discussion is important.9 These authors recommended using an approach called asktellask. This approach emphasizes the importance of asking questions to assess a patient's needs, telling the patient the information that meets those needs, and asking again to assess the patient's understanding.

Patients will rarely raise the issue of bad news with providers. In general, the provider must initiate the discussion.10 Surveys of patient preferences for delivery of bad news lend insight into this process and help guide providers during this challenging time. Patients report poor delivery is often characterized by bluntness, a lack of hope, and initiation of this serious conversation at an inappropriate time or place.11 Patients prefer providers to speak in clear, simple terms, being careful not to use technical jargon.12 Clinicians often use euphemisms to soften the blow of bad news, but this can lead to ambiguity. In addition to the clarity of the message, privacy, the attitude of the doctor, and the ability to answer questions are most important to patients and families receiving bad news.13 Although most would encourage touching in these difficult situations, it has been reported that up to a third of patients surveyed do not want physical contact.

Contrary to what providers may believe, diagnostic disclosure is not the most important part of a bad‐news discussion. Many patients believe it is most important to receive information on prognosis and treatment options. Often, patients want to discuss life expectancy. However, physicians are hesitant to address this issue. One study revealed that despite these requests, 22% of physicians would not provide any estimates at all, and when they did, 36% offered an overestimate.14 The authors hypothesized that how confident physicians are in this prognostic estimate and how much and what type of practice experience they have may influence their willingness to communicate a frank survival estimate.

The traditional dilemma of balancing hope with realism is reframed by Hagerty et al., who found that 98% of the patients they surveyed preferred a realistic and individualized approach.15 Use of euphemism and apparent unease of the provider actually decreases hope. Clayton et al. added that nurturing hope can also be facilitated by emphasizing what can be done, such as symptomatic management, emotional support, and practical support, particularly in terms of day‐to‐day living.16

In the case discussed in this article, preparation should have included asking the patient (1) whom he wanted present during the meeting, (2) how much information he and his family wanted to know, and (3) how involved he wanted his family to be. The informational needs of patients and their families will evolve over time as they process and accept the news. Thus, the asktellask approach remains a key concept to keep in mind as the dialogue continues beyond the initial encounter.

Importance of Empathy

The team failed in its inability to respond to emotion in this case. The emotional turmoil was apparent, but the team members made no attempt to acknowledge this emotion or to arrange additional emotional support. This could, at least in part, be a result of the providers' inability to process and manage their own emotions. A preparatory meeting beforehand and a debriefing session afterward for all the team members may have helped. Awareness of patients' coping strategies and various effective responses to these coping strategies may have better prepared the team to react and validate this patient's emotions. The role that psychology and emotion play cannot be overemphasized and clearly are important considerations. A useful mnemonic highlighting the components of the empathetic response is NURSE: naming, understanding, respect, supporting, and exploring.9 Using this technique, the provider starts by naming the emotion (anger, fear, disbelief); confirms a clear understanding of the patient's feelings; expresses respect verbally or nonverbally, letting the patient know the emotion is important; uses supporting statements that may express concern, reiterate understanding, or indicate a willingness to help; and closes by exploring additional concerns.

Certain phrases such as the one uttered by the fellowThere is nothing that can be doneshould be avoided.

Cultural Issues

Cultural diversity is increasingly common in contemporary medical practice in the United States. Some have suggested the Western value of autonomy is not embraced by all cultures. It has been suggested that non‐English‐speaking patients may receive less optimal end‐of‐life care than their English‐speaking counterparts.17 Beyond the language barrier, this observation may be a reflection of associated cultural barriers as well. Effective strategies for key issues of truth telling, language, family involvement, and decision making may help effective cross‐cultural communication and understanding and thus be effective patient‐centered care.18

A study of Korean patients and family members revealed a marked discordance between family and patients in the desire for disclosure.19 Almost all patients wanted to be informed that they had terminal illness, whereas a quarter of family members did not want physicians to relay this bad news to the patient. Interestingly, this study found patients would prefer to be told by the physician, whereas their family members would prefer to be the ones to deliver the news.

In some cultures it is believed that disclosure of bad news may cause patients to lose hope and hasten death. Physicians in these cultures may be more likely to honor family wishes. Language barriers may make a difficult situation even more complex. It is important to ascertain early on in what language patients and their families want to hold discussions. A medical interpreter should always be utilized for discussions with patients and their families. Dependence on a family member to interpret is not advised because the objective point of view may be lost in the interpretation. In addition, this places an enormous burden on the family member to be the bearer of bad news, which could have a lasting emotional impact. Although in the case discussed in this article, the patient's daughter was bilingual and could have translated, an interpreter should have been present for all discussions with this patient. Again, the importance of soliciting the patient's preference is critical.

Prior knowledge of the language requirement and of the patient's need for his daughter's involvement would also aid in the planning process. Because the patient's primary care physician (PCP) shared JR's Hispanic heritage, consultation with this PCP might have provided important insight, resulting in better preparation and planning.

How to Deliver Bad News

A number of guidelines are available to help physicians structure their conversations.2022 Baile and Buckman outline a 6‐step approach (Table 2). Rabow and Mcphee recommend the ABCDE mnemonic to help providers remember techniques for delivering bad news (Table 3). These recommendations are largely based on the literature to date. Both these approaches first emphasize preparations and planning. A private and quiet space, the presence of significant others if desired, arrangement to minimize interruptions, and provider emotional preparations are all prerequisites for the success of this type of encounter before the actual dialogue begins. As the encounter begins, it is key to assess a patient's needs if not already done before conveying information. Unidirectional transfer of information most likely will fail to satisfy the patient. The resident's initial conversation suffered in this key aspect.

Armed with knowledge of a patient's individual preferences, it is then possible to effectively convey information in a clear manner without jargon, using a direct but not blunt style. Both the SPIKES and ABCDE approaches similarly emphasize the asktellask approach. The attending physician was fairly effective in applying this communication approach in the subsequent encounter. However, the team left the room without providing a summary and follow‐up plan. Even though the patient and his daughter were quite emotional, acknowledging their reactions and appropriately ending the meeting with a summary and plans for the next steps would have been helpful in this continuing dialogue.

Hospitalist‐Specific Issues

Hospitalists may face special challenges when delivering bad news to patients. Without the benefit of preexisting longitudinal relationships with their patients, they lack prior understanding of a patient's values, family support system, and other cultural, spiritual, and social issues. Thus, preparation for these conversations is more difficult, and establishing rapport is more time‐consuming. There are no data available to describe the impact that not having a previous relationship with a patient has on these encounters. It is possible that the newness of the hospitalistpatient relationship may allow more candid, transparent communication than would be possible with established providers, who may themselves be struggling with the news and how it reflects on their care or the emotional impact of the impending loss.

Handoffs are a frequent part of the care the hospitalist provides, but communicating bad news is often a longitudinal process. One hospitalist may have the initial conversation regarding the patient's disease and prognosis, but the follow‐up often falls to a different hospitalist. Continuity of communication and awareness about what has been said previously are critical. It is important to explicitly document these conversations and their content in the medical record. In additional, summaries of pivotal conversations should be included in sign‐out. At discharge, whether patients are transitioning to postacute care or back to the outpatient arena, the hospitalist should carefully and vigilantly communicate critical conversations and predictions about patients' emotional needs.

Hospitalists do have some advantages when it comes to communication with patients. Unlike in outpatient practice, where clinicians are under pressure to keep up with a heavily loaded patient schedule, the hospitalist often has the flexibility and ability to allot time to each patient according to that patient's need. In addition, by definition, a hospitalist is in a hospital; this availability allows for more timely meetings, minimal delay in delivery of news, and accommodating the schedules of other people the patient may want included in any conversations.

Training

What can be done to improve the effectiveness and satisfaction of these interactions for patients, providers, and families? Awareness of guidelines and effective strategies is a start but is unlikely to really change behavior or improve skills. Communication skills must be practiced, implemented, and observed with opportunity for feedback.23 Graduate and postgraduate training is probably the best time to develop these skills, and formal training in this area should be incorporated in a curriculum. Workshops on communicating bad news are offered frequently to oncologists and oncology fellows at various regional and national meetings. Ideally, these workshops would be offered at CME meetings specifically designed for hospitalists already in practice. Comprehensive palliative care training and materials, including specific modules and live workshops for delivering bad news, are available via the Education in Palliative and End‐of‐Life Care Project (EPEC) and the End‐of‐Life/Palliative Education Resource Center (EPERC) at the University of Wisconsin.

Hospitalists and trainees fortunate enough to practice in an institution with a palliative care service have the opportunity to learn from a multidisciplinary team, often including social workers, nursing staff, physicians and spiritual leaders. This interdisciplinary model is likely a more effective way to address the diverse physical, emotional, social, and spiritual needs of patients receiving difficult news and provides an ideal framework for this training.

Provider Perspective

The responsibility of breaking bad news to patients weighs heavily on clinicians. As in this case, most providers' first experience with breaking bad news occurs with patients they have known for only a few hours or days. Even for the more experienced, this part of the job is rated as at least moderately stressful. Notably, most also feel that this stress lasts beyond the encounter, despite their perceived ability to manage their own stress during these situations.4 Additional training on clinicians' own coping skills may alleviate some of the emotional burden.

Provider's awareness and management of distress may enhance ability to provide comfort to patients or to specifically address their needs. Medical providers may try to suppress personal thoughts and feelings in these situations, but they bring emotional attachments to almost all encounters with patients.5 Emotional preparation by the provider is an important step prior to delivering bad news. Self‐reflection helps to identify personal emotions of sadness, anger, fear, or guilt and will help the provider not to disengage from the delivery of bad news.6 It is normal to have strong feelings, especially in difficult situations. Encouraging and validating these emotions personally will lead to a more therapeutic presence during a patient's time of need.7

Clinicians' perceptions of their interactions with patients when discussing bad news are probably more strongly influenced by the content of the discussion rather than the process itself. When asked to analyze their own videotaped consultations, doctors thought performance was worse when discussing palliative therapy than when discussing curative therapy.8

Traditionally, greatest emphasis has been placed on the acquisition and assessment of medical knowledge in medical training, and thus the focus on content is understandable. But more recently, efforts have been made at all levels of medical education to shift this focus toward encompassing many other competencies including professionalism and communication skills, which should translate into equal emphasis on the quality of these interactions.

As many hospitalists work closely with trainees, they are in the ideal position to serve as mentors and role models for communication. The case discussed in this article provides an example of a missed teaching opportunity. Ideally, the attending would have gone through the steps of preparation with the resident prior to the meeting, reviewed one or several of the suggested approaches discussed below, and observed the conversation and provided immediate feedback and a forum for processing afterward. It is especially helpful when first developing this skill to be familiar with helpful phrases to open the conversation, clarify patient preferences for communication, and convey empathy. It is also helpful to be aware of phrases that should be avoided (Table 1).

Family/Patient Perspective

As patient preferences for receiving bad news vary widely, it is these preferences that should determine the approach to the delivery, content, and context in which the news will be received. Some patients want information, and some do not; this needs to be clarified before beginning the discussion. The amount of detail should be negotiated in advance as well. As suggested by Back et al., soliciting patient preference prior to a discussion is important.9 These authors recommended using an approach called asktellask. This approach emphasizes the importance of asking questions to assess a patient's needs, telling the patient the information that meets those needs, and asking again to assess the patient's understanding.

Patients will rarely raise the issue of bad news with providers. In general, the provider must initiate the discussion.10 Surveys of patient preferences for delivery of bad news lend insight into this process and help guide providers during this challenging time. Patients report poor delivery is often characterized by bluntness, a lack of hope, and initiation of this serious conversation at an inappropriate time or place.11 Patients prefer providers to speak in clear, simple terms, being careful not to use technical jargon.12 Clinicians often use euphemisms to soften the blow of bad news, but this can lead to ambiguity. In addition to the clarity of the message, privacy, the attitude of the doctor, and the ability to answer questions are most important to patients and families receiving bad news.13 Although most would encourage touching in these difficult situations, it has been reported that up to a third of patients surveyed do not want physical contact.

Contrary to what providers may believe, diagnostic disclosure is not the most important part of a bad‐news discussion. Many patients believe it is most important to receive information on prognosis and treatment options. Often, patients want to discuss life expectancy. However, physicians are hesitant to address this issue. One study revealed that despite these requests, 22% of physicians would not provide any estimates at all, and when they did, 36% offered an overestimate.14 The authors hypothesized that how confident physicians are in this prognostic estimate and how much and what type of practice experience they have may influence their willingness to communicate a frank survival estimate.

The traditional dilemma of balancing hope with realism is reframed by Hagerty et al., who found that 98% of the patients they surveyed preferred a realistic and individualized approach.15 Use of euphemism and apparent unease of the provider actually decreases hope. Clayton et al. added that nurturing hope can also be facilitated by emphasizing what can be done, such as symptomatic management, emotional support, and practical support, particularly in terms of day‐to‐day living.16

In the case discussed in this article, preparation should have included asking the patient (1) whom he wanted present during the meeting, (2) how much information he and his family wanted to know, and (3) how involved he wanted his family to be. The informational needs of patients and their families will evolve over time as they process and accept the news. Thus, the asktellask approach remains a key concept to keep in mind as the dialogue continues beyond the initial encounter.

Importance of Empathy

The team failed in its inability to respond to emotion in this case. The emotional turmoil was apparent, but the team members made no attempt to acknowledge this emotion or to arrange additional emotional support. This could, at least in part, be a result of the providers' inability to process and manage their own emotions. A preparatory meeting beforehand and a debriefing session afterward for all the team members may have helped. Awareness of patients' coping strategies and various effective responses to these coping strategies may have better prepared the team to react and validate this patient's emotions. The role that psychology and emotion play cannot be overemphasized and clearly are important considerations. A useful mnemonic highlighting the components of the empathetic response is NURSE: naming, understanding, respect, supporting, and exploring.9 Using this technique, the provider starts by naming the emotion (anger, fear, disbelief); confirms a clear understanding of the patient's feelings; expresses respect verbally or nonverbally, letting the patient know the emotion is important; uses supporting statements that may express concern, reiterate understanding, or indicate a willingness to help; and closes by exploring additional concerns.

Certain phrases such as the one uttered by the fellowThere is nothing that can be doneshould be avoided.

Cultural Issues

Cultural diversity is increasingly common in contemporary medical practice in the United States. Some have suggested the Western value of autonomy is not embraced by all cultures. It has been suggested that non‐English‐speaking patients may receive less optimal end‐of‐life care than their English‐speaking counterparts.17 Beyond the language barrier, this observation may be a reflection of associated cultural barriers as well. Effective strategies for key issues of truth telling, language, family involvement, and decision making may help effective cross‐cultural communication and understanding and thus be effective patient‐centered care.18

A study of Korean patients and family members revealed a marked discordance between family and patients in the desire for disclosure.19 Almost all patients wanted to be informed that they had terminal illness, whereas a quarter of family members did not want physicians to relay this bad news to the patient. Interestingly, this study found patients would prefer to be told by the physician, whereas their family members would prefer to be the ones to deliver the news.

In some cultures it is believed that disclosure of bad news may cause patients to lose hope and hasten death. Physicians in these cultures may be more likely to honor family wishes. Language barriers may make a difficult situation even more complex. It is important to ascertain early on in what language patients and their families want to hold discussions. A medical interpreter should always be utilized for discussions with patients and their families. Dependence on a family member to interpret is not advised because the objective point of view may be lost in the interpretation. In addition, this places an enormous burden on the family member to be the bearer of bad news, which could have a lasting emotional impact. Although in the case discussed in this article, the patient's daughter was bilingual and could have translated, an interpreter should have been present for all discussions with this patient. Again, the importance of soliciting the patient's preference is critical.

Prior knowledge of the language requirement and of the patient's need for his daughter's involvement would also aid in the planning process. Because the patient's primary care physician (PCP) shared JR's Hispanic heritage, consultation with this PCP might have provided important insight, resulting in better preparation and planning.

How to Deliver Bad News

A number of guidelines are available to help physicians structure their conversations.2022 Baile and Buckman outline a 6‐step approach (Table 2). Rabow and Mcphee recommend the ABCDE mnemonic to help providers remember techniques for delivering bad news (Table 3). These recommendations are largely based on the literature to date. Both these approaches first emphasize preparations and planning. A private and quiet space, the presence of significant others if desired, arrangement to minimize interruptions, and provider emotional preparations are all prerequisites for the success of this type of encounter before the actual dialogue begins. As the encounter begins, it is key to assess a patient's needs if not already done before conveying information. Unidirectional transfer of information most likely will fail to satisfy the patient. The resident's initial conversation suffered in this key aspect.

Armed with knowledge of a patient's individual preferences, it is then possible to effectively convey information in a clear manner without jargon, using a direct but not blunt style. Both the SPIKES and ABCDE approaches similarly emphasize the asktellask approach. The attending physician was fairly effective in applying this communication approach in the subsequent encounter. However, the team left the room without providing a summary and follow‐up plan. Even though the patient and his daughter were quite emotional, acknowledging their reactions and appropriately ending the meeting with a summary and plans for the next steps would have been helpful in this continuing dialogue.

Hospitalist‐Specific Issues

Hospitalists may face special challenges when delivering bad news to patients. Without the benefit of preexisting longitudinal relationships with their patients, they lack prior understanding of a patient's values, family support system, and other cultural, spiritual, and social issues. Thus, preparation for these conversations is more difficult, and establishing rapport is more time‐consuming. There are no data available to describe the impact that not having a previous relationship with a patient has on these encounters. It is possible that the newness of the hospitalistpatient relationship may allow more candid, transparent communication than would be possible with established providers, who may themselves be struggling with the news and how it reflects on their care or the emotional impact of the impending loss.

Handoffs are a frequent part of the care the hospitalist provides, but communicating bad news is often a longitudinal process. One hospitalist may have the initial conversation regarding the patient's disease and prognosis, but the follow‐up often falls to a different hospitalist. Continuity of communication and awareness about what has been said previously are critical. It is important to explicitly document these conversations and their content in the medical record. In additional, summaries of pivotal conversations should be included in sign‐out. At discharge, whether patients are transitioning to postacute care or back to the outpatient arena, the hospitalist should carefully and vigilantly communicate critical conversations and predictions about patients' emotional needs.

Hospitalists do have some advantages when it comes to communication with patients. Unlike in outpatient practice, where clinicians are under pressure to keep up with a heavily loaded patient schedule, the hospitalist often has the flexibility and ability to allot time to each patient according to that patient's need. In addition, by definition, a hospitalist is in a hospital; this availability allows for more timely meetings, minimal delay in delivery of news, and accommodating the schedules of other people the patient may want included in any conversations.

Training

What can be done to improve the effectiveness and satisfaction of these interactions for patients, providers, and families? Awareness of guidelines and effective strategies is a start but is unlikely to really change behavior or improve skills. Communication skills must be practiced, implemented, and observed with opportunity for feedback.23 Graduate and postgraduate training is probably the best time to develop these skills, and formal training in this area should be incorporated in a curriculum. Workshops on communicating bad news are offered frequently to oncologists and oncology fellows at various regional and national meetings. Ideally, these workshops would be offered at CME meetings specifically designed for hospitalists already in practice. Comprehensive palliative care training and materials, including specific modules and live workshops for delivering bad news, are available via the Education in Palliative and End‐of‐Life Care Project (EPEC) and the End‐of‐Life/Palliative Education Resource Center (EPERC) at the University of Wisconsin.

Hospitalists and trainees fortunate enough to practice in an institution with a palliative care service have the opportunity to learn from a multidisciplinary team, often including social workers, nursing staff, physicians and spiritual leaders. This interdisciplinary model is likely a more effective way to address the diverse physical, emotional, social, and spiritual needs of patients receiving difficult news and provides an ideal framework for this training.

Provider Perspective

The responsibility of breaking bad news to patients weighs heavily on clinicians. As in this case, most providers' first experience with breaking bad news occurs with patients they have known for only a few hours or days. Even for the more experienced, this part of the job is rated as at least moderately stressful. Notably, most also feel that this stress lasts beyond the encounter, despite their perceived ability to manage their own stress during these situations.4 Additional training on clinicians' own coping skills may alleviate some of the emotional burden.

Provider's awareness and management of distress may enhance ability to provide comfort to patients or to specifically address their needs. Medical providers may try to suppress personal thoughts and feelings in these situations, but they bring emotional attachments to almost all encounters with patients.5 Emotional preparation by the provider is an important step prior to delivering bad news. Self‐reflection helps to identify personal emotions of sadness, anger, fear, or guilt and will help the provider not to disengage from the delivery of bad news.6 It is normal to have strong feelings, especially in difficult situations. Encouraging and validating these emotions personally will lead to a more therapeutic presence during a patient's time of need.7

Clinicians' perceptions of their interactions with patients when discussing bad news are probably more strongly influenced by the content of the discussion rather than the process itself. When asked to analyze their own videotaped consultations, doctors thought performance was worse when discussing palliative therapy than when discussing curative therapy.8

Traditionally, greatest emphasis has been placed on the acquisition and assessment of medical knowledge in medical training, and thus the focus on content is understandable. But more recently, efforts have been made at all levels of medical education to shift this focus toward encompassing many other competencies including professionalism and communication skills, which should translate into equal emphasis on the quality of these interactions.

As many hospitalists work closely with trainees, they are in the ideal position to serve as mentors and role models for communication. The case discussed in this article provides an example of a missed teaching opportunity. Ideally, the attending would have gone through the steps of preparation with the resident prior to the meeting, reviewed one or several of the suggested approaches discussed below, and observed the conversation and provided immediate feedback and a forum for processing afterward. It is especially helpful when first developing this skill to be familiar with helpful phrases to open the conversation, clarify patient preferences for communication, and convey empathy. It is also helpful to be aware of phrases that should be avoided (Table 1).

Family/Patient Perspective

As patient preferences for receiving bad news vary widely, it is these preferences that should determine the approach to the delivery, content, and context in which the news will be received. Some patients want information, and some do not; this needs to be clarified before beginning the discussion. The amount of detail should be negotiated in advance as well. As suggested by Back et al., soliciting patient preference prior to a discussion is important.9 These authors recommended using an approach called asktellask. This approach emphasizes the importance of asking questions to assess a patient's needs, telling the patient the information that meets those needs, and asking again to assess the patient's understanding.

Patients will rarely raise the issue of bad news with providers. In general, the provider must initiate the discussion.10 Surveys of patient preferences for delivery of bad news lend insight into this process and help guide providers during this challenging time. Patients report poor delivery is often characterized by bluntness, a lack of hope, and initiation of this serious conversation at an inappropriate time or place.11 Patients prefer providers to speak in clear, simple terms, being careful not to use technical jargon.12 Clinicians often use euphemisms to soften the blow of bad news, but this can lead to ambiguity. In addition to the clarity of the message, privacy, the attitude of the doctor, and the ability to answer questions are most important to patients and families receiving bad news.13 Although most would encourage touching in these difficult situations, it has been reported that up to a third of patients surveyed do not want physical contact.

Contrary to what providers may believe, diagnostic disclosure is not the most important part of a bad‐news discussion. Many patients believe it is most important to receive information on prognosis and treatment options. Often, patients want to discuss life expectancy. However, physicians are hesitant to address this issue. One study revealed that despite these requests, 22% of physicians would not provide any estimates at all, and when they did, 36% offered an overestimate.14 The authors hypothesized that how confident physicians are in this prognostic estimate and how much and what type of practice experience they have may influence their willingness to communicate a frank survival estimate.

The traditional dilemma of balancing hope with realism is reframed by Hagerty et al., who found that 98% of the patients they surveyed preferred a realistic and individualized approach.15 Use of euphemism and apparent unease of the provider actually decreases hope. Clayton et al. added that nurturing hope can also be facilitated by emphasizing what can be done, such as symptomatic management, emotional support, and practical support, particularly in terms of day‐to‐day living.16

In the case discussed in this article, preparation should have included asking the patient (1) whom he wanted present during the meeting, (2) how much information he and his family wanted to know, and (3) how involved he wanted his family to be. The informational needs of patients and their families will evolve over time as they process and accept the news. Thus, the asktellask approach remains a key concept to keep in mind as the dialogue continues beyond the initial encounter.

Importance of Empathy

The team failed in its inability to respond to emotion in this case. The emotional turmoil was apparent, but the team members made no attempt to acknowledge this emotion or to arrange additional emotional support. This could, at least in part, be a result of the providers' inability to process and manage their own emotions. A preparatory meeting beforehand and a debriefing session afterward for all the team members may have helped. Awareness of patients' coping strategies and various effective responses to these coping strategies may have better prepared the team to react and validate this patient's emotions. The role that psychology and emotion play cannot be overemphasized and clearly are important considerations. A useful mnemonic highlighting the components of the empathetic response is NURSE: naming, understanding, respect, supporting, and exploring.9 Using this technique, the provider starts by naming the emotion (anger, fear, disbelief); confirms a clear understanding of the patient's feelings; expresses respect verbally or nonverbally, letting the patient know the emotion is important; uses supporting statements that may express concern, reiterate understanding, or indicate a willingness to help; and closes by exploring additional concerns.

Certain phrases such as the one uttered by the fellowThere is nothing that can be doneshould be avoided.

Cultural Issues

Cultural diversity is increasingly common in contemporary medical practice in the United States. Some have suggested the Western value of autonomy is not embraced by all cultures. It has been suggested that non‐English‐speaking patients may receive less optimal end‐of‐life care than their English‐speaking counterparts.17 Beyond the language barrier, this observation may be a reflection of associated cultural barriers as well. Effective strategies for key issues of truth telling, language, family involvement, and decision making may help effective cross‐cultural communication and understanding and thus be effective patient‐centered care.18

A study of Korean patients and family members revealed a marked discordance between family and patients in the desire for disclosure.19 Almost all patients wanted to be informed that they had terminal illness, whereas a quarter of family members did not want physicians to relay this bad news to the patient. Interestingly, this study found patients would prefer to be told by the physician, whereas their family members would prefer to be the ones to deliver the news.

In some cultures it is believed that disclosure of bad news may cause patients to lose hope and hasten death. Physicians in these cultures may be more likely to honor family wishes. Language barriers may make a difficult situation even more complex. It is important to ascertain early on in what language patients and their families want to hold discussions. A medical interpreter should always be utilized for discussions with patients and their families. Dependence on a family member to interpret is not advised because the objective point of view may be lost in the interpretation. In addition, this places an enormous burden on the family member to be the bearer of bad news, which could have a lasting emotional impact. Although in the case discussed in this article, the patient's daughter was bilingual and could have translated, an interpreter should have been present for all discussions with this patient. Again, the importance of soliciting the patient's preference is critical.

Prior knowledge of the language requirement and of the patient's need for his daughter's involvement would also aid in the planning process. Because the patient's primary care physician (PCP) shared JR's Hispanic heritage, consultation with this PCP might have provided important insight, resulting in better preparation and planning.

How to Deliver Bad News

A number of guidelines are available to help physicians structure their conversations.2022 Baile and Buckman outline a 6‐step approach (Table 2). Rabow and Mcphee recommend the ABCDE mnemonic to help providers remember techniques for delivering bad news (Table 3). These recommendations are largely based on the literature to date. Both these approaches first emphasize preparations and planning. A private and quiet space, the presence of significant others if desired, arrangement to minimize interruptions, and provider emotional preparations are all prerequisites for the success of this type of encounter before the actual dialogue begins. As the encounter begins, it is key to assess a patient's needs if not already done before conveying information. Unidirectional transfer of information most likely will fail to satisfy the patient. The resident's initial conversation suffered in this key aspect.

Armed with knowledge of a patient's individual preferences, it is then possible to effectively convey information in a clear manner without jargon, using a direct but not blunt style. Both the SPIKES and ABCDE approaches similarly emphasize the asktellask approach. The attending physician was fairly effective in applying this communication approach in the subsequent encounter. However, the team left the room without providing a summary and follow‐up plan. Even though the patient and his daughter were quite emotional, acknowledging their reactions and appropriately ending the meeting with a summary and plans for the next steps would have been helpful in this continuing dialogue.

Hospitalist‐Specific Issues

Hospitalists may face special challenges when delivering bad news to patients. Without the benefit of preexisting longitudinal relationships with their patients, they lack prior understanding of a patient's values, family support system, and other cultural, spiritual, and social issues. Thus, preparation for these conversations is more difficult, and establishing rapport is more time‐consuming. There are no data available to describe the impact that not having a previous relationship with a patient has on these encounters. It is possible that the newness of the hospitalistpatient relationship may allow more candid, transparent communication than would be possible with established providers, who may themselves be struggling with the news and how it reflects on their care or the emotional impact of the impending loss.

Handoffs are a frequent part of the care the hospitalist provides, but communicating bad news is often a longitudinal process. One hospitalist may have the initial conversation regarding the patient's disease and prognosis, but the follow‐up often falls to a different hospitalist. Continuity of communication and awareness about what has been said previously are critical. It is important to explicitly document these conversations and their content in the medical record. In additional, summaries of pivotal conversations should be included in sign‐out. At discharge, whether patients are transitioning to postacute care or back to the outpatient arena, the hospitalist should carefully and vigilantly communicate critical conversations and predictions about patients' emotional needs.

Hospitalists do have some advantages when it comes to communication with patients. Unlike in outpatient practice, where clinicians are under pressure to keep up with a heavily loaded patient schedule, the hospitalist often has the flexibility and ability to allot time to each patient according to that patient's need. In addition, by definition, a hospitalist is in a hospital; this availability allows for more timely meetings, minimal delay in delivery of news, and accommodating the schedules of other people the patient may want included in any conversations.

Training

What can be done to improve the effectiveness and satisfaction of these interactions for patients, providers, and families? Awareness of guidelines and effective strategies is a start but is unlikely to really change behavior or improve skills. Communication skills must be practiced, implemented, and observed with opportunity for feedback.23 Graduate and postgraduate training is probably the best time to develop these skills, and formal training in this area should be incorporated in a curriculum. Workshops on communicating bad news are offered frequently to oncologists and oncology fellows at various regional and national meetings. Ideally, these workshops would be offered at CME meetings specifically designed for hospitalists already in practice. Comprehensive palliative care training and materials, including specific modules and live workshops for delivering bad news, are available via the Education in Palliative and End‐of‐Life Care Project (EPEC) and the End‐of‐Life/Palliative Education Resource Center (EPERC) at the University of Wisconsin.

Hospitalists and trainees fortunate enough to practice in an institution with a palliative care service have the opportunity to learn from a multidisciplinary team, often including social workers, nursing staff, physicians and spiritual leaders. This interdisciplinary model is likely a more effective way to address the diverse physical, emotional, social, and spiritual needs of patients receiving difficult news and provides an ideal framework for this training.

Process Measures

Students were required to submit logs at the end of their rotations, recording patients' names, ages, diagnoses, and admission dates. The accuracy and completeness of these logs were not independently verified.

As there was no authoritative list of key diagnoses third‐year medical students should encounter in the patients they care for during their inpatient rotations, we relied on expert opinion at our institution. The Council on Medical Student Education in Pediatrics' curriculum was not used because it did not differentiate between inpatient and ambulatory contexts. A preliminary list of 93 diagnoses was developed from the table of contents of Pediatric Hospital Medicine.10 This list was distributed to the 26 clinical faculty members in the Divisions of Pediatric Inpatient Medicine and General Pediatrics who were asked to select the 10 most important diagnoses. Surveys were numerically coded to permit 1 reminder.

The survey had a response rate of 92.3% (24 of 26 surveys). One survey was excluded because the respondent significantly deviated from the instructions. The 10 key diagnoses and the percentages of respondents who selected each individual diagnosis are: asthma (100%), febrile infant (95.6%), diarrhea and dehydration (91.3%), bronchiolitis (78.2%), diabetes mellitus and diabetic ketoacidemia (60.9%), failure to thrive (56.5%), urinary tract infections (52.1%), pneumonia (47.8%), upper airway infections such as croup (43.5%), and seizures and status epilepticus (43.5%).

Two of the authors independently coded the diagnoses on the students' patient logs in terms of these 93 diagnoses. The authors were blinded to the students' service assignment. As many students reported more than 1 diagnosis, the authors prioritized primary, secondary, and tertiary diagnoses to simplify the evaluation. The most likely cause of admission was listed as the primary diagnosis. If the authors could not reconcile divergent views, a third party was consulted.

Student Performance

Students were evaluated by both the attending physician(s) and senior resident(s) using a standardized evaluation form available from the corresponding author. The evaluation contained 18 items in 7 categories: data gathering, data recording/reporting, knowledge, data interpretation, clinical performance, professional attitudes, and professional demeanor. The student was rated exceptional, above expectations, meets expectations, below expectations, unacceptable, or not observed on each item. A short narrative description illustrated each rating. The ratings were converted to a 5‐point scale, with exceptional being 5. If the evaluator marked the line between 2 ratings, it was recorded as half. When multiple attendings or residents evaluated a student, the scores for a given item were collapsed into an average score.

Students also completed a NBME pediatric subject examination on the last day of their rotation.

Additionally, students were requested to complete a questionnaire during the final week of the clerkship. The items on the questionnaire were meant to access students' perceptions of the quality of their attendings' and residents' teaching, a potentially confounding variable. The survey was piloted on a group of similar subjects. Informed consent was obtained for survey completion. The survey was anonymous and required approximately 7 minutes to complete.

Resource Utilization

Last, resource utilization data, length of stay and total charges, for the 4 most common primary diagnoses were compared between the intervention and the traditional services. The 4 most common primary diagnoses and the percentage of total diagnoses (n = 2047) that each represents were bronchiolitis, 13%; febrile infant, 8.6%; pneumonia, 7.1%; and asthma, 6.5% (the diagnosis other accounted for 12% of the total diagnoses). Unique patient identifiers were used to obtain length of stay and total charges from the hospital's database. All‐Patient‐Refined Diagnosis‐Related Groups Severity of Illness (APR‐DRG‐SOI) were also obtained and used to construct multivariate models. Patients who were admitted to the pediatric intensive care unit (PICU) were excluded from the analysis.

Statistical Analysis

Statistical analyses were conducted and frequencies and percentages were calculated using Stata SE version 8.0 (College Station, TX). For all interval and ratio‐scaled variables, distributions were tested for normality using the Shapiro‐Wilks test to determine whether to use parametric or nonparametric statistical tests. For distributions meeting the normality assumption, the unpaired t test was used to compare the intervention service with traditional services. Where the normality assumption was not met, the Mann‐Whitney test was used. Categorically scaled data were compared using Pearson's chi‐square test. The standardized mean differences, reported as d values, were calculated to determine the effect size. Small, medium, and large effect sizes were defined as d values of 0.20, 0.50, and 0.80, respectively.11 Teaching quality, an effect modifier, was entered as a covariate into a linear regression model. Analyses of length of stay and total charges were conducted using multivariate linear regression controlling for patient age and severity of illness.

This study was approved by the University of Utah and Primary Children's Medical Center's Institutional Review Board.

Process Measures

Overall, 96.6% of students (196 of 203) submitted patient logs; 97.7% of students (125 of 128) on the intervention service and 94.7% of students (71 of 75) on the traditional services. The students on the intervention service admitted a median of 10 patients, whereas the students on the traditional services admitted a median of 11 patients (d = 0.45, P < .01). Age data were recorded on 137 patient logs (69.9% of submitted logs, 72.0% of students on the intervention service vs. 66.2% of students on the traditional services). The percentage of students who saw at least 1 newborn (birth‐23 months), child (2‐12 years), and adolescent (12‐18 years) was 34.8% on the intervention service and 33.3% on the traditional services (P = .87) (Table 2).

Students on the intervention service encountered, on average, a larger number of the 10 key diagnoses (4.4 vs. 3.6, d = 0.48, P < .01) and a higher percentage of their patients had clinical conditions among the key diagnoses (59.3 vs. 46.8, d = 0.62, P < .01). To determine if this higher percentage was the result of admitting multiple patients with the same diagnosis, we examined the percentage of unique primary diagnosesthe number of different primary diagnoses divided by the total number of patientsand found no differences (Table 2).

Student Performance

The faculty and resident evaluations of the students showed statistically significant differences between those in the intervention service and those in the traditional services in only 2 of the 18 items. These items were analysis in the data interpretation category (3.81 vs. 3.64, d = 0.35, P = .02) and patient interaction in the professional demeanor category (3.89 vs. 3.76, d = 0.31, P < .05). Both differences favored the intervention service. There were no statistical differences by service in student performance on the NBME subject examination (73.2 vs. 72.3, P = .39).

Student Satisfaction

Overall, 87.2% of students (177 of 203) completed the survey; 87.5% of students (112 of 124) on the intervention service and 86.7% of students (65 of 75) on the traditional services. The students on the intervention service both had a more positive overall attitude about their rotation and were more likely to find it a satisfying educational experience. Students on the intervention service also reported greater participation in patient care. Effect sizes ranged from small to medium (Table 3). The internal consistency of answers about participation in patient care was high (Pearson correlation coefficient r = 0.80).

Students on the intervention service rated the teaching of their attendings, but not of their residents, higher than did students on the traditional services. Controlling for the perceived quality of the attending, 3 of 6 satisfaction outcomes remained statistically significant: role on rotation (P < .01), input into patient care decisions (P < .01), and direct responsibility for patient care (P = .04). Students on both services believed they were appropriately supervised (P = .19). Despite the students on the traditional services on average admitting more patients, there was no significant difference by service in the students' rating of patient load (P = .33).

Career Choice

The odds ratio and 95% confidence interval for students enrolling in a pediatric subinternship was 1.94 (0.83‐4.49) and matching in a pediatric residency was 2.52 (0.99‐6.37). There were no statistically significant differences by service in the percentage of students enrolling in primary care (pediatric, internal medicine, and family practice) subinternships or residencies (Table 4).

Resource Utilization

One hundred and thirty‐five patients were excluded from the resource utilization analysis (n = 594) because their unique identifiers could not be found or they had been admitted to the PICU. Univariate analysis demonstrated statistically significant differences for patients with asthma, but not patients with bronchiolitis, febrile infants, or patients with pneumonia, favoring the intervention service. Patients with asthma admitted to the intervention service had a shorter length of stay (49.9 vs. 70.1 hours, P = .02) and lower total charges ($3600 vs. $4600, P = .02), as shown in Table 5. Of 4 multivariate models controlling for age and severity of illness, each with length of stay and total charges as the dependant variables, length of stay was significantly less for patients with asthma admitted to the intervention service only. Such patients were discharged an average of 23.3 hours earlier than patients with asthma admitted to the traditional services (P = .02).

Methods

The methods used in this study compare favorably with other evaluations of educational interventions. The present study incorporated a randomized controlled design.12 Although several studies of ambulatory clerkships used a randomized design, few randomized all eligible students.7, 8 The others used some form of selection prior to randomization. For example, in the Pangaro et al. study, students selected their clerkship site by lottery, with students selecting a certain site then offered the opportunity to participate in the intervention.6 The present study manifested several additional strengths. Multiple outcomes, including effects on patient care, were evaluated. Moreover, this study had a relatively large intervention group and total sample size compared with those in other medical education studies. Finally, because the intervention service had been in place for several years prior to its evaluation, the confounding influence of difficulties working out its implementation was minimized.

Results

Few studies of ambulatory experiences demonstrated statistically significant, let alone clinically significant, results. Most studies showed no statistically significant differences in student evaluations or examination scores. An exception is Grum et al., who showed improvements on 3 of 5 examinations.4 A few studies have found improved student satisfaction.3 None of the randomized controlled trials demonstrate increases in students matching in internal medicine or primary care residencies.4, 68 In contrast, this study produced statistically or programmatically significant results in process measures, evaluations, satisfaction, and career choices.

Several of our specific findings deserve additional comment. Although the admitting residents were instructed to assign patients to the intervention service based on their acuity and complexity, it is important to examine these residents' actual behavior. Several of our hypotheses were not validated. The students on the intervention service admitted fewer patients and were no more likely to see at least 1 patient in each age category. The admitting resident may have limited the number of patients admitted to the intervention service based on the workload of the supervising resident not that of the student. The supervising resident on the intervention service must round on all the patients, whereas the oversight of patients seen by students on the traditional services is shared with the interns. Having the attending on the intervention service share this supervising responsibility might improve this outcome.

Students on the intervention service had more positive attitudes toward the rotation. In addition, potentially negative attitudes were not manifest. For example, it might be argued that third‐year medical students are not prepared to bear this increased responsibility. However, there was not a significant difference in students' perception of the quality of supervision or the workload.

Although the goal of medical education is the production of competent physicians, it is important that the process not place undo burdens on patients and the health care system. Univariate analysis showed similar resource utilization. It might be contended that the admitting resident assigned the intervention service patients who were less acutely ill. Therefore, we performed multivariate analysis using APR‐DRG‐SOI to control for severity of illness. Of 8 comparisons, the only statistically significant difference, length of stay of patients with asthma, favored the intervention service.

Limitations

Although this study had numerous strengths, it also had several limitations. The primary limitations were lack of generalizability, difficulty in obtaining authentic assessments, the potential difference between statistical and educational significance, and inability to identify which components of the intervention service were responsible for the outcomes. This study's findings may not be generalizable to other institutions. For example, institutions without age or organ systembased teams may not observe increases in the number of key diagnoses encountered in the patients cared for. Regarding the assessments, there may be better measures of clinical competence, such as an objective structured clinical examination (OSCE),13 than those used in this study. However, there were not sufficient resources to implement an OSCE at the end of the rotation.

Some might question whether the statistically significant differences have educational significance. Although that is an important concern, this study should be compared with other educational interventions that found few statistically significant, let alone educationally significant, differences. To address this concern, we calculated effect sizes. The differences in student satisfaction were small to moderate. Although the lower limit of the 95% confidence interval of the odds ratio for matching in a pediatric residency was 0.99, the magnitude was programmatically important.

Finally, this study was an evaluation of an existing program. The authors were unable to control some potential confounders including patient allocation, average daily census, and quality of teaching. For example, Griffith and colleagues have shown that working with the best teachers improves student performance.14 We were not able to randomly assign the faculty among the services, and unequal distribution of better teachers could have biased this study's outcomes. The students on the intervention service rated their attendings, but not their residents, higher than did the students on the other services. However, the linear regression model showed that the perceived quality of the attending did not account for all the differences in student satisfaction. It was not possible to control for this factor in comparing student performance or subinternship or residency selection because the survey, which included the faculty evaluations, was anonymous and therefore could not be linked to the other data sets.

The perceived differences in the quality of teaching may not have been the result of differences in the attendings but instead of differences in the structure of the services. Accessibility is one of the characteristics of excellent clinical teachers.15 The intervention structure may permit faculty to spend more time with students, and this may increase the perceived quality of the teaching. However, it is not possible to resolve this issue with the available data.

Process Measures

Students were required to submit logs at the end of their rotations, recording patients' names, ages, diagnoses, and admission dates. The accuracy and completeness of these logs were not independently verified.

As there was no authoritative list of key diagnoses third‐year medical students should encounter in the patients they care for during their inpatient rotations, we relied on expert opinion at our institution. The Council on Medical Student Education in Pediatrics' curriculum was not used because it did not differentiate between inpatient and ambulatory contexts. A preliminary list of 93 diagnoses was developed from the table of contents of Pediatric Hospital Medicine.10 This list was distributed to the 26 clinical faculty members in the Divisions of Pediatric Inpatient Medicine and General Pediatrics who were asked to select the 10 most important diagnoses. Surveys were numerically coded to permit 1 reminder.

The survey had a response rate of 92.3% (24 of 26 surveys). One survey was excluded because the respondent significantly deviated from the instructions. The 10 key diagnoses and the percentages of respondents who selected each individual diagnosis are: asthma (100%), febrile infant (95.6%), diarrhea and dehydration (91.3%), bronchiolitis (78.2%), diabetes mellitus and diabetic ketoacidemia (60.9%), failure to thrive (56.5%), urinary tract infections (52.1%), pneumonia (47.8%), upper airway infections such as croup (43.5%), and seizures and status epilepticus (43.5%).

Two of the authors independently coded the diagnoses on the students' patient logs in terms of these 93 diagnoses. The authors were blinded to the students' service assignment. As many students reported more than 1 diagnosis, the authors prioritized primary, secondary, and tertiary diagnoses to simplify the evaluation. The most likely cause of admission was listed as the primary diagnosis. If the authors could not reconcile divergent views, a third party was consulted.

Student Performance

Students were evaluated by both the attending physician(s) and senior resident(s) using a standardized evaluation form available from the corresponding author. The evaluation contained 18 items in 7 categories: data gathering, data recording/reporting, knowledge, data interpretation, clinical performance, professional attitudes, and professional demeanor. The student was rated exceptional, above expectations, meets expectations, below expectations, unacceptable, or not observed on each item. A short narrative description illustrated each rating. The ratings were converted to a 5‐point scale, with exceptional being 5. If the evaluator marked the line between 2 ratings, it was recorded as half. When multiple attendings or residents evaluated a student, the scores for a given item were collapsed into an average score.

Students also completed a NBME pediatric subject examination on the last day of their rotation.

Additionally, students were requested to complete a questionnaire during the final week of the clerkship. The items on the questionnaire were meant to access students' perceptions of the quality of their attendings' and residents' teaching, a potentially confounding variable. The survey was piloted on a group of similar subjects. Informed consent was obtained for survey completion. The survey was anonymous and required approximately 7 minutes to complete.

Resource Utilization

Last, resource utilization data, length of stay and total charges, for the 4 most common primary diagnoses were compared between the intervention and the traditional services. The 4 most common primary diagnoses and the percentage of total diagnoses (n = 2047) that each represents were bronchiolitis, 13%; febrile infant, 8.6%; pneumonia, 7.1%; and asthma, 6.5% (the diagnosis other accounted for 12% of the total diagnoses). Unique patient identifiers were used to obtain length of stay and total charges from the hospital's database. All‐Patient‐Refined Diagnosis‐Related Groups Severity of Illness (APR‐DRG‐SOI) were also obtained and used to construct multivariate models. Patients who were admitted to the pediatric intensive care unit (PICU) were excluded from the analysis.

Statistical Analysis

Statistical analyses were conducted and frequencies and percentages were calculated using Stata SE version 8.0 (College Station, TX). For all interval and ratio‐scaled variables, distributions were tested for normality using the Shapiro‐Wilks test to determine whether to use parametric or nonparametric statistical tests. For distributions meeting the normality assumption, the unpaired t test was used to compare the intervention service with traditional services. Where the normality assumption was not met, the Mann‐Whitney test was used. Categorically scaled data were compared using Pearson's chi‐square test. The standardized mean differences, reported as d values, were calculated to determine the effect size. Small, medium, and large effect sizes were defined as d values of 0.20, 0.50, and 0.80, respectively.11 Teaching quality, an effect modifier, was entered as a covariate into a linear regression model. Analyses of length of stay and total charges were conducted using multivariate linear regression controlling for patient age and severity of illness.

This study was approved by the University of Utah and Primary Children's Medical Center's Institutional Review Board.

Process Measures

Overall, 96.6% of students (196 of 203) submitted patient logs; 97.7% of students (125 of 128) on the intervention service and 94.7% of students (71 of 75) on the traditional services. The students on the intervention service admitted a median of 10 patients, whereas the students on the traditional services admitted a median of 11 patients (d = 0.45, P < .01). Age data were recorded on 137 patient logs (69.9% of submitted logs, 72.0% of students on the intervention service vs. 66.2% of students on the traditional services). The percentage of students who saw at least 1 newborn (birth‐23 months), child (2‐12 years), and adolescent (12‐18 years) was 34.8% on the intervention service and 33.3% on the traditional services (P = .87) (Table 2).

Students on the intervention service encountered, on average, a larger number of the 10 key diagnoses (4.4 vs. 3.6, d = 0.48, P < .01) and a higher percentage of their patients had clinical conditions among the key diagnoses (59.3 vs. 46.8, d = 0.62, P < .01). To determine if this higher percentage was the result of admitting multiple patients with the same diagnosis, we examined the percentage of unique primary diagnosesthe number of different primary diagnoses divided by the total number of patientsand found no differences (Table 2).

Student Performance

The faculty and resident evaluations of the students showed statistically significant differences between those in the intervention service and those in the traditional services in only 2 of the 18 items. These items were analysis in the data interpretation category (3.81 vs. 3.64, d = 0.35, P = .02) and patient interaction in the professional demeanor category (3.89 vs. 3.76, d = 0.31, P < .05). Both differences favored the intervention service. There were no statistical differences by service in student performance on the NBME subject examination (73.2 vs. 72.3, P = .39).

Student Satisfaction

Overall, 87.2% of students (177 of 203) completed the survey; 87.5% of students (112 of 124) on the intervention service and 86.7% of students (65 of 75) on the traditional services. The students on the intervention service both had a more positive overall attitude about their rotation and were more likely to find it a satisfying educational experience. Students on the intervention service also reported greater participation in patient care. Effect sizes ranged from small to medium (Table 3). The internal consistency of answers about participation in patient care was high (Pearson correlation coefficient r = 0.80).

Students on the intervention service rated the teaching of their attendings, but not of their residents, higher than did students on the traditional services. Controlling for the perceived quality of the attending, 3 of 6 satisfaction outcomes remained statistically significant: role on rotation (P < .01), input into patient care decisions (P < .01), and direct responsibility for patient care (P = .04). Students on both services believed they were appropriately supervised (P = .19). Despite the students on the traditional services on average admitting more patients, there was no significant difference by service in the students' rating of patient load (P = .33).

Career Choice

The odds ratio and 95% confidence interval for students enrolling in a pediatric subinternship was 1.94 (0.83‐4.49) and matching in a pediatric residency was 2.52 (0.99‐6.37). There were no statistically significant differences by service in the percentage of students enrolling in primary care (pediatric, internal medicine, and family practice) subinternships or residencies (Table 4).

Resource Utilization

One hundred and thirty‐five patients were excluded from the resource utilization analysis (n = 594) because their unique identifiers could not be found or they had been admitted to the PICU. Univariate analysis demonstrated statistically significant differences for patients with asthma, but not patients with bronchiolitis, febrile infants, or patients with pneumonia, favoring the intervention service. Patients with asthma admitted to the intervention service had a shorter length of stay (49.9 vs. 70.1 hours, P = .02) and lower total charges ($3600 vs. $4600, P = .02), as shown in Table 5. Of 4 multivariate models controlling for age and severity of illness, each with length of stay and total charges as the dependant variables, length of stay was significantly less for patients with asthma admitted to the intervention service only. Such patients were discharged an average of 23.3 hours earlier than patients with asthma admitted to the traditional services (P = .02).

Methods

The methods used in this study compare favorably with other evaluations of educational interventions. The present study incorporated a randomized controlled design.12 Although several studies of ambulatory clerkships used a randomized design, few randomized all eligible students.7, 8 The others used some form of selection prior to randomization. For example, in the Pangaro et al. study, students selected their clerkship site by lottery, with students selecting a certain site then offered the opportunity to participate in the intervention.6 The present study manifested several additional strengths. Multiple outcomes, including effects on patient care, were evaluated. Moreover, this study had a relatively large intervention group and total sample size compared with those in other medical education studies. Finally, because the intervention service had been in place for several years prior to its evaluation, the confounding influence of difficulties working out its implementation was minimized.

Results

Few studies of ambulatory experiences demonstrated statistically significant, let alone clinically significant, results. Most studies showed no statistically significant differences in student evaluations or examination scores. An exception is Grum et al., who showed improvements on 3 of 5 examinations.4 A few studies have found improved student satisfaction.3 None of the randomized controlled trials demonstrate increases in students matching in internal medicine or primary care residencies.4, 68 In contrast, this study produced statistically or programmatically significant results in process measures, evaluations, satisfaction, and career choices.

Several of our specific findings deserve additional comment. Although the admitting residents were instructed to assign patients to the intervention service based on their acuity and complexity, it is important to examine these residents' actual behavior. Several of our hypotheses were not validated. The students on the intervention service admitted fewer patients and were no more likely to see at least 1 patient in each age category. The admitting resident may have limited the number of patients admitted to the intervention service based on the workload of the supervising resident not that of the student. The supervising resident on the intervention service must round on all the patients, whereas the oversight of patients seen by students on the traditional services is shared with the interns. Having the attending on the intervention service share this supervising responsibility might improve this outcome.

Students on the intervention service had more positive attitudes toward the rotation. In addition, potentially negative attitudes were not manifest. For example, it might be argued that third‐year medical students are not prepared to bear this increased responsibility. However, there was not a significant difference in students' perception of the quality of supervision or the workload.

Although the goal of medical education is the production of competent physicians, it is important that the process not place undo burdens on patients and the health care system. Univariate analysis showed similar resource utilization. It might be contended that the admitting resident assigned the intervention service patients who were less acutely ill. Therefore, we performed multivariate analysis using APR‐DRG‐SOI to control for severity of illness. Of 8 comparisons, the only statistically significant difference, length of stay of patients with asthma, favored the intervention service.

Limitations

Although this study had numerous strengths, it also had several limitations. The primary limitations were lack of generalizability, difficulty in obtaining authentic assessments, the potential difference between statistical and educational significance, and inability to identify which components of the intervention service were responsible for the outcomes. This study's findings may not be generalizable to other institutions. For example, institutions without age or organ systembased teams may not observe increases in the number of key diagnoses encountered in the patients cared for. Regarding the assessments, there may be better measures of clinical competence, such as an objective structured clinical examination (OSCE),13 than those used in this study. However, there were not sufficient resources to implement an OSCE at the end of the rotation.

Some might question whether the statistically significant differences have educational significance. Although that is an important concern, this study should be compared with other educational interventions that found few statistically significant, let alone educationally significant, differences. To address this concern, we calculated effect sizes. The differences in student satisfaction were small to moderate. Although the lower limit of the 95% confidence interval of the odds ratio for matching in a pediatric residency was 0.99, the magnitude was programmatically important.

Finally, this study was an evaluation of an existing program. The authors were unable to control some potential confounders including patient allocation, average daily census, and quality of teaching. For example, Griffith and colleagues have shown that working with the best teachers improves student performance.14 We were not able to randomly assign the faculty among the services, and unequal distribution of better teachers could have biased this study's outcomes. The students on the intervention service rated their attendings, but not their residents, higher than did the students on the other services. However, the linear regression model showed that the perceived quality of the attending did not account for all the differences in student satisfaction. It was not possible to control for this factor in comparing student performance or subinternship or residency selection because the survey, which included the faculty evaluations, was anonymous and therefore could not be linked to the other data sets.

The perceived differences in the quality of teaching may not have been the result of differences in the attendings but instead of differences in the structure of the services. Accessibility is one of the characteristics of excellent clinical teachers.15 The intervention structure may permit faculty to spend more time with students, and this may increase the perceived quality of the teaching. However, it is not possible to resolve this issue with the available data.

Process Measures

Students were required to submit logs at the end of their rotations, recording patients' names, ages, diagnoses, and admission dates. The accuracy and completeness of these logs were not independently verified.

As there was no authoritative list of key diagnoses third‐year medical students should encounter in the patients they care for during their inpatient rotations, we relied on expert opinion at our institution. The Council on Medical Student Education in Pediatrics' curriculum was not used because it did not differentiate between inpatient and ambulatory contexts. A preliminary list of 93 diagnoses was developed from the table of contents of Pediatric Hospital Medicine.10 This list was distributed to the 26 clinical faculty members in the Divisions of Pediatric Inpatient Medicine and General Pediatrics who were asked to select the 10 most important diagnoses. Surveys were numerically coded to permit 1 reminder.

The survey had a response rate of 92.3% (24 of 26 surveys). One survey was excluded because the respondent significantly deviated from the instructions. The 10 key diagnoses and the percentages of respondents who selected each individual diagnosis are: asthma (100%), febrile infant (95.6%), diarrhea and dehydration (91.3%), bronchiolitis (78.2%), diabetes mellitus and diabetic ketoacidemia (60.9%), failure to thrive (56.5%), urinary tract infections (52.1%), pneumonia (47.8%), upper airway infections such as croup (43.5%), and seizures and status epilepticus (43.5%).

Two of the authors independently coded the diagnoses on the students' patient logs in terms of these 93 diagnoses. The authors were blinded to the students' service assignment. As many students reported more than 1 diagnosis, the authors prioritized primary, secondary, and tertiary diagnoses to simplify the evaluation. The most likely cause of admission was listed as the primary diagnosis. If the authors could not reconcile divergent views, a third party was consulted.

Student Performance

Students were evaluated by both the attending physician(s) and senior resident(s) using a standardized evaluation form available from the corresponding author. The evaluation contained 18 items in 7 categories: data gathering, data recording/reporting, knowledge, data interpretation, clinical performance, professional attitudes, and professional demeanor. The student was rated exceptional, above expectations, meets expectations, below expectations, unacceptable, or not observed on each item. A short narrative description illustrated each rating. The ratings were converted to a 5‐point scale, with exceptional being 5. If the evaluator marked the line between 2 ratings, it was recorded as half. When multiple attendings or residents evaluated a student, the scores for a given item were collapsed into an average score.

Students also completed a NBME pediatric subject examination on the last day of their rotation.

Additionally, students were requested to complete a questionnaire during the final week of the clerkship. The items on the questionnaire were meant to access students' perceptions of the quality of their attendings' and residents' teaching, a potentially confounding variable. The survey was piloted on a group of similar subjects. Informed consent was obtained for survey completion. The survey was anonymous and required approximately 7 minutes to complete.

Resource Utilization

Last, resource utilization data, length of stay and total charges, for the 4 most common primary diagnoses were compared between the intervention and the traditional services. The 4 most common primary diagnoses and the percentage of total diagnoses (n = 2047) that each represents were bronchiolitis, 13%; febrile infant, 8.6%; pneumonia, 7.1%; and asthma, 6.5% (the diagnosis other accounted for 12% of the total diagnoses). Unique patient identifiers were used to obtain length of stay and total charges from the hospital's database. All‐Patient‐Refined Diagnosis‐Related Groups Severity of Illness (APR‐DRG‐SOI) were also obtained and used to construct multivariate models. Patients who were admitted to the pediatric intensive care unit (PICU) were excluded from the analysis.

Statistical Analysis

Statistical analyses were conducted and frequencies and percentages were calculated using Stata SE version 8.0 (College Station, TX). For all interval and ratio‐scaled variables, distributions were tested for normality using the Shapiro‐Wilks test to determine whether to use parametric or nonparametric statistical tests. For distributions meeting the normality assumption, the unpaired t test was used to compare the intervention service with traditional services. Where the normality assumption was not met, the Mann‐Whitney test was used. Categorically scaled data were compared using Pearson's chi‐square test. The standardized mean differences, reported as d values, were calculated to determine the effect size. Small, medium, and large effect sizes were defined as d values of 0.20, 0.50, and 0.80, respectively.11 Teaching quality, an effect modifier, was entered as a covariate into a linear regression model. Analyses of length of stay and total charges were conducted using multivariate linear regression controlling for patient age and severity of illness.

This study was approved by the University of Utah and Primary Children's Medical Center's Institutional Review Board.

Process Measures

Overall, 96.6% of students (196 of 203) submitted patient logs; 97.7% of students (125 of 128) on the intervention service and 94.7% of students (71 of 75) on the traditional services. The students on the intervention service admitted a median of 10 patients, whereas the students on the traditional services admitted a median of 11 patients (d = 0.45, P < .01). Age data were recorded on 137 patient logs (69.9% of submitted logs, 72.0% of students on the intervention service vs. 66.2% of students on the traditional services). The percentage of students who saw at least 1 newborn (birth‐23 months), child (2‐12 years), and adolescent (12‐18 years) was 34.8% on the intervention service and 33.3% on the traditional services (P = .87) (Table 2).

Students on the intervention service encountered, on average, a larger number of the 10 key diagnoses (4.4 vs. 3.6, d = 0.48, P < .01) and a higher percentage of their patients had clinical conditions among the key diagnoses (59.3 vs. 46.8, d = 0.62, P < .01). To determine if this higher percentage was the result of admitting multiple patients with the same diagnosis, we examined the percentage of unique primary diagnosesthe number of different primary diagnoses divided by the total number of patientsand found no differences (Table 2).

Student Performance

The faculty and resident evaluations of the students showed statistically significant differences between those in the intervention service and those in the traditional services in only 2 of the 18 items. These items were analysis in the data interpretation category (3.81 vs. 3.64, d = 0.35, P = .02) and patient interaction in the professional demeanor category (3.89 vs. 3.76, d = 0.31, P < .05). Both differences favored the intervention service. There were no statistical differences by service in student performance on the NBME subject examination (73.2 vs. 72.3, P = .39).

Student Satisfaction

Overall, 87.2% of students (177 of 203) completed the survey; 87.5% of students (112 of 124) on the intervention service and 86.7% of students (65 of 75) on the traditional services. The students on the intervention service both had a more positive overall attitude about their rotation and were more likely to find it a satisfying educational experience. Students on the intervention service also reported greater participation in patient care. Effect sizes ranged from small to medium (Table 3). The internal consistency of answers about participation in patient care was high (Pearson correlation coefficient r = 0.80).

Students on the intervention service rated the teaching of their attendings, but not of their residents, higher than did students on the traditional services. Controlling for the perceived quality of the attending, 3 of 6 satisfaction outcomes remained statistically significant: role on rotation (P < .01), input into patient care decisions (P < .01), and direct responsibility for patient care (P = .04). Students on both services believed they were appropriately supervised (P = .19). Despite the students on the traditional services on average admitting more patients, there was no significant difference by service in the students' rating of patient load (P = .33).

Career Choice

The odds ratio and 95% confidence interval for students enrolling in a pediatric subinternship was 1.94 (0.83‐4.49) and matching in a pediatric residency was 2.52 (0.99‐6.37). There were no statistically significant differences by service in the percentage of students enrolling in primary care (pediatric, internal medicine, and family practice) subinternships or residencies (Table 4).

Resource Utilization

One hundred and thirty‐five patients were excluded from the resource utilization analysis (n = 594) because their unique identifiers could not be found or they had been admitted to the PICU. Univariate analysis demonstrated statistically significant differences for patients with asthma, but not patients with bronchiolitis, febrile infants, or patients with pneumonia, favoring the intervention service. Patients with asthma admitted to the intervention service had a shorter length of stay (49.9 vs. 70.1 hours, P = .02) and lower total charges ($3600 vs. $4600, P = .02), as shown in Table 5. Of 4 multivariate models controlling for age and severity of illness, each with length of stay and total charges as the dependant variables, length of stay was significantly less for patients with asthma admitted to the intervention service only. Such patients were discharged an average of 23.3 hours earlier than patients with asthma admitted to the traditional services (P = .02).

Methods

The methods used in this study compare favorably with other evaluations of educational interventions. The present study incorporated a randomized controlled design.12 Although several studies of ambulatory clerkships used a randomized design, few randomized all eligible students.7, 8 The others used some form of selection prior to randomization. For example, in the Pangaro et al. study, students selected their clerkship site by lottery, with students selecting a certain site then offered the opportunity to participate in the intervention.6 The present study manifested several additional strengths. Multiple outcomes, including effects on patient care, were evaluated. Moreover, this study had a relatively large intervention group and total sample size compared with those in other medical education studies. Finally, because the intervention service had been in place for several years prior to its evaluation, the confounding influence of difficulties working out its implementation was minimized.

Results

Few studies of ambulatory experiences demonstrated statistically significant, let alone clinically significant, results. Most studies showed no statistically significant differences in student evaluations or examination scores. An exception is Grum et al., who showed improvements on 3 of 5 examinations.4 A few studies have found improved student satisfaction.3 None of the randomized controlled trials demonstrate increases in students matching in internal medicine or primary care residencies.4, 68 In contrast, this study produced statistically or programmatically significant results in process measures, evaluations, satisfaction, and career choices.

Several of our specific findings deserve additional comment. Although the admitting residents were instructed to assign patients to the intervention service based on their acuity and complexity, it is important to examine these residents' actual behavior. Several of our hypotheses were not validated. The students on the intervention service admitted fewer patients and were no more likely to see at least 1 patient in each age category. The admitting resident may have limited the number of patients admitted to the intervention service based on the workload of the supervising resident not that of the student. The supervising resident on the intervention service must round on all the patients, whereas the oversight of patients seen by students on the traditional services is shared with the interns. Having the attending on the intervention service share this supervising responsibility might improve this outcome.

Students on the intervention service had more positive attitudes toward the rotation. In addition, potentially negative attitudes were not manifest. For example, it might be argued that third‐year medical students are not prepared to bear this increased responsibility. However, there was not a significant difference in students' perception of the quality of supervision or the workload.

Although the goal of medical education is the production of competent physicians, it is important that the process not place undo burdens on patients and the health care system. Univariate analysis showed similar resource utilization. It might be contended that the admitting resident assigned the intervention service patients who were less acutely ill. Therefore, we performed multivariate analysis using APR‐DRG‐SOI to control for severity of illness. Of 8 comparisons, the only statistically significant difference, length of stay of patients with asthma, favored the intervention service.

Limitations

Although this study had numerous strengths, it also had several limitations. The primary limitations were lack of generalizability, difficulty in obtaining authentic assessments, the potential difference between statistical and educational significance, and inability to identify which components of the intervention service were responsible for the outcomes. This study's findings may not be generalizable to other institutions. For example, institutions without age or organ systembased teams may not observe increases in the number of key diagnoses encountered in the patients cared for. Regarding the assessments, there may be better measures of clinical competence, such as an objective structured clinical examination (OSCE),13 than those used in this study. However, there were not sufficient resources to implement an OSCE at the end of the rotation.

Some might question whether the statistically significant differences have educational significance. Although that is an important concern, this study should be compared with other educational interventions that found few statistically significant, let alone educationally significant, differences. To address this concern, we calculated effect sizes. The differences in student satisfaction were small to moderate. Although the lower limit of the 95% confidence interval of the odds ratio for matching in a pediatric residency was 0.99, the magnitude was programmatically important.

Finally, this study was an evaluation of an existing program. The authors were unable to control some potential confounders including patient allocation, average daily census, and quality of teaching. For example, Griffith and colleagues have shown that working with the best teachers improves student performance.14 We were not able to randomly assign the faculty among the services, and unequal distribution of better teachers could have biased this study's outcomes. The students on the intervention service rated their attendings, but not their residents, higher than did the students on the other services. However, the linear regression model showed that the perceived quality of the attending did not account for all the differences in student satisfaction. It was not possible to control for this factor in comparing student performance or subinternship or residency selection because the survey, which included the faculty evaluations, was anonymous and therefore could not be linked to the other data sets.

The perceived differences in the quality of teaching may not have been the result of differences in the attendings but instead of differences in the structure of the services. Accessibility is one of the characteristics of excellent clinical teachers.15 The intervention structure may permit faculty to spend more time with students, and this may increase the perceived quality of the teaching. However, it is not possible to resolve this issue with the available data.