User login
All together now: Impact of a regionalization and bedside rounding initiative on the efficiency and inclusiveness of clinical rounds
Attending rounds at academic medical centers are often disconnected from patients and non-physician care team members. Time spent bedside is consistently less than one third of total rounding time, with observational studies reporting a range of 9% to 33% over the past several decades.1-8 Rounds are often conducted outside patient rooms, denying patients, families, and nurses the opportunity to participate and offer valuable insights. Lack of bedside rounds thus limits patient and family engagement, patient input into the care plan, teaching of the physical examination, and communication and collaboration with nurses. In one study, physicians and nurses on rounds engaged in interprofessional communication in only 12% of patient cases.1 Studies have found interdisciplinary bedside rounds have several benefits, including subjectively improved communication and teamwork between physicians and nurses; increased patient satisfaction, including feeling more cared for by the medical team; and decreased length of stay and costs of care.2-10
However, there are many barriers to conducting interdisciplinary bedside rounds at large academic medical centers. Patients cared for by a single medical team are often geographically dispersed to several nursing units, and nurses are unable to predict when physicians will round on their patients. This situation limits nursing involvement on rounds and keeps doctors and nurses isolated from each other.2 Regionalization of care teams reduces this fragmentation by facilitating more interaction among doctors, patients, families, and nursing staff.
There are few data on how regionalized patients and interdisciplinary bedside rounds affect rounding time and the nature of rounds. This information is needed to understand how these structural changes mediate their effects, whether other steps are required to optimize outcomes, and how to maximize efficiency. We used time-motion analysis (TMA) to investigate how regionalization of medical teams, encouragement of bedside rounding, and systematic inclusion of nurses on ward rounds affect amount of time spent with patients, nursing presence on rounds, and total rounding time.
METHODS
Setting
This prospective interventional study, approved by the Institutional Review Board of Partners HealthCare, was conducted on the general medical wards at Brigham and Women’s Hospital, an academic 793-bed tertiary-care center in Boston, Massachusetts. Housestaff teams consist of 1 attending, 1 resident, and 2 interns with or without a medical student. Before June 20, 2013, daily rounds on medical inpatients were conducted largely on the patient unit but outside patient rooms. After completing most of a rounding discussion outside a patient’s room, the team might walk in to examine or speak with the patient. A typical medical team had patients dispersed over 7 medical units on average, and over as many as 13. As nurses were unit based, they did not consistently participate in rounds.
Intervention
In June 2013, as part of a general medical service care redesign initiative, the general medical teams were regionalized to specific inpatient units. The goal was to have teams admit patients predominantly to the team’s designated unit and to have all patients on a unit be cared for by the unit’s assigned team as often as possible, with an 85% goal for both. Toward those ends, the admitting structure was changed from a traditional 4-day call cycle to daily admitting for all teams, based on each unit’s bed availability.11
Teams were also expected to conduct rounds with nurses, and a system for facilitating these rounds was established. As physician and nurse care teams were now geographically co-located, it became possible for residents and nurses to check a rounding sheet for the planned patient rounding order, which had been set by the resident and nurse-in-charge before rounds. No more than about 5 minutes was needed to prepare each day’s order. The rounding sheet prioritized sick patients, newly admitted patients, and planned morning discharges, but patients were also always grouped by nurse. For example, the physician team rounded with the first nurse on all 3 of a nurse’s patients, and then proceeded to the next group of 3 patients with the next nurse, until all patients were seen.
Teams were encouraged to conduct patient- and family-centered rounds exclusively at bedside, except when bedside rounding was thought to be detrimental to a patient (eg, one with delirium). After an intern’s bedside presentation, which included a brief summary and details about overnight events and vital signs, the concerns of the patient, family, and nurse were shared, a focused physical examination performed, relevant data (eg, laboratory test results and imaging studies) reviewed, and the day’s plan formulated. The entire team, including the attending, was expected to have read new patients’ admission notes before rounds. Bedside rounds could thus be focused more on patient assessment and patient/family engagement and less on data transfer.
Several actions were taken to facilitate these changes. Residents, attendings, nurses, and other interdisciplinary team members participated in a series of focus groups and conferences to define workflows and share best practices for patient- and family-centered bedside rounds. Tips on bedside rounding were included in a general medicine rotation guidebook made available to residents and attendings. At the beginning of each post-intervention general medicine rotation, attendings and residents attended brief orientation sessions to review the new daily schedule, have interdisciplinary huddles, and share expectations for patient- and family-centered bedside rounds. On the general medicine units, new medical directors were hired to partner with existing nursing directors to support adoption of the workflows. Last, an interdisciplinary leadership team was formed to support the care redesign efforts. This team started meeting every 2 weeks.
Study Design
We used a pre–post analysis to study the effects of care redesign. Analysis was performed at the same time of year for 2 consecutive years to control for the stage of training and experience of the housestaff. TMA was performed by trained medical students using computer tablets linked to a customized Microsoft Access database form (Redmond, Washington). The form and the database were designed with specific buttons that, when pressed, recorded the time of particular events, such as the coming and going of each participant, the location of rounds, and the beginning and the end of rounding encounters with a patient. One research assistant using an Access entry form was able to dynamically track all events in real time, as they occurred. We collected data on 4 teams at baseline and 5 teams after the intervention. Each of the 4 baseline teams was followed for 4 consecutive weekdays—16 rounds total, April-June 2013—to capture the 4-day call cycle. Each of the 5 post-intervention teams was followed for 5 consecutive weekdays—25 rounds total, April–June 2014—to capture the 5-day cycle. (Because of technical difficulties, data from 1 rounding session were not captured.) For inclusion in the statistical analyses, TMA captured 166 on-service patients before the intervention and 304 afterward. Off-service patients, those with an attending other than the team attending, were excluded because their rounds were conducted separately.
We examined 2 primary outcomes, the proportion of time each clinical team member was present on rounds and the proportion of bedside rounding time. Secondary outcomes were round duration, rounding time per patient, and total non-patient time per rounding session (total rounding time minus total patient time).
Statistical Analysis
TMA data were organized in an Access database and analyzed with SAS Version 9.3 (SAS Institute, Cary, North Carolina). We analyzed the data by round session as well as by patient.
Data are presented as means with standard deviations, medians with interquartile ranges, and proportions, as appropriate. For analyses by round session, we used unadjusted linear regression; for patient-level analyses, we used general estimating equations to adjust for clustering of patients within each session; for nurse presence during any part of a round by patient, we used a χ2 test. Total non-patient time per round session was compared with use of patient-clustered general estimating equations using a γ distribution to account for the non-normality of the data.
RESULTS
Patient and Care Team Characteristics
Over the first year of the initiative, 85% of a team’s patients were on their assigned unit, and 87% of a unit’s patients were with the assigned team. Census numbers were 10.4 patients per general medicine team in April-June 2013 and 12.7 patients per team in April-June 2014, a 22% increase after care redesign. There were no statistically significant differences in patient characteristics, including age, sex, race, language, admission source, and comorbidity measure (Elixhauser score), between the pre-intervention and post-intervention study periods, except for a slightly higher proportion of patients admitted from home and fewer patients admitted directly from clinic (Table 1).
Primary Outcomes
Mean proportion of time the nurse was present on rounds per round session increased significantly (P < 0.001), from 24.1% to 67.8% (Figure 1A, Table 2). For individual patient encounters, the increased overall nursing presence was attributable to having more nurses on rounds and having nurses present for a larger proportion of individual rounding encounters (Figure 1B, Table 2). Nurses were present for at least some part of rounds for 53% of patients before the intervention and 93% afterward (P < 0.001). Mean proportion of round time by each of the 2 interns on each team decreased from 59.6% to 49.6% (P = 0.007).
Total bedside rounding time increased significantly ( P < 0.001), from 39.9% before the intervention to 55.8% afterward (Table 2). Meanwhile, percentage of rounding time spent on the unit but outside patient rooms decreased significantly ( P = 0.004), from 55.2% to 42.2%, as did rounding time on a unit completely different from the patient’s (4.9% before intervention, 2.0% afterward; P = 0.03). Again, patient-level results were similar (Figure 2, Table 2), but the decreased time spent on the unit, outside the patient rooms, was not significant.
Secondary Outcomes
Total rounding time decreased significantly, from a mean of 182 minutes (3.0 hours) at baseline to a mean of 146 minutes (2.4 hours) after the intervention, despite the higher post-intervention census. (When adjusted for patient census, the difference increased from 35.5 to 53.8 minutes; Table 2.) Mean rounding time per patient decreased significantly, from 14.7 minutes at baseline to 10.5 minutes after the intervention. For newly admitted patients, mean rounding time per patient decreased from 30.0 minutes before implementation to 16.3 minutes afterward. Mean rounding time also decreased, though much less, for subsequent-day patients (Table 2). For both new and existing patients, the decrease in rounding time largely was a reduction in time spent rounding outside patient rooms, with minimal impact on bedside time (Table 2). Mean time nurses were present during a patient’s rounds increased significantly, from 4.5 to 8.0 minutes (Table 2). Total nurse rounding time increased from 45.1 minutes per session to 98.8 minutes. Rounding time not related to patient discussion or evaluation decreased from 22.7 minutes per session to 13.3 minutes ( P = 0.003).
DISCUSSION
TMA of our care redesign initiative showed that this multipronged intervention, which included team regionalization, encouragement of bedside rounding with nurses, call structure changes, and attendings’ reading of admission notes before rounds, resulted in an increased proportion of rounding time spent with patients and an increased proportion of time nurses were present on rounds. Secondarily, round duration decreased even as patient census increased.
Regionalized teams have been found to improve interdisciplinary communication.1 The present study elaborates on that finding by demonstrating a dramatic increase in nursing presence on rounds, likely resulting from the unit’s use of rounding schedules and nurses’ prioritization of rounding orders, both of which were made possible by geographic co-localization. Other research has noted that one of the most significant barriers to interdisciplinary rounds is difficulty coordinating the start times of physician/nurse bedside rounding encounters. The system we have studied directly addresses this difficulty.9 Of note, nursing presence on rounds is necessary but not sufficient for true physician–nurse collaboration and effective communication,1 as reflected in a separate study of the intervention showing no significant difference in the concordance of the patient care plan between nurses and physicians before and after regionalization.12 Additional interventions may be needed to ensure that communication during bedside rounds is effective.
Our regionalized teams spent a significantly higher proportion of rounding time bedside, likely because of a cultural shift in expectations and the increased convenience of seeing patients on the team’s unit. Nevertheless, bedside time was not 100%. Structural barriers (eg, patients off-unit for dialysis) and cultural barriers likely contributed to the less than full adoption of bedside rounding. As described previously, cultural barriers to bedside rounding include trainees’ anxiety about being questioned in front of patients, the desire to freely exchange academic ideas in a conference room, and attendings’ doubts about their bedside teaching ability.1,9,13 Bedside rounds provide an important opportunity to apply the principles of patient- and family-centered care, including promotion of dignity and respect, information sharing, and collaboration. Thus, overcoming the concerns of housestaff and attendings and helping them feel prepared for bedside rounds can benefit the patient experience. More attention should be given to these practices as these types of interventions are implemented at Brigham and Women’s Hospital and elsewhere.1,13-15
Another primary concern about interdisciplinary bedside rounding is the perception that it takes more time.9 Therefore, it was important for us to measure round duration as a balancing measure to be considered for our intervention. Fortunately, we found round duration decreased with regionalization and encouragement of bedside rounding. This decrease was driven largely by a significant decrease in mean rounding time per new patient, which may be attributable at least in part to setting expectations that attendings and residents will read admission notes before rounds and that interns will summarize rather than recount information from admission notes. However, we also found rounding time decreases for subsequent-day patients, suggesting an underlying time savings. Spending a larger proportion of time bedside may therefore result in more efficient rounds. Bedside presentations can reduce redundancies, such as discussing a patient’s case outside his or her room and subsequently walking in and going over much of the same information with the patient. Our model de-emphasizes data transfer in favor of discussion of care plans. There was also a decrease in non-patient time, likely reflecting reduced transit time for regionalized teams. This decrease aligns with a recent finding that bedside rounding was at least as efficient as rounding outside the room.16
Of note, though a larger percentage of time was spent bedside after implementation of the care redesign, the absolute amount of bedside time did not change significantly. Our data showed that, even with shorter rounds, the same amount of absolute time can be spent bedside, face to face with the patient, by increasing the proportion of bedside rounding time. In other words, teams on average did not spend more time with patients, though the content and the structure of those encounters may have changed. This finding may be attributable to eliminating redundancy, forgoing the outside-the-room discussion, and thus the largest time reductions were realized there. In addition, teams incompletely adopted beside rounds, as reflected in the data. We expect that, with more complete adoption, an even larger proportion of time will be spent bedside, and absolute time bedside might increase as a result.
An unexpected result of the care redesign was that interns’ proportion of rounding time decreased after the intervention. This decrease most likely is attributable to interns’ being less likely to participate in rounds for a co-intern’s patient, and to their staying outside that patient’s room to give themselves more time to advance the care of their own patients. Before the intervention, when more rounding time was spent outside patient rooms, interns were more likely to join rounds for their co-intern’s patients because they could easily break away, as needed, to continue care of their own patients. The resident is now encouraged to use the morning huddle to identify which patients likely have the most educational value, and both interns are expected to join the bedside rounds for these patients.
This study had a few limitations. First, the pre–post design made it difficult to exclude the possibility that other temporal changes may have affected outcomes, though we did account for time-of-year effects by aligning our data-collection phases. In addition, the authors, including the director of the general medical service, are unaware of any co-interventions during the study period. Second, the multipronged intervention included care team regionalization, encouragement of bedside rounding with nurses, call structure changes (from 4 days to daily admitting), and attendings’ reading of admission notes before rounds. Thus, parsing which component(s) contributed to the results was difficult, though all the changes instituted likely were necessary for system redesign. For example, regionalization of clinicians to unit-based teams was made possible by switching to a daily admitting system.
Time that team members spent preparing for rounds was not recorded before or after the intervention. Thus, the decrease in total rounding time could have been accompanied by an increase in time spent preparing for rounds. However, admission notes were available in our electronic medical record before and after the intervention, and most residents and attendings were already reading them pre-intervention. After the intervention, pre-round note reading was more clearly defined as an expectation, and we were able to set the expectation that interns should use their presentations to summarize rather than recount information. In addition, in the post-intervention period, we did not include time spent preparing rounding orders; as already noted, however, preparation took only 5 minutes per day. Also, we did not analyze the content or the quality of the discussion on rounds, but simply recorded who was present where and when. Regarding the effect of the intervention on patient care, results were mixed. As reported in 2016, we saw no difference in frequency of adverse events with this intervention.12 However, a more sensitive measure of adverse events—used in a study on handoffs—showed our regionalization efforts had an additive effect on reducing overnight adverse events.17Researchers should now focus on the effects of care redesign on clinical outcomes, interdisciplinary care team communication, patient engagement and satisfaction, provider opinions of communication, workflow, patient care, and housestaff education. Our methodology can be used as a model to link structure, process, and outcome related to rounds and thereby better understand how best to optimize patient care and efficiency. Additional studies are needed to analyze the content of rounds and their association with patient and educational outcomes. Last, it will be important to conduct a study to see if the effects we have identified can be sustained. Such a study is already under way.
In conclusion, creating regionalized care teams and encouraging focused bedside rounds increased the proportion of bedside time and the presence of nurses on rounds. Rounds were shorter despite higher patient census. TMA revealed that regionalized care teams and bedside rounding at a large academic hospital are feasible, and are useful in establishing the necessary structures for increasing physician–nurse and provider–patient interactions.
Acknowledgments
The authors acknowledge Dr. Stan Ashley, Dr. Jacqueline Somerville, and Sheila Harris for their support of the regionalization initiative.
Disclosures
Dr. Schnipper received funding from Sanofi-aventis to conduct an investigator-initiated study to implement and evaluate a multi-faceted intervention to improve transitions of care in patients discharged home on insulin. The study was also supported by funding from the Marshall A. Wolf Medical Education Fund, Brigham and Women’s Hospital, and Dr. Stan Ashley, Chief Medical Officer, Brigham and Women’s Hospital. Some of the content of this article was orally presented at the annual meeting of the Society of Hospital Medicine; March 29-April 1, 2015; National Harbor, MD.
1. Crumlish CM, Yialamas MA, McMahon GT. Quantification of bedside teaching by an academic hospitalist group. J Hosp Med. 2009;4(5):304-307. PubMed
2. Gonzalo JD, Masters PA, Simons RJ, Chuang CH. Attending rounds and bedside case presentations: medical student and medicine resident experiences and attitudes. Teach Learn Med. 2009;21(2):105-110. PubMed
3. Elliot DL, Hickam DH. Attending rounds on in-patient units: differences between medical and non-medical services. Med Educ. 1993;27(6):503-508. PubMed
4. Payson HE, Barchas JD. A time study of medical teaching rounds. N Engl J Med. 1965;273(27):1468-1471. PubMed
5. Tremonti LP, Biddle WB. Teaching behaviors of residents and faculty members. J Med Educ. 1982;57(11):854-859. PubMed
6. Miller M, Johnson B, Greene HL, Baier M, Nowlin S. An observational study of attending rounds. J Gen Intern Med. 1992;7(6):646-648. PubMed
7. Collins GF, Cassie JM, Daggett CJ. The role of the attending physician in clinical training. J Med Educ. 1978;53(5):429-431. PubMed
8. Ward DR, Ghali WA, Graham A, Lemaire JB. A real-time locating system observes physician time-motion patterns during walk-rounds: a pilot study. BMC Med Educ. 2014;14:37. PubMed
9. Gonzalo JD, Kuperman E, Lehman E, Haidet P. Bedside interprofessional rounds: perceptions of benefits and barriers by internal medicine nursing staff, attending physicians, and housestaff physicians. J Hosp Med. 2014;9(10):646-651. PubMed
10. Stickrath C, Noble M, Prochazka A, et al. Attending rounds in the current era: what is and is not happening. JAMA Intern Med. 2013;173(12):1084-1089. PubMed
11. Boxer R, Vitale M, Gershanik EF, et al. 5th time’s a charm: creation of unit-based care teams in a high occupancy hospital [abstract]. J Hosp Med. 2015;10(suppl 2).
12. Mueller SK, Schnipper JL, Giannelli K, Roy CL, Boxer R. Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services. J Hosp Med. 2016;11(9):620-627. PubMed
13. Chauke HL, Pattinson RC. Ward rounds—bedside or conference room? S Afr Med J. 2006;96(5):398-400. PubMed
14. Wang-Cheng RM, Barnas GP, Sigmann P, Riendl PA, Young MJ. Bedside case presentations: why patients like them but learners don’t. J Gen Intern Med. 1989;4(4):284-287. PubMed
15. Lehmann LS, Brancati FL, Chen MC, Roter D, Dobs AS. The effect of bedside case presentations on patients’ perceptions of their medical care. N Engl J Med. 1997;336(16):1150-1155. PubMed
16. Gonzalo JD, Chuang CH, Huang G, Smith C. The return of bedside rounds: an educational intervention. J Gen Intern Med. 2010;25(8):792-798. PubMed
17. Mueller SK, Yoon C, Schnipper JL. Association of a web-based handoff tool with rates of medical errors. JAMA Intern Med. 2016;176(9):1400-1402. PubMed
Attending rounds at academic medical centers are often disconnected from patients and non-physician care team members. Time spent bedside is consistently less than one third of total rounding time, with observational studies reporting a range of 9% to 33% over the past several decades.1-8 Rounds are often conducted outside patient rooms, denying patients, families, and nurses the opportunity to participate and offer valuable insights. Lack of bedside rounds thus limits patient and family engagement, patient input into the care plan, teaching of the physical examination, and communication and collaboration with nurses. In one study, physicians and nurses on rounds engaged in interprofessional communication in only 12% of patient cases.1 Studies have found interdisciplinary bedside rounds have several benefits, including subjectively improved communication and teamwork between physicians and nurses; increased patient satisfaction, including feeling more cared for by the medical team; and decreased length of stay and costs of care.2-10
However, there are many barriers to conducting interdisciplinary bedside rounds at large academic medical centers. Patients cared for by a single medical team are often geographically dispersed to several nursing units, and nurses are unable to predict when physicians will round on their patients. This situation limits nursing involvement on rounds and keeps doctors and nurses isolated from each other.2 Regionalization of care teams reduces this fragmentation by facilitating more interaction among doctors, patients, families, and nursing staff.
There are few data on how regionalized patients and interdisciplinary bedside rounds affect rounding time and the nature of rounds. This information is needed to understand how these structural changes mediate their effects, whether other steps are required to optimize outcomes, and how to maximize efficiency. We used time-motion analysis (TMA) to investigate how regionalization of medical teams, encouragement of bedside rounding, and systematic inclusion of nurses on ward rounds affect amount of time spent with patients, nursing presence on rounds, and total rounding time.
METHODS
Setting
This prospective interventional study, approved by the Institutional Review Board of Partners HealthCare, was conducted on the general medical wards at Brigham and Women’s Hospital, an academic 793-bed tertiary-care center in Boston, Massachusetts. Housestaff teams consist of 1 attending, 1 resident, and 2 interns with or without a medical student. Before June 20, 2013, daily rounds on medical inpatients were conducted largely on the patient unit but outside patient rooms. After completing most of a rounding discussion outside a patient’s room, the team might walk in to examine or speak with the patient. A typical medical team had patients dispersed over 7 medical units on average, and over as many as 13. As nurses were unit based, they did not consistently participate in rounds.
Intervention
In June 2013, as part of a general medical service care redesign initiative, the general medical teams were regionalized to specific inpatient units. The goal was to have teams admit patients predominantly to the team’s designated unit and to have all patients on a unit be cared for by the unit’s assigned team as often as possible, with an 85% goal for both. Toward those ends, the admitting structure was changed from a traditional 4-day call cycle to daily admitting for all teams, based on each unit’s bed availability.11
Teams were also expected to conduct rounds with nurses, and a system for facilitating these rounds was established. As physician and nurse care teams were now geographically co-located, it became possible for residents and nurses to check a rounding sheet for the planned patient rounding order, which had been set by the resident and nurse-in-charge before rounds. No more than about 5 minutes was needed to prepare each day’s order. The rounding sheet prioritized sick patients, newly admitted patients, and planned morning discharges, but patients were also always grouped by nurse. For example, the physician team rounded with the first nurse on all 3 of a nurse’s patients, and then proceeded to the next group of 3 patients with the next nurse, until all patients were seen.
Teams were encouraged to conduct patient- and family-centered rounds exclusively at bedside, except when bedside rounding was thought to be detrimental to a patient (eg, one with delirium). After an intern’s bedside presentation, which included a brief summary and details about overnight events and vital signs, the concerns of the patient, family, and nurse were shared, a focused physical examination performed, relevant data (eg, laboratory test results and imaging studies) reviewed, and the day’s plan formulated. The entire team, including the attending, was expected to have read new patients’ admission notes before rounds. Bedside rounds could thus be focused more on patient assessment and patient/family engagement and less on data transfer.
Several actions were taken to facilitate these changes. Residents, attendings, nurses, and other interdisciplinary team members participated in a series of focus groups and conferences to define workflows and share best practices for patient- and family-centered bedside rounds. Tips on bedside rounding were included in a general medicine rotation guidebook made available to residents and attendings. At the beginning of each post-intervention general medicine rotation, attendings and residents attended brief orientation sessions to review the new daily schedule, have interdisciplinary huddles, and share expectations for patient- and family-centered bedside rounds. On the general medicine units, new medical directors were hired to partner with existing nursing directors to support adoption of the workflows. Last, an interdisciplinary leadership team was formed to support the care redesign efforts. This team started meeting every 2 weeks.
Study Design
We used a pre–post analysis to study the effects of care redesign. Analysis was performed at the same time of year for 2 consecutive years to control for the stage of training and experience of the housestaff. TMA was performed by trained medical students using computer tablets linked to a customized Microsoft Access database form (Redmond, Washington). The form and the database were designed with specific buttons that, when pressed, recorded the time of particular events, such as the coming and going of each participant, the location of rounds, and the beginning and the end of rounding encounters with a patient. One research assistant using an Access entry form was able to dynamically track all events in real time, as they occurred. We collected data on 4 teams at baseline and 5 teams after the intervention. Each of the 4 baseline teams was followed for 4 consecutive weekdays—16 rounds total, April-June 2013—to capture the 4-day call cycle. Each of the 5 post-intervention teams was followed for 5 consecutive weekdays—25 rounds total, April–June 2014—to capture the 5-day cycle. (Because of technical difficulties, data from 1 rounding session were not captured.) For inclusion in the statistical analyses, TMA captured 166 on-service patients before the intervention and 304 afterward. Off-service patients, those with an attending other than the team attending, were excluded because their rounds were conducted separately.
We examined 2 primary outcomes, the proportion of time each clinical team member was present on rounds and the proportion of bedside rounding time. Secondary outcomes were round duration, rounding time per patient, and total non-patient time per rounding session (total rounding time minus total patient time).
Statistical Analysis
TMA data were organized in an Access database and analyzed with SAS Version 9.3 (SAS Institute, Cary, North Carolina). We analyzed the data by round session as well as by patient.
Data are presented as means with standard deviations, medians with interquartile ranges, and proportions, as appropriate. For analyses by round session, we used unadjusted linear regression; for patient-level analyses, we used general estimating equations to adjust for clustering of patients within each session; for nurse presence during any part of a round by patient, we used a χ2 test. Total non-patient time per round session was compared with use of patient-clustered general estimating equations using a γ distribution to account for the non-normality of the data.
RESULTS
Patient and Care Team Characteristics
Over the first year of the initiative, 85% of a team’s patients were on their assigned unit, and 87% of a unit’s patients were with the assigned team. Census numbers were 10.4 patients per general medicine team in April-June 2013 and 12.7 patients per team in April-June 2014, a 22% increase after care redesign. There were no statistically significant differences in patient characteristics, including age, sex, race, language, admission source, and comorbidity measure (Elixhauser score), between the pre-intervention and post-intervention study periods, except for a slightly higher proportion of patients admitted from home and fewer patients admitted directly from clinic (Table 1).
Primary Outcomes
Mean proportion of time the nurse was present on rounds per round session increased significantly (P < 0.001), from 24.1% to 67.8% (Figure 1A, Table 2). For individual patient encounters, the increased overall nursing presence was attributable to having more nurses on rounds and having nurses present for a larger proportion of individual rounding encounters (Figure 1B, Table 2). Nurses were present for at least some part of rounds for 53% of patients before the intervention and 93% afterward (P < 0.001). Mean proportion of round time by each of the 2 interns on each team decreased from 59.6% to 49.6% (P = 0.007).
Total bedside rounding time increased significantly ( P < 0.001), from 39.9% before the intervention to 55.8% afterward (Table 2). Meanwhile, percentage of rounding time spent on the unit but outside patient rooms decreased significantly ( P = 0.004), from 55.2% to 42.2%, as did rounding time on a unit completely different from the patient’s (4.9% before intervention, 2.0% afterward; P = 0.03). Again, patient-level results were similar (Figure 2, Table 2), but the decreased time spent on the unit, outside the patient rooms, was not significant.
Secondary Outcomes
Total rounding time decreased significantly, from a mean of 182 minutes (3.0 hours) at baseline to a mean of 146 minutes (2.4 hours) after the intervention, despite the higher post-intervention census. (When adjusted for patient census, the difference increased from 35.5 to 53.8 minutes; Table 2.) Mean rounding time per patient decreased significantly, from 14.7 minutes at baseline to 10.5 minutes after the intervention. For newly admitted patients, mean rounding time per patient decreased from 30.0 minutes before implementation to 16.3 minutes afterward. Mean rounding time also decreased, though much less, for subsequent-day patients (Table 2). For both new and existing patients, the decrease in rounding time largely was a reduction in time spent rounding outside patient rooms, with minimal impact on bedside time (Table 2). Mean time nurses were present during a patient’s rounds increased significantly, from 4.5 to 8.0 minutes (Table 2). Total nurse rounding time increased from 45.1 minutes per session to 98.8 minutes. Rounding time not related to patient discussion or evaluation decreased from 22.7 minutes per session to 13.3 minutes ( P = 0.003).
DISCUSSION
TMA of our care redesign initiative showed that this multipronged intervention, which included team regionalization, encouragement of bedside rounding with nurses, call structure changes, and attendings’ reading of admission notes before rounds, resulted in an increased proportion of rounding time spent with patients and an increased proportion of time nurses were present on rounds. Secondarily, round duration decreased even as patient census increased.
Regionalized teams have been found to improve interdisciplinary communication.1 The present study elaborates on that finding by demonstrating a dramatic increase in nursing presence on rounds, likely resulting from the unit’s use of rounding schedules and nurses’ prioritization of rounding orders, both of which were made possible by geographic co-localization. Other research has noted that one of the most significant barriers to interdisciplinary rounds is difficulty coordinating the start times of physician/nurse bedside rounding encounters. The system we have studied directly addresses this difficulty.9 Of note, nursing presence on rounds is necessary but not sufficient for true physician–nurse collaboration and effective communication,1 as reflected in a separate study of the intervention showing no significant difference in the concordance of the patient care plan between nurses and physicians before and after regionalization.12 Additional interventions may be needed to ensure that communication during bedside rounds is effective.
Our regionalized teams spent a significantly higher proportion of rounding time bedside, likely because of a cultural shift in expectations and the increased convenience of seeing patients on the team’s unit. Nevertheless, bedside time was not 100%. Structural barriers (eg, patients off-unit for dialysis) and cultural barriers likely contributed to the less than full adoption of bedside rounding. As described previously, cultural barriers to bedside rounding include trainees’ anxiety about being questioned in front of patients, the desire to freely exchange academic ideas in a conference room, and attendings’ doubts about their bedside teaching ability.1,9,13 Bedside rounds provide an important opportunity to apply the principles of patient- and family-centered care, including promotion of dignity and respect, information sharing, and collaboration. Thus, overcoming the concerns of housestaff and attendings and helping them feel prepared for bedside rounds can benefit the patient experience. More attention should be given to these practices as these types of interventions are implemented at Brigham and Women’s Hospital and elsewhere.1,13-15
Another primary concern about interdisciplinary bedside rounding is the perception that it takes more time.9 Therefore, it was important for us to measure round duration as a balancing measure to be considered for our intervention. Fortunately, we found round duration decreased with regionalization and encouragement of bedside rounding. This decrease was driven largely by a significant decrease in mean rounding time per new patient, which may be attributable at least in part to setting expectations that attendings and residents will read admission notes before rounds and that interns will summarize rather than recount information from admission notes. However, we also found rounding time decreases for subsequent-day patients, suggesting an underlying time savings. Spending a larger proportion of time bedside may therefore result in more efficient rounds. Bedside presentations can reduce redundancies, such as discussing a patient’s case outside his or her room and subsequently walking in and going over much of the same information with the patient. Our model de-emphasizes data transfer in favor of discussion of care plans. There was also a decrease in non-patient time, likely reflecting reduced transit time for regionalized teams. This decrease aligns with a recent finding that bedside rounding was at least as efficient as rounding outside the room.16
Of note, though a larger percentage of time was spent bedside after implementation of the care redesign, the absolute amount of bedside time did not change significantly. Our data showed that, even with shorter rounds, the same amount of absolute time can be spent bedside, face to face with the patient, by increasing the proportion of bedside rounding time. In other words, teams on average did not spend more time with patients, though the content and the structure of those encounters may have changed. This finding may be attributable to eliminating redundancy, forgoing the outside-the-room discussion, and thus the largest time reductions were realized there. In addition, teams incompletely adopted beside rounds, as reflected in the data. We expect that, with more complete adoption, an even larger proportion of time will be spent bedside, and absolute time bedside might increase as a result.
An unexpected result of the care redesign was that interns’ proportion of rounding time decreased after the intervention. This decrease most likely is attributable to interns’ being less likely to participate in rounds for a co-intern’s patient, and to their staying outside that patient’s room to give themselves more time to advance the care of their own patients. Before the intervention, when more rounding time was spent outside patient rooms, interns were more likely to join rounds for their co-intern’s patients because they could easily break away, as needed, to continue care of their own patients. The resident is now encouraged to use the morning huddle to identify which patients likely have the most educational value, and both interns are expected to join the bedside rounds for these patients.
This study had a few limitations. First, the pre–post design made it difficult to exclude the possibility that other temporal changes may have affected outcomes, though we did account for time-of-year effects by aligning our data-collection phases. In addition, the authors, including the director of the general medical service, are unaware of any co-interventions during the study period. Second, the multipronged intervention included care team regionalization, encouragement of bedside rounding with nurses, call structure changes (from 4 days to daily admitting), and attendings’ reading of admission notes before rounds. Thus, parsing which component(s) contributed to the results was difficult, though all the changes instituted likely were necessary for system redesign. For example, regionalization of clinicians to unit-based teams was made possible by switching to a daily admitting system.
Time that team members spent preparing for rounds was not recorded before or after the intervention. Thus, the decrease in total rounding time could have been accompanied by an increase in time spent preparing for rounds. However, admission notes were available in our electronic medical record before and after the intervention, and most residents and attendings were already reading them pre-intervention. After the intervention, pre-round note reading was more clearly defined as an expectation, and we were able to set the expectation that interns should use their presentations to summarize rather than recount information. In addition, in the post-intervention period, we did not include time spent preparing rounding orders; as already noted, however, preparation took only 5 minutes per day. Also, we did not analyze the content or the quality of the discussion on rounds, but simply recorded who was present where and when. Regarding the effect of the intervention on patient care, results were mixed. As reported in 2016, we saw no difference in frequency of adverse events with this intervention.12 However, a more sensitive measure of adverse events—used in a study on handoffs—showed our regionalization efforts had an additive effect on reducing overnight adverse events.17Researchers should now focus on the effects of care redesign on clinical outcomes, interdisciplinary care team communication, patient engagement and satisfaction, provider opinions of communication, workflow, patient care, and housestaff education. Our methodology can be used as a model to link structure, process, and outcome related to rounds and thereby better understand how best to optimize patient care and efficiency. Additional studies are needed to analyze the content of rounds and their association with patient and educational outcomes. Last, it will be important to conduct a study to see if the effects we have identified can be sustained. Such a study is already under way.
In conclusion, creating regionalized care teams and encouraging focused bedside rounds increased the proportion of bedside time and the presence of nurses on rounds. Rounds were shorter despite higher patient census. TMA revealed that regionalized care teams and bedside rounding at a large academic hospital are feasible, and are useful in establishing the necessary structures for increasing physician–nurse and provider–patient interactions.
Acknowledgments
The authors acknowledge Dr. Stan Ashley, Dr. Jacqueline Somerville, and Sheila Harris for their support of the regionalization initiative.
Disclosures
Dr. Schnipper received funding from Sanofi-aventis to conduct an investigator-initiated study to implement and evaluate a multi-faceted intervention to improve transitions of care in patients discharged home on insulin. The study was also supported by funding from the Marshall A. Wolf Medical Education Fund, Brigham and Women’s Hospital, and Dr. Stan Ashley, Chief Medical Officer, Brigham and Women’s Hospital. Some of the content of this article was orally presented at the annual meeting of the Society of Hospital Medicine; March 29-April 1, 2015; National Harbor, MD.
Attending rounds at academic medical centers are often disconnected from patients and non-physician care team members. Time spent bedside is consistently less than one third of total rounding time, with observational studies reporting a range of 9% to 33% over the past several decades.1-8 Rounds are often conducted outside patient rooms, denying patients, families, and nurses the opportunity to participate and offer valuable insights. Lack of bedside rounds thus limits patient and family engagement, patient input into the care plan, teaching of the physical examination, and communication and collaboration with nurses. In one study, physicians and nurses on rounds engaged in interprofessional communication in only 12% of patient cases.1 Studies have found interdisciplinary bedside rounds have several benefits, including subjectively improved communication and teamwork between physicians and nurses; increased patient satisfaction, including feeling more cared for by the medical team; and decreased length of stay and costs of care.2-10
However, there are many barriers to conducting interdisciplinary bedside rounds at large academic medical centers. Patients cared for by a single medical team are often geographically dispersed to several nursing units, and nurses are unable to predict when physicians will round on their patients. This situation limits nursing involvement on rounds and keeps doctors and nurses isolated from each other.2 Regionalization of care teams reduces this fragmentation by facilitating more interaction among doctors, patients, families, and nursing staff.
There are few data on how regionalized patients and interdisciplinary bedside rounds affect rounding time and the nature of rounds. This information is needed to understand how these structural changes mediate their effects, whether other steps are required to optimize outcomes, and how to maximize efficiency. We used time-motion analysis (TMA) to investigate how regionalization of medical teams, encouragement of bedside rounding, and systematic inclusion of nurses on ward rounds affect amount of time spent with patients, nursing presence on rounds, and total rounding time.
METHODS
Setting
This prospective interventional study, approved by the Institutional Review Board of Partners HealthCare, was conducted on the general medical wards at Brigham and Women’s Hospital, an academic 793-bed tertiary-care center in Boston, Massachusetts. Housestaff teams consist of 1 attending, 1 resident, and 2 interns with or without a medical student. Before June 20, 2013, daily rounds on medical inpatients were conducted largely on the patient unit but outside patient rooms. After completing most of a rounding discussion outside a patient’s room, the team might walk in to examine or speak with the patient. A typical medical team had patients dispersed over 7 medical units on average, and over as many as 13. As nurses were unit based, they did not consistently participate in rounds.
Intervention
In June 2013, as part of a general medical service care redesign initiative, the general medical teams were regionalized to specific inpatient units. The goal was to have teams admit patients predominantly to the team’s designated unit and to have all patients on a unit be cared for by the unit’s assigned team as often as possible, with an 85% goal for both. Toward those ends, the admitting structure was changed from a traditional 4-day call cycle to daily admitting for all teams, based on each unit’s bed availability.11
Teams were also expected to conduct rounds with nurses, and a system for facilitating these rounds was established. As physician and nurse care teams were now geographically co-located, it became possible for residents and nurses to check a rounding sheet for the planned patient rounding order, which had been set by the resident and nurse-in-charge before rounds. No more than about 5 minutes was needed to prepare each day’s order. The rounding sheet prioritized sick patients, newly admitted patients, and planned morning discharges, but patients were also always grouped by nurse. For example, the physician team rounded with the first nurse on all 3 of a nurse’s patients, and then proceeded to the next group of 3 patients with the next nurse, until all patients were seen.
Teams were encouraged to conduct patient- and family-centered rounds exclusively at bedside, except when bedside rounding was thought to be detrimental to a patient (eg, one with delirium). After an intern’s bedside presentation, which included a brief summary and details about overnight events and vital signs, the concerns of the patient, family, and nurse were shared, a focused physical examination performed, relevant data (eg, laboratory test results and imaging studies) reviewed, and the day’s plan formulated. The entire team, including the attending, was expected to have read new patients’ admission notes before rounds. Bedside rounds could thus be focused more on patient assessment and patient/family engagement and less on data transfer.
Several actions were taken to facilitate these changes. Residents, attendings, nurses, and other interdisciplinary team members participated in a series of focus groups and conferences to define workflows and share best practices for patient- and family-centered bedside rounds. Tips on bedside rounding were included in a general medicine rotation guidebook made available to residents and attendings. At the beginning of each post-intervention general medicine rotation, attendings and residents attended brief orientation sessions to review the new daily schedule, have interdisciplinary huddles, and share expectations for patient- and family-centered bedside rounds. On the general medicine units, new medical directors were hired to partner with existing nursing directors to support adoption of the workflows. Last, an interdisciplinary leadership team was formed to support the care redesign efforts. This team started meeting every 2 weeks.
Study Design
We used a pre–post analysis to study the effects of care redesign. Analysis was performed at the same time of year for 2 consecutive years to control for the stage of training and experience of the housestaff. TMA was performed by trained medical students using computer tablets linked to a customized Microsoft Access database form (Redmond, Washington). The form and the database were designed with specific buttons that, when pressed, recorded the time of particular events, such as the coming and going of each participant, the location of rounds, and the beginning and the end of rounding encounters with a patient. One research assistant using an Access entry form was able to dynamically track all events in real time, as they occurred. We collected data on 4 teams at baseline and 5 teams after the intervention. Each of the 4 baseline teams was followed for 4 consecutive weekdays—16 rounds total, April-June 2013—to capture the 4-day call cycle. Each of the 5 post-intervention teams was followed for 5 consecutive weekdays—25 rounds total, April–June 2014—to capture the 5-day cycle. (Because of technical difficulties, data from 1 rounding session were not captured.) For inclusion in the statistical analyses, TMA captured 166 on-service patients before the intervention and 304 afterward. Off-service patients, those with an attending other than the team attending, were excluded because their rounds were conducted separately.
We examined 2 primary outcomes, the proportion of time each clinical team member was present on rounds and the proportion of bedside rounding time. Secondary outcomes were round duration, rounding time per patient, and total non-patient time per rounding session (total rounding time minus total patient time).
Statistical Analysis
TMA data were organized in an Access database and analyzed with SAS Version 9.3 (SAS Institute, Cary, North Carolina). We analyzed the data by round session as well as by patient.
Data are presented as means with standard deviations, medians with interquartile ranges, and proportions, as appropriate. For analyses by round session, we used unadjusted linear regression; for patient-level analyses, we used general estimating equations to adjust for clustering of patients within each session; for nurse presence during any part of a round by patient, we used a χ2 test. Total non-patient time per round session was compared with use of patient-clustered general estimating equations using a γ distribution to account for the non-normality of the data.
RESULTS
Patient and Care Team Characteristics
Over the first year of the initiative, 85% of a team’s patients were on their assigned unit, and 87% of a unit’s patients were with the assigned team. Census numbers were 10.4 patients per general medicine team in April-June 2013 and 12.7 patients per team in April-June 2014, a 22% increase after care redesign. There were no statistically significant differences in patient characteristics, including age, sex, race, language, admission source, and comorbidity measure (Elixhauser score), between the pre-intervention and post-intervention study periods, except for a slightly higher proportion of patients admitted from home and fewer patients admitted directly from clinic (Table 1).
Primary Outcomes
Mean proportion of time the nurse was present on rounds per round session increased significantly (P < 0.001), from 24.1% to 67.8% (Figure 1A, Table 2). For individual patient encounters, the increased overall nursing presence was attributable to having more nurses on rounds and having nurses present for a larger proportion of individual rounding encounters (Figure 1B, Table 2). Nurses were present for at least some part of rounds for 53% of patients before the intervention and 93% afterward (P < 0.001). Mean proportion of round time by each of the 2 interns on each team decreased from 59.6% to 49.6% (P = 0.007).
Total bedside rounding time increased significantly ( P < 0.001), from 39.9% before the intervention to 55.8% afterward (Table 2). Meanwhile, percentage of rounding time spent on the unit but outside patient rooms decreased significantly ( P = 0.004), from 55.2% to 42.2%, as did rounding time on a unit completely different from the patient’s (4.9% before intervention, 2.0% afterward; P = 0.03). Again, patient-level results were similar (Figure 2, Table 2), but the decreased time spent on the unit, outside the patient rooms, was not significant.
Secondary Outcomes
Total rounding time decreased significantly, from a mean of 182 minutes (3.0 hours) at baseline to a mean of 146 minutes (2.4 hours) after the intervention, despite the higher post-intervention census. (When adjusted for patient census, the difference increased from 35.5 to 53.8 minutes; Table 2.) Mean rounding time per patient decreased significantly, from 14.7 minutes at baseline to 10.5 minutes after the intervention. For newly admitted patients, mean rounding time per patient decreased from 30.0 minutes before implementation to 16.3 minutes afterward. Mean rounding time also decreased, though much less, for subsequent-day patients (Table 2). For both new and existing patients, the decrease in rounding time largely was a reduction in time spent rounding outside patient rooms, with minimal impact on bedside time (Table 2). Mean time nurses were present during a patient’s rounds increased significantly, from 4.5 to 8.0 minutes (Table 2). Total nurse rounding time increased from 45.1 minutes per session to 98.8 minutes. Rounding time not related to patient discussion or evaluation decreased from 22.7 minutes per session to 13.3 minutes ( P = 0.003).
DISCUSSION
TMA of our care redesign initiative showed that this multipronged intervention, which included team regionalization, encouragement of bedside rounding with nurses, call structure changes, and attendings’ reading of admission notes before rounds, resulted in an increased proportion of rounding time spent with patients and an increased proportion of time nurses were present on rounds. Secondarily, round duration decreased even as patient census increased.
Regionalized teams have been found to improve interdisciplinary communication.1 The present study elaborates on that finding by demonstrating a dramatic increase in nursing presence on rounds, likely resulting from the unit’s use of rounding schedules and nurses’ prioritization of rounding orders, both of which were made possible by geographic co-localization. Other research has noted that one of the most significant barriers to interdisciplinary rounds is difficulty coordinating the start times of physician/nurse bedside rounding encounters. The system we have studied directly addresses this difficulty.9 Of note, nursing presence on rounds is necessary but not sufficient for true physician–nurse collaboration and effective communication,1 as reflected in a separate study of the intervention showing no significant difference in the concordance of the patient care plan between nurses and physicians before and after regionalization.12 Additional interventions may be needed to ensure that communication during bedside rounds is effective.
Our regionalized teams spent a significantly higher proportion of rounding time bedside, likely because of a cultural shift in expectations and the increased convenience of seeing patients on the team’s unit. Nevertheless, bedside time was not 100%. Structural barriers (eg, patients off-unit for dialysis) and cultural barriers likely contributed to the less than full adoption of bedside rounding. As described previously, cultural barriers to bedside rounding include trainees’ anxiety about being questioned in front of patients, the desire to freely exchange academic ideas in a conference room, and attendings’ doubts about their bedside teaching ability.1,9,13 Bedside rounds provide an important opportunity to apply the principles of patient- and family-centered care, including promotion of dignity and respect, information sharing, and collaboration. Thus, overcoming the concerns of housestaff and attendings and helping them feel prepared for bedside rounds can benefit the patient experience. More attention should be given to these practices as these types of interventions are implemented at Brigham and Women’s Hospital and elsewhere.1,13-15
Another primary concern about interdisciplinary bedside rounding is the perception that it takes more time.9 Therefore, it was important for us to measure round duration as a balancing measure to be considered for our intervention. Fortunately, we found round duration decreased with regionalization and encouragement of bedside rounding. This decrease was driven largely by a significant decrease in mean rounding time per new patient, which may be attributable at least in part to setting expectations that attendings and residents will read admission notes before rounds and that interns will summarize rather than recount information from admission notes. However, we also found rounding time decreases for subsequent-day patients, suggesting an underlying time savings. Spending a larger proportion of time bedside may therefore result in more efficient rounds. Bedside presentations can reduce redundancies, such as discussing a patient’s case outside his or her room and subsequently walking in and going over much of the same information with the patient. Our model de-emphasizes data transfer in favor of discussion of care plans. There was also a decrease in non-patient time, likely reflecting reduced transit time for regionalized teams. This decrease aligns with a recent finding that bedside rounding was at least as efficient as rounding outside the room.16
Of note, though a larger percentage of time was spent bedside after implementation of the care redesign, the absolute amount of bedside time did not change significantly. Our data showed that, even with shorter rounds, the same amount of absolute time can be spent bedside, face to face with the patient, by increasing the proportion of bedside rounding time. In other words, teams on average did not spend more time with patients, though the content and the structure of those encounters may have changed. This finding may be attributable to eliminating redundancy, forgoing the outside-the-room discussion, and thus the largest time reductions were realized there. In addition, teams incompletely adopted beside rounds, as reflected in the data. We expect that, with more complete adoption, an even larger proportion of time will be spent bedside, and absolute time bedside might increase as a result.
An unexpected result of the care redesign was that interns’ proportion of rounding time decreased after the intervention. This decrease most likely is attributable to interns’ being less likely to participate in rounds for a co-intern’s patient, and to their staying outside that patient’s room to give themselves more time to advance the care of their own patients. Before the intervention, when more rounding time was spent outside patient rooms, interns were more likely to join rounds for their co-intern’s patients because they could easily break away, as needed, to continue care of their own patients. The resident is now encouraged to use the morning huddle to identify which patients likely have the most educational value, and both interns are expected to join the bedside rounds for these patients.
This study had a few limitations. First, the pre–post design made it difficult to exclude the possibility that other temporal changes may have affected outcomes, though we did account for time-of-year effects by aligning our data-collection phases. In addition, the authors, including the director of the general medical service, are unaware of any co-interventions during the study period. Second, the multipronged intervention included care team regionalization, encouragement of bedside rounding with nurses, call structure changes (from 4 days to daily admitting), and attendings’ reading of admission notes before rounds. Thus, parsing which component(s) contributed to the results was difficult, though all the changes instituted likely were necessary for system redesign. For example, regionalization of clinicians to unit-based teams was made possible by switching to a daily admitting system.
Time that team members spent preparing for rounds was not recorded before or after the intervention. Thus, the decrease in total rounding time could have been accompanied by an increase in time spent preparing for rounds. However, admission notes were available in our electronic medical record before and after the intervention, and most residents and attendings were already reading them pre-intervention. After the intervention, pre-round note reading was more clearly defined as an expectation, and we were able to set the expectation that interns should use their presentations to summarize rather than recount information. In addition, in the post-intervention period, we did not include time spent preparing rounding orders; as already noted, however, preparation took only 5 minutes per day. Also, we did not analyze the content or the quality of the discussion on rounds, but simply recorded who was present where and when. Regarding the effect of the intervention on patient care, results were mixed. As reported in 2016, we saw no difference in frequency of adverse events with this intervention.12 However, a more sensitive measure of adverse events—used in a study on handoffs—showed our regionalization efforts had an additive effect on reducing overnight adverse events.17Researchers should now focus on the effects of care redesign on clinical outcomes, interdisciplinary care team communication, patient engagement and satisfaction, provider opinions of communication, workflow, patient care, and housestaff education. Our methodology can be used as a model to link structure, process, and outcome related to rounds and thereby better understand how best to optimize patient care and efficiency. Additional studies are needed to analyze the content of rounds and their association with patient and educational outcomes. Last, it will be important to conduct a study to see if the effects we have identified can be sustained. Such a study is already under way.
In conclusion, creating regionalized care teams and encouraging focused bedside rounds increased the proportion of bedside time and the presence of nurses on rounds. Rounds were shorter despite higher patient census. TMA revealed that regionalized care teams and bedside rounding at a large academic hospital are feasible, and are useful in establishing the necessary structures for increasing physician–nurse and provider–patient interactions.
Acknowledgments
The authors acknowledge Dr. Stan Ashley, Dr. Jacqueline Somerville, and Sheila Harris for their support of the regionalization initiative.
Disclosures
Dr. Schnipper received funding from Sanofi-aventis to conduct an investigator-initiated study to implement and evaluate a multi-faceted intervention to improve transitions of care in patients discharged home on insulin. The study was also supported by funding from the Marshall A. Wolf Medical Education Fund, Brigham and Women’s Hospital, and Dr. Stan Ashley, Chief Medical Officer, Brigham and Women’s Hospital. Some of the content of this article was orally presented at the annual meeting of the Society of Hospital Medicine; March 29-April 1, 2015; National Harbor, MD.
1. Crumlish CM, Yialamas MA, McMahon GT. Quantification of bedside teaching by an academic hospitalist group. J Hosp Med. 2009;4(5):304-307. PubMed
2. Gonzalo JD, Masters PA, Simons RJ, Chuang CH. Attending rounds and bedside case presentations: medical student and medicine resident experiences and attitudes. Teach Learn Med. 2009;21(2):105-110. PubMed
3. Elliot DL, Hickam DH. Attending rounds on in-patient units: differences between medical and non-medical services. Med Educ. 1993;27(6):503-508. PubMed
4. Payson HE, Barchas JD. A time study of medical teaching rounds. N Engl J Med. 1965;273(27):1468-1471. PubMed
5. Tremonti LP, Biddle WB. Teaching behaviors of residents and faculty members. J Med Educ. 1982;57(11):854-859. PubMed
6. Miller M, Johnson B, Greene HL, Baier M, Nowlin S. An observational study of attending rounds. J Gen Intern Med. 1992;7(6):646-648. PubMed
7. Collins GF, Cassie JM, Daggett CJ. The role of the attending physician in clinical training. J Med Educ. 1978;53(5):429-431. PubMed
8. Ward DR, Ghali WA, Graham A, Lemaire JB. A real-time locating system observes physician time-motion patterns during walk-rounds: a pilot study. BMC Med Educ. 2014;14:37. PubMed
9. Gonzalo JD, Kuperman E, Lehman E, Haidet P. Bedside interprofessional rounds: perceptions of benefits and barriers by internal medicine nursing staff, attending physicians, and housestaff physicians. J Hosp Med. 2014;9(10):646-651. PubMed
10. Stickrath C, Noble M, Prochazka A, et al. Attending rounds in the current era: what is and is not happening. JAMA Intern Med. 2013;173(12):1084-1089. PubMed
11. Boxer R, Vitale M, Gershanik EF, et al. 5th time’s a charm: creation of unit-based care teams in a high occupancy hospital [abstract]. J Hosp Med. 2015;10(suppl 2).
12. Mueller SK, Schnipper JL, Giannelli K, Roy CL, Boxer R. Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services. J Hosp Med. 2016;11(9):620-627. PubMed
13. Chauke HL, Pattinson RC. Ward rounds—bedside or conference room? S Afr Med J. 2006;96(5):398-400. PubMed
14. Wang-Cheng RM, Barnas GP, Sigmann P, Riendl PA, Young MJ. Bedside case presentations: why patients like them but learners don’t. J Gen Intern Med. 1989;4(4):284-287. PubMed
15. Lehmann LS, Brancati FL, Chen MC, Roter D, Dobs AS. The effect of bedside case presentations on patients’ perceptions of their medical care. N Engl J Med. 1997;336(16):1150-1155. PubMed
16. Gonzalo JD, Chuang CH, Huang G, Smith C. The return of bedside rounds: an educational intervention. J Gen Intern Med. 2010;25(8):792-798. PubMed
17. Mueller SK, Yoon C, Schnipper JL. Association of a web-based handoff tool with rates of medical errors. JAMA Intern Med. 2016;176(9):1400-1402. PubMed
1. Crumlish CM, Yialamas MA, McMahon GT. Quantification of bedside teaching by an academic hospitalist group. J Hosp Med. 2009;4(5):304-307. PubMed
2. Gonzalo JD, Masters PA, Simons RJ, Chuang CH. Attending rounds and bedside case presentations: medical student and medicine resident experiences and attitudes. Teach Learn Med. 2009;21(2):105-110. PubMed
3. Elliot DL, Hickam DH. Attending rounds on in-patient units: differences between medical and non-medical services. Med Educ. 1993;27(6):503-508. PubMed
4. Payson HE, Barchas JD. A time study of medical teaching rounds. N Engl J Med. 1965;273(27):1468-1471. PubMed
5. Tremonti LP, Biddle WB. Teaching behaviors of residents and faculty members. J Med Educ. 1982;57(11):854-859. PubMed
6. Miller M, Johnson B, Greene HL, Baier M, Nowlin S. An observational study of attending rounds. J Gen Intern Med. 1992;7(6):646-648. PubMed
7. Collins GF, Cassie JM, Daggett CJ. The role of the attending physician in clinical training. J Med Educ. 1978;53(5):429-431. PubMed
8. Ward DR, Ghali WA, Graham A, Lemaire JB. A real-time locating system observes physician time-motion patterns during walk-rounds: a pilot study. BMC Med Educ. 2014;14:37. PubMed
9. Gonzalo JD, Kuperman E, Lehman E, Haidet P. Bedside interprofessional rounds: perceptions of benefits and barriers by internal medicine nursing staff, attending physicians, and housestaff physicians. J Hosp Med. 2014;9(10):646-651. PubMed
10. Stickrath C, Noble M, Prochazka A, et al. Attending rounds in the current era: what is and is not happening. JAMA Intern Med. 2013;173(12):1084-1089. PubMed
11. Boxer R, Vitale M, Gershanik EF, et al. 5th time’s a charm: creation of unit-based care teams in a high occupancy hospital [abstract]. J Hosp Med. 2015;10(suppl 2).
12. Mueller SK, Schnipper JL, Giannelli K, Roy CL, Boxer R. Impact of regionalized care on concordance of plan and preventable adverse events on general medicine services. J Hosp Med. 2016;11(9):620-627. PubMed
13. Chauke HL, Pattinson RC. Ward rounds—bedside or conference room? S Afr Med J. 2006;96(5):398-400. PubMed
14. Wang-Cheng RM, Barnas GP, Sigmann P, Riendl PA, Young MJ. Bedside case presentations: why patients like them but learners don’t. J Gen Intern Med. 1989;4(4):284-287. PubMed
15. Lehmann LS, Brancati FL, Chen MC, Roter D, Dobs AS. The effect of bedside case presentations on patients’ perceptions of their medical care. N Engl J Med. 1997;336(16):1150-1155. PubMed
16. Gonzalo JD, Chuang CH, Huang G, Smith C. The return of bedside rounds: an educational intervention. J Gen Intern Med. 2010;25(8):792-798. PubMed
17. Mueller SK, Yoon C, Schnipper JL. Association of a web-based handoff tool with rates of medical errors. JAMA Intern Med. 2016;176(9):1400-1402. PubMed
© 2017 Society of Hospital Medicine
Knowledge of Selected Medical Procedures
Medical procedures, an essential and highly valued part of medical education, are often undertaught and inconsistently evaluated. Hospitalists play an increasingly important role in developing the skills of resident‐learners. Alumni rate procedure skills as some of the most important skills learned during residency training,1, 2 but frequently identify training in procedural skills as having been insufficient.3, 4 For certification in internal medicine, the American Board of Internal Medicine (ABIM) has identified a limited set of procedures in which it expects all candidates to be cognitively competent with regard to their knowledge of these procedures. Although active participation in procedures is recommended for certification in internal medicine, the demonstration of procedural proficiency is not required.5
Resident competence in performing procedures remains highly variable and procedural complications can be a source of morbidity and mortality.2, 6, 7 A validated tool for the assessment of procedure related knowledge is currently lacking. In existing standardized tests, including the in‐training examination (ITE) and ABIM certification examination, only a fraction of questions pertain to medical procedures. The necessity for a specifically designed, standardized instrument that can objectively measure procedure related knowledge has been highlighted by studies that have demonstrated that there is little correlation between the rate of procedure‐related complications and ABIM/ITE scores.8 A validated tool to assess the knowledge of residents in selected medical procedures could serve to assess the readiness of residents to begin supervised practice and form part of a proficiency assessment.
In this study we aimed to develop a valid and reliable test of procedural knowledge in 3 procedures associated with potentially serious complications.
Methods
Placement of an arterial line, central venous catheter and thoracentesis were selected as the focus for test development. Using the National Board of Medical Examiners question development guidelines, multiple‐choice questions were developed to test residents on specific points of a prepared curriculum. Questions were designed to test the essential cognitive aspects of medical procedures, including indications, contraindications, and the management of complications, with an emphasis on the elements that were considered by a panel of experts to be frequently misunderstood. Questions were written by faculty trained in question writing (G.M.) and assessed for clarity by other members of faculty. Content evidence of the 36‐item examination (12 questions per procedure) was established by a panel of 4 critical care specialists with expertise in medical education. The study was approved by the Institutional Review Board at all sites.
Item performance characteristics were evaluated by administering the test online to a series of 30 trainees and specialty clinicians. Postadministration interviews with the critical care experts were performed to determine whether test questions were clear and appropriate for residents. Following initial testing, 4 test items with the lowest discrimination according to a point‐biserial correlation (Integrity; Castle Rock Research, Canada) were deleted from the test. The resulting 32‐item test contained items of varying difficulty to allow for effective discrimination between examinees (Appendix 1).
The test was then administered to residents beginning rotations in either the medical intensive care unit or in the coronary care unit at 4 medical centers in Massachusetts (Brigham and Women's Hospital; Massachusetts General Hospital; Faulkner Hospital; and North Shore Medical Center). In addition to completing the on‐line, self‐administered examination, participants provided baseline data including year of residency training, anticipated career path, and the number of prior procedures performed. On a 5‐point Likert scale participants estimated their self‐perceived confidence at performing the procedure (with and without supervision) and supervising each of the procedures. Residents were invited to complete a second test before the end of their rotation (2‐4 weeks after the initial test) in order to assess test‐retest reliability. Answers were made available only after the conclusion of the study.
Reliability of the 32‐item instrument was measured by Cronbach's analysis; a value of 0.6 is considered adequate and values of 0.7 or higher indicate good reliability. Pearson's correlation (Pearson's r) was used to compute test‐retest reliability. Univariate analyses were used to assess the association of the demographic variables with the test scores. Comparison of test scores between groups was made using a t test/Wilcoxon rank sum (2 groups) and analysis of variance (ANOVA)/Kruskal‐Wallis (3 or more groups). The associations of number of prior procedures attempted and self‐reported confidence with test scores was explored using Spearman's correlation. Inferences were made at the 0.05 level of significance, using 2‐tailed tests. Statistical analyses were performed using SPSS 15.0 (SPSS, Inc., Chicago, IL).
Results
Of the 192 internal medicine residents who consented to participate in the study between February and June 2006, 188 completed the initial and repeat test. Subject characteristics are detailed in Table 1.
| Number (%) | |
|---|---|
| Total residents | 192 |
| Males | 113 (59) |
| Year of residency training | |
| First | 101 (52) |
| Second | 64 (33) |
| Third/fourth | 27 (14) |
| Anticipated career path | |
| General medicine/primary care | 26 (14) |
| Critical care | 47 (24) |
| Medical subspecialties | 54 (28) |
| Undecided/other | 65 (34) |
Reliability of the 32‐item instrument measured by Cronbach's was 0.79 and its test‐retest reliability was 0.82. The items difficulty mean was 0.52 with a corrected point biserial correlation mean of 0.26. The test was of high difficulty, with a mean overall score of 50% (median 53%, interquartile range 44‐59%). Baseline scores differed significantly by residency program (P = 0.03). Residents with anticipated careers in critical care had significantly higher scores than those with anticipated careers in primary care (median scores critical care 56%, primary care and other nonprocedural medical subspecialties 50%, P = 0.01).
Residents in their final year reported performing a median of 13 arterial lines, 14 central venous lines, and 3 thoracenteses over the course of their residency training (Table 2). Increase in the number of performed procedures (central lines, arterial lines, and thoracenteses) was associated with an increase in test score (Spearman's correlation coefficient 0.35, P < 0.001). Residents in the highest and lowest decile of procedures performed had median scores of 56% and 43%, respectively (P < 0.001). Increasing seniority in residency was associated with an increase in overall test scores (median score by program year 49%, 54%, 50%, and 64%, P = 0.02).
| Year of Residency Training | Median Number of Procedures (Interquartile Range) | ||
|---|---|---|---|
| Arterial Line Insertion | Central Venous Line Insertion | Thoracentesis | |
| First | 1 (03) | 1 (04) | 0 (01) |
| Second | 8.5 (618) | 10 (518) | 2 (04) |
| Third/fourth | 13 (820) | 14 (1027) | 3 (26) |
Increase in self‐reported confidence was significantly associated with an increase in the number of performed procedures (Spearman's correlation coefficients for central line 0.83, arterial lines 0.76, and thoracentesis 0.78, all P < 0.001) and increasing seniority (0.66, 0.59, and 0.52, respectively, all P < 0.001).
Discussion
The determination of procedural competence has long been a challenge for trainers and internal medicine programs; methods for measuring procedural skills have not been rigorously studied. Procedural competence requires a combination of theoretical knowledge and practical skill. However, given the declining number of procedures performed by internists,4 the new ABIM guidelines mandate cognitive competence in contrast to the demonstration of hands‐on procedural proficiency.
We therefore sought to develop and validate the results of an examination of the theoretical knowledge necessary to perform 3 procedures associated with potentially serious complications. Following establishment of content evidence, item performance characteristics and postadministration interviews were used to develop a 32‐item test. We confirmed the test's internal structure by assessment of reliability and assessed the association of test scores with other variables for which correlation would be expected.
We found that residents performed poorly on test content considered to be important by procedure specialists. These findings highlight the limitations in current procedure training that is frequently sporadic and often variable. The numbers of procedures reported over the duration of residency by residents at these centers were low. It is unclear if the low number of procedures performed was due to limitations in resident content knowledge or if it reflects the increasing use of interventional services with fewer opportunities for experiential learning. Nevertheless, an increasing number of prior procedures was associated with higher self‐reported confidence for all procedures and translated to higher test scores.
This study was limited to 4 teaching hospitals and further studies may be needed to investigate the wider generalizability of the study instrument. However, participants were from 3 distinct internal medicine residency programs that included both community and university hospitals. We relied on resident self‐reports and did not independently verify the number of prior procedures performed. However, similar assumptions have been made in prior studies that physicians who rarely perform procedures are able to provide reasonable estimates of the total number performed.3
The reliability of the 32‐item test (Cronbach's = 0.79) is in the expected range for this length of test and indicates good reliability.9, 10 Given the potential complications associated with advanced medical procedures, there is increasing need to establish criteria for competence. Although we have not established a score threshold, the development of this validated tool to assess procedural knowledge is an important step toward establishing such a goal.
This test may facilitate efforts by hospitalists and others to evaluate the efficacy and refine existing methods of procedure training. Feedback to educators using this assessment tool may assist in the improvement of teaching strategies. In addition, the assessment of cognitive competence in procedure‐related knowledge using a rigorous and reliable means of assessment such as outlined in this study may help identify residents who need further training. Recognition for the necessity for additional training and oversight are likely to be especially important if residents are expected to perform procedures safely yet have fewer opportunities for practice.
Acknowledgements
The authors thank Dr. Stephen Wright, Haley Hamlin, and Matt Johnston for their contributions to the data collection and analysis.
- ,,, et al.Altering residency curriculum in response to a changing practice environment: use of the Mayo internal medicine residency alumni survey.Mayo Clin Proc.1990;65(6):809–817.
- ,,,,,.Preparation for practice in internal medicine. A study of ten years of residency graduates.Arch Intern Med.1988;148(4):853–856.
- ,,,,,.Procedural experience and comfort level in internal medicine trainees.J Gen Intern Med.2000;15(10):716–722.
- .Training internists in procedural skills.Ann Intern Med.1992;116(12 Pt 2):1091–1093.
- ABIM. Policies and Procedures for Certification in Internal Medicine2008. Available at: http://www.abim.org/certification/policies/imss/im.aspx. Accessed August 2009.
- ,,, et al.Confidence of graduating internal medicine residents to perform ambulatory procedures.J Gen Intern Med.2000;15(6):361–365.
- ,,.The lasting value of clinical skills.JAMA.1985;254(1):70–76.
- ,,.Are commonly used resident measurements associated with procedural skills in internal medicine residency training?J Gen Intern Med.2007;22(3):357–361.
- .Psychometric Theory.New York:McGraw Hill;1978.
- .Coefficient alpha and the internal structure of tests.Psychometrika.1951;16:297–334.
Medical procedures, an essential and highly valued part of medical education, are often undertaught and inconsistently evaluated. Hospitalists play an increasingly important role in developing the skills of resident‐learners. Alumni rate procedure skills as some of the most important skills learned during residency training,1, 2 but frequently identify training in procedural skills as having been insufficient.3, 4 For certification in internal medicine, the American Board of Internal Medicine (ABIM) has identified a limited set of procedures in which it expects all candidates to be cognitively competent with regard to their knowledge of these procedures. Although active participation in procedures is recommended for certification in internal medicine, the demonstration of procedural proficiency is not required.5
Resident competence in performing procedures remains highly variable and procedural complications can be a source of morbidity and mortality.2, 6, 7 A validated tool for the assessment of procedure related knowledge is currently lacking. In existing standardized tests, including the in‐training examination (ITE) and ABIM certification examination, only a fraction of questions pertain to medical procedures. The necessity for a specifically designed, standardized instrument that can objectively measure procedure related knowledge has been highlighted by studies that have demonstrated that there is little correlation between the rate of procedure‐related complications and ABIM/ITE scores.8 A validated tool to assess the knowledge of residents in selected medical procedures could serve to assess the readiness of residents to begin supervised practice and form part of a proficiency assessment.
In this study we aimed to develop a valid and reliable test of procedural knowledge in 3 procedures associated with potentially serious complications.
Methods
Placement of an arterial line, central venous catheter and thoracentesis were selected as the focus for test development. Using the National Board of Medical Examiners question development guidelines, multiple‐choice questions were developed to test residents on specific points of a prepared curriculum. Questions were designed to test the essential cognitive aspects of medical procedures, including indications, contraindications, and the management of complications, with an emphasis on the elements that were considered by a panel of experts to be frequently misunderstood. Questions were written by faculty trained in question writing (G.M.) and assessed for clarity by other members of faculty. Content evidence of the 36‐item examination (12 questions per procedure) was established by a panel of 4 critical care specialists with expertise in medical education. The study was approved by the Institutional Review Board at all sites.
Item performance characteristics were evaluated by administering the test online to a series of 30 trainees and specialty clinicians. Postadministration interviews with the critical care experts were performed to determine whether test questions were clear and appropriate for residents. Following initial testing, 4 test items with the lowest discrimination according to a point‐biserial correlation (Integrity; Castle Rock Research, Canada) were deleted from the test. The resulting 32‐item test contained items of varying difficulty to allow for effective discrimination between examinees (Appendix 1).
The test was then administered to residents beginning rotations in either the medical intensive care unit or in the coronary care unit at 4 medical centers in Massachusetts (Brigham and Women's Hospital; Massachusetts General Hospital; Faulkner Hospital; and North Shore Medical Center). In addition to completing the on‐line, self‐administered examination, participants provided baseline data including year of residency training, anticipated career path, and the number of prior procedures performed. On a 5‐point Likert scale participants estimated their self‐perceived confidence at performing the procedure (with and without supervision) and supervising each of the procedures. Residents were invited to complete a second test before the end of their rotation (2‐4 weeks after the initial test) in order to assess test‐retest reliability. Answers were made available only after the conclusion of the study.
Reliability of the 32‐item instrument was measured by Cronbach's analysis; a value of 0.6 is considered adequate and values of 0.7 or higher indicate good reliability. Pearson's correlation (Pearson's r) was used to compute test‐retest reliability. Univariate analyses were used to assess the association of the demographic variables with the test scores. Comparison of test scores between groups was made using a t test/Wilcoxon rank sum (2 groups) and analysis of variance (ANOVA)/Kruskal‐Wallis (3 or more groups). The associations of number of prior procedures attempted and self‐reported confidence with test scores was explored using Spearman's correlation. Inferences were made at the 0.05 level of significance, using 2‐tailed tests. Statistical analyses were performed using SPSS 15.0 (SPSS, Inc., Chicago, IL).
Results
Of the 192 internal medicine residents who consented to participate in the study between February and June 2006, 188 completed the initial and repeat test. Subject characteristics are detailed in Table 1.
| Number (%) | |
|---|---|
| Total residents | 192 |
| Males | 113 (59) |
| Year of residency training | |
| First | 101 (52) |
| Second | 64 (33) |
| Third/fourth | 27 (14) |
| Anticipated career path | |
| General medicine/primary care | 26 (14) |
| Critical care | 47 (24) |
| Medical subspecialties | 54 (28) |
| Undecided/other | 65 (34) |
Reliability of the 32‐item instrument measured by Cronbach's was 0.79 and its test‐retest reliability was 0.82. The items difficulty mean was 0.52 with a corrected point biserial correlation mean of 0.26. The test was of high difficulty, with a mean overall score of 50% (median 53%, interquartile range 44‐59%). Baseline scores differed significantly by residency program (P = 0.03). Residents with anticipated careers in critical care had significantly higher scores than those with anticipated careers in primary care (median scores critical care 56%, primary care and other nonprocedural medical subspecialties 50%, P = 0.01).
Residents in their final year reported performing a median of 13 arterial lines, 14 central venous lines, and 3 thoracenteses over the course of their residency training (Table 2). Increase in the number of performed procedures (central lines, arterial lines, and thoracenteses) was associated with an increase in test score (Spearman's correlation coefficient 0.35, P < 0.001). Residents in the highest and lowest decile of procedures performed had median scores of 56% and 43%, respectively (P < 0.001). Increasing seniority in residency was associated with an increase in overall test scores (median score by program year 49%, 54%, 50%, and 64%, P = 0.02).
| Year of Residency Training | Median Number of Procedures (Interquartile Range) | ||
|---|---|---|---|
| Arterial Line Insertion | Central Venous Line Insertion | Thoracentesis | |
| First | 1 (03) | 1 (04) | 0 (01) |
| Second | 8.5 (618) | 10 (518) | 2 (04) |
| Third/fourth | 13 (820) | 14 (1027) | 3 (26) |
Increase in self‐reported confidence was significantly associated with an increase in the number of performed procedures (Spearman's correlation coefficients for central line 0.83, arterial lines 0.76, and thoracentesis 0.78, all P < 0.001) and increasing seniority (0.66, 0.59, and 0.52, respectively, all P < 0.001).
Discussion
The determination of procedural competence has long been a challenge for trainers and internal medicine programs; methods for measuring procedural skills have not been rigorously studied. Procedural competence requires a combination of theoretical knowledge and practical skill. However, given the declining number of procedures performed by internists,4 the new ABIM guidelines mandate cognitive competence in contrast to the demonstration of hands‐on procedural proficiency.
We therefore sought to develop and validate the results of an examination of the theoretical knowledge necessary to perform 3 procedures associated with potentially serious complications. Following establishment of content evidence, item performance characteristics and postadministration interviews were used to develop a 32‐item test. We confirmed the test's internal structure by assessment of reliability and assessed the association of test scores with other variables for which correlation would be expected.
We found that residents performed poorly on test content considered to be important by procedure specialists. These findings highlight the limitations in current procedure training that is frequently sporadic and often variable. The numbers of procedures reported over the duration of residency by residents at these centers were low. It is unclear if the low number of procedures performed was due to limitations in resident content knowledge or if it reflects the increasing use of interventional services with fewer opportunities for experiential learning. Nevertheless, an increasing number of prior procedures was associated with higher self‐reported confidence for all procedures and translated to higher test scores.
This study was limited to 4 teaching hospitals and further studies may be needed to investigate the wider generalizability of the study instrument. However, participants were from 3 distinct internal medicine residency programs that included both community and university hospitals. We relied on resident self‐reports and did not independently verify the number of prior procedures performed. However, similar assumptions have been made in prior studies that physicians who rarely perform procedures are able to provide reasonable estimates of the total number performed.3
The reliability of the 32‐item test (Cronbach's = 0.79) is in the expected range for this length of test and indicates good reliability.9, 10 Given the potential complications associated with advanced medical procedures, there is increasing need to establish criteria for competence. Although we have not established a score threshold, the development of this validated tool to assess procedural knowledge is an important step toward establishing such a goal.
This test may facilitate efforts by hospitalists and others to evaluate the efficacy and refine existing methods of procedure training. Feedback to educators using this assessment tool may assist in the improvement of teaching strategies. In addition, the assessment of cognitive competence in procedure‐related knowledge using a rigorous and reliable means of assessment such as outlined in this study may help identify residents who need further training. Recognition for the necessity for additional training and oversight are likely to be especially important if residents are expected to perform procedures safely yet have fewer opportunities for practice.
Acknowledgements
The authors thank Dr. Stephen Wright, Haley Hamlin, and Matt Johnston for their contributions to the data collection and analysis.
Medical procedures, an essential and highly valued part of medical education, are often undertaught and inconsistently evaluated. Hospitalists play an increasingly important role in developing the skills of resident‐learners. Alumni rate procedure skills as some of the most important skills learned during residency training,1, 2 but frequently identify training in procedural skills as having been insufficient.3, 4 For certification in internal medicine, the American Board of Internal Medicine (ABIM) has identified a limited set of procedures in which it expects all candidates to be cognitively competent with regard to their knowledge of these procedures. Although active participation in procedures is recommended for certification in internal medicine, the demonstration of procedural proficiency is not required.5
Resident competence in performing procedures remains highly variable and procedural complications can be a source of morbidity and mortality.2, 6, 7 A validated tool for the assessment of procedure related knowledge is currently lacking. In existing standardized tests, including the in‐training examination (ITE) and ABIM certification examination, only a fraction of questions pertain to medical procedures. The necessity for a specifically designed, standardized instrument that can objectively measure procedure related knowledge has been highlighted by studies that have demonstrated that there is little correlation between the rate of procedure‐related complications and ABIM/ITE scores.8 A validated tool to assess the knowledge of residents in selected medical procedures could serve to assess the readiness of residents to begin supervised practice and form part of a proficiency assessment.
In this study we aimed to develop a valid and reliable test of procedural knowledge in 3 procedures associated with potentially serious complications.
Methods
Placement of an arterial line, central venous catheter and thoracentesis were selected as the focus for test development. Using the National Board of Medical Examiners question development guidelines, multiple‐choice questions were developed to test residents on specific points of a prepared curriculum. Questions were designed to test the essential cognitive aspects of medical procedures, including indications, contraindications, and the management of complications, with an emphasis on the elements that were considered by a panel of experts to be frequently misunderstood. Questions were written by faculty trained in question writing (G.M.) and assessed for clarity by other members of faculty. Content evidence of the 36‐item examination (12 questions per procedure) was established by a panel of 4 critical care specialists with expertise in medical education. The study was approved by the Institutional Review Board at all sites.
Item performance characteristics were evaluated by administering the test online to a series of 30 trainees and specialty clinicians. Postadministration interviews with the critical care experts were performed to determine whether test questions were clear and appropriate for residents. Following initial testing, 4 test items with the lowest discrimination according to a point‐biserial correlation (Integrity; Castle Rock Research, Canada) were deleted from the test. The resulting 32‐item test contained items of varying difficulty to allow for effective discrimination between examinees (Appendix 1).
The test was then administered to residents beginning rotations in either the medical intensive care unit or in the coronary care unit at 4 medical centers in Massachusetts (Brigham and Women's Hospital; Massachusetts General Hospital; Faulkner Hospital; and North Shore Medical Center). In addition to completing the on‐line, self‐administered examination, participants provided baseline data including year of residency training, anticipated career path, and the number of prior procedures performed. On a 5‐point Likert scale participants estimated their self‐perceived confidence at performing the procedure (with and without supervision) and supervising each of the procedures. Residents were invited to complete a second test before the end of their rotation (2‐4 weeks after the initial test) in order to assess test‐retest reliability. Answers were made available only after the conclusion of the study.
Reliability of the 32‐item instrument was measured by Cronbach's analysis; a value of 0.6 is considered adequate and values of 0.7 or higher indicate good reliability. Pearson's correlation (Pearson's r) was used to compute test‐retest reliability. Univariate analyses were used to assess the association of the demographic variables with the test scores. Comparison of test scores between groups was made using a t test/Wilcoxon rank sum (2 groups) and analysis of variance (ANOVA)/Kruskal‐Wallis (3 or more groups). The associations of number of prior procedures attempted and self‐reported confidence with test scores was explored using Spearman's correlation. Inferences were made at the 0.05 level of significance, using 2‐tailed tests. Statistical analyses were performed using SPSS 15.0 (SPSS, Inc., Chicago, IL).
Results
Of the 192 internal medicine residents who consented to participate in the study between February and June 2006, 188 completed the initial and repeat test. Subject characteristics are detailed in Table 1.
| Number (%) | |
|---|---|
| Total residents | 192 |
| Males | 113 (59) |
| Year of residency training | |
| First | 101 (52) |
| Second | 64 (33) |
| Third/fourth | 27 (14) |
| Anticipated career path | |
| General medicine/primary care | 26 (14) |
| Critical care | 47 (24) |
| Medical subspecialties | 54 (28) |
| Undecided/other | 65 (34) |
Reliability of the 32‐item instrument measured by Cronbach's was 0.79 and its test‐retest reliability was 0.82. The items difficulty mean was 0.52 with a corrected point biserial correlation mean of 0.26. The test was of high difficulty, with a mean overall score of 50% (median 53%, interquartile range 44‐59%). Baseline scores differed significantly by residency program (P = 0.03). Residents with anticipated careers in critical care had significantly higher scores than those with anticipated careers in primary care (median scores critical care 56%, primary care and other nonprocedural medical subspecialties 50%, P = 0.01).
Residents in their final year reported performing a median of 13 arterial lines, 14 central venous lines, and 3 thoracenteses over the course of their residency training (Table 2). Increase in the number of performed procedures (central lines, arterial lines, and thoracenteses) was associated with an increase in test score (Spearman's correlation coefficient 0.35, P < 0.001). Residents in the highest and lowest decile of procedures performed had median scores of 56% and 43%, respectively (P < 0.001). Increasing seniority in residency was associated with an increase in overall test scores (median score by program year 49%, 54%, 50%, and 64%, P = 0.02).
| Year of Residency Training | Median Number of Procedures (Interquartile Range) | ||
|---|---|---|---|
| Arterial Line Insertion | Central Venous Line Insertion | Thoracentesis | |
| First | 1 (03) | 1 (04) | 0 (01) |
| Second | 8.5 (618) | 10 (518) | 2 (04) |
| Third/fourth | 13 (820) | 14 (1027) | 3 (26) |
Increase in self‐reported confidence was significantly associated with an increase in the number of performed procedures (Spearman's correlation coefficients for central line 0.83, arterial lines 0.76, and thoracentesis 0.78, all P < 0.001) and increasing seniority (0.66, 0.59, and 0.52, respectively, all P < 0.001).
Discussion
The determination of procedural competence has long been a challenge for trainers and internal medicine programs; methods for measuring procedural skills have not been rigorously studied. Procedural competence requires a combination of theoretical knowledge and practical skill. However, given the declining number of procedures performed by internists,4 the new ABIM guidelines mandate cognitive competence in contrast to the demonstration of hands‐on procedural proficiency.
We therefore sought to develop and validate the results of an examination of the theoretical knowledge necessary to perform 3 procedures associated with potentially serious complications. Following establishment of content evidence, item performance characteristics and postadministration interviews were used to develop a 32‐item test. We confirmed the test's internal structure by assessment of reliability and assessed the association of test scores with other variables for which correlation would be expected.
We found that residents performed poorly on test content considered to be important by procedure specialists. These findings highlight the limitations in current procedure training that is frequently sporadic and often variable. The numbers of procedures reported over the duration of residency by residents at these centers were low. It is unclear if the low number of procedures performed was due to limitations in resident content knowledge or if it reflects the increasing use of interventional services with fewer opportunities for experiential learning. Nevertheless, an increasing number of prior procedures was associated with higher self‐reported confidence for all procedures and translated to higher test scores.
This study was limited to 4 teaching hospitals and further studies may be needed to investigate the wider generalizability of the study instrument. However, participants were from 3 distinct internal medicine residency programs that included both community and university hospitals. We relied on resident self‐reports and did not independently verify the number of prior procedures performed. However, similar assumptions have been made in prior studies that physicians who rarely perform procedures are able to provide reasonable estimates of the total number performed.3
The reliability of the 32‐item test (Cronbach's = 0.79) is in the expected range for this length of test and indicates good reliability.9, 10 Given the potential complications associated with advanced medical procedures, there is increasing need to establish criteria for competence. Although we have not established a score threshold, the development of this validated tool to assess procedural knowledge is an important step toward establishing such a goal.
This test may facilitate efforts by hospitalists and others to evaluate the efficacy and refine existing methods of procedure training. Feedback to educators using this assessment tool may assist in the improvement of teaching strategies. In addition, the assessment of cognitive competence in procedure‐related knowledge using a rigorous and reliable means of assessment such as outlined in this study may help identify residents who need further training. Recognition for the necessity for additional training and oversight are likely to be especially important if residents are expected to perform procedures safely yet have fewer opportunities for practice.
Acknowledgements
The authors thank Dr. Stephen Wright, Haley Hamlin, and Matt Johnston for their contributions to the data collection and analysis.
- ,,, et al.Altering residency curriculum in response to a changing practice environment: use of the Mayo internal medicine residency alumni survey.Mayo Clin Proc.1990;65(6):809–817.
- ,,,,,.Preparation for practice in internal medicine. A study of ten years of residency graduates.Arch Intern Med.1988;148(4):853–856.
- ,,,,,.Procedural experience and comfort level in internal medicine trainees.J Gen Intern Med.2000;15(10):716–722.
- .Training internists in procedural skills.Ann Intern Med.1992;116(12 Pt 2):1091–1093.
- ABIM. Policies and Procedures for Certification in Internal Medicine2008. Available at: http://www.abim.org/certification/policies/imss/im.aspx. Accessed August 2009.
- ,,, et al.Confidence of graduating internal medicine residents to perform ambulatory procedures.J Gen Intern Med.2000;15(6):361–365.
- ,,.The lasting value of clinical skills.JAMA.1985;254(1):70–76.
- ,,.Are commonly used resident measurements associated with procedural skills in internal medicine residency training?J Gen Intern Med.2007;22(3):357–361.
- .Psychometric Theory.New York:McGraw Hill;1978.
- .Coefficient alpha and the internal structure of tests.Psychometrika.1951;16:297–334.
- ,,, et al.Altering residency curriculum in response to a changing practice environment: use of the Mayo internal medicine residency alumni survey.Mayo Clin Proc.1990;65(6):809–817.
- ,,,,,.Preparation for practice in internal medicine. A study of ten years of residency graduates.Arch Intern Med.1988;148(4):853–856.
- ,,,,,.Procedural experience and comfort level in internal medicine trainees.J Gen Intern Med.2000;15(10):716–722.
- .Training internists in procedural skills.Ann Intern Med.1992;116(12 Pt 2):1091–1093.
- ABIM. Policies and Procedures for Certification in Internal Medicine2008. Available at: http://www.abim.org/certification/policies/imss/im.aspx. Accessed August 2009.
- ,,, et al.Confidence of graduating internal medicine residents to perform ambulatory procedures.J Gen Intern Med.2000;15(6):361–365.
- ,,.The lasting value of clinical skills.JAMA.1985;254(1):70–76.
- ,,.Are commonly used resident measurements associated with procedural skills in internal medicine residency training?J Gen Intern Med.2007;22(3):357–361.
- .Psychometric Theory.New York:McGraw Hill;1978.
- .Coefficient alpha and the internal structure of tests.Psychometrika.1951;16:297–334.
Copyright © 2009 Society of Hospital Medicine
Physician Assistant/Hospitalist Service
Midlevel providers (physician assistants and nurse practitioners) have long been employed by academic medical centers, predominantly on surgical services, or on medical subspecialty services, where they have typically had a limited scope of practice, focused in a narrowly defined area or set of procedures.17 In contrast, there are relatively few reports of experiences deploying midlevel providers to replace house staff on inpatient general medicine services in academic centers,810 and few studies of the effect of midlevel providers on quality and efficiency of care in the academic setting. Despite this, reductions in house officer duty hours as mandated by the Accreditation Council on Graduate Medical Education (ACGME)11 have resulted in academic centers increasingly using midlevel providers to decrease house staff workload on inpatient services.12, 13 In general, midlevel practitioners on general medicine services have been deployed to: (1) care for a population of patients separate from and in parallel with house staff; this population may be narrowly defined (eg, patients with chest pain) or not; (2) assist with the management of patients cared for by house staff by performing certain tasks (eg, scheduling appointments, discharging patients). Even as midlevel providers become more prevalent on academic general medicine services, the best model of care incorporating them into clinical care remains unclear, and few studies have rigorously examined the care provided on services that use them.
We developed an inpatient general medicine service within a large academic medical center staffed by physician assistants and hospitalists to help our residency program meet ACGME duty hour requirements. We hypothesized that by creating a service that is geographically localized and supervised by full‐time hospitalists, by instituting multidisciplinary rounds, and by investing in the professional development of highly‐skilled physician assistants, we could provide care for medically complex, acutely ill general medicine inpatients with similar quality and efficiency as compared to house staff teams. We report our experience during the first year of implementing the service, and compare quality and efficiency of care on this service with that of our traditional house staff services. We also evaluate the effects of this service on patient satisfaction and self‐reported house staff workload.
PATIENTS AND METHODS
Study Setting
The study was conducted in a 747‐bed urban, academic medical center in the northeastern United States. The hospital's human research committee reviewed and approved the study design. The hospital has accredited residency and fellowship programs in all major specialties. Prior to July 2005, physician assistants were employed only on surgical and medical subspecialty services (ie, bone marrow transplant, interventional cardiology); none were employed on the inpatient general medicine service. There were approximately 44,000 inpatient admissions during the year of the study, with approximately 6500 of these to the general medicine service.
Description of the General Medicine Service
The General Medicine Service consisted of 8 traditional house staff teams, with 1 attending, 1 junior or senior resident, 2 interns, and 1 or 2 medical students. These teams admitted patients on a rotating basis every fourth day. On 4 of these teams, the attending was a hospitalist, with clinical responsibility for the majority of the patients admitted to the team. On the remaining 4 teams, the teaching attending was a primary care physician or medical subspecialist, responsible for the direct care of a small number of the team's patients, with the remainder cared for by private primary care physicians or subspecialists.
Description of the Physician Assistant/Hospitalist Service
The Physician Assistant/Clinician Educator (PACE) service opened in July 2005, and consisted of 15 beds localized to 2 adjacent inpatient pods, staffed by a single cadre of nurses and medically staffed by 1 hospitalist and 2 physician assistants from 7:00 AM to 7:00 PM on weekdays and by 1 hospitalist, 1 physician assistant, and 1 moonlighter (usually a senior medical resident or fellow) from 7:00 AM to 7:00 PM on weekends. A moonlighter, typically a senior resident or medical subspecialty fellow, admitted patients and covered nights on the service from 7:00 PM to 7:00 AM 7 days a week. The daily census goal for the service was 15 patients, limited by the number of available beds on the 2 pods, and the service accepted admissions 24 hours per day, 7 days per week, whenever beds were available. Daily morning rounds occurred at 8:00 AM and included the hospitalist, physician assistants, nurses, a care coordinator, and a pharmacist. The PACE service did not have triage guidelines related to diagnosis, complexity, or acuity, but only accepted patients via the emergency department or via a primary care physician's office, and did not accept patients transferred from outside hospitals or from the intensive care units.
Physician Assistants
All of the physician assistants on the PACE service had prior inpatient medicine experience, ranging from 6 months to 5 years. The physician assistants worked in 3‐day to 6‐day blocks of 12‐hour shifts. Their clinical responsibilities were similar to those of interns at the study hospital, and included taking histories and performing physical examinations, writing notes and orders, reviewing and assimilating data, creating and updating patient signouts, completing discharge summaries, consulting other services as needed, and communicating with nurses and family members.
Many physician assistants also had nonclinical responsibilities, taking on physician‐mentored roles in education, quality improvement, and administration. They were involved in several initiatives: (1) developing a physician assistant curriculum in hospital medicine, (2) presenting at hospital‐wide physician assistant grand rounds, (3) surveying and tracking patient and family satisfaction on the service, (4) reviewing all 72‐hour hospital readmissions, intensive care unit transfers, and deaths on the service, and (5) maintaining the service's compliance with state regulations regarding physician assistant scope of practice and prescribing.
Hospitalists
The 3 hospitalists on the PACE service worked in 7‐day blocks of 12‐hour shifts (7:00 AM to 7:00 PM). They directly supervised the physician assistants and had no competing responsibilities. The hospitalists were all recent graduates of the study hospital's internal medicine residency, with no prior clinical experience beyond residency. All were planning to work on the service for 1 to 2 years before beginning a subspecialty fellowship. In addition to supervising the clinical work of the physician assistants, the hospitalists were responsible for teaching the physician assistants on rounds and in weekly didactic sessions, guided by a curriculum in hospital medicine that focused on the most common general medicine diagnoses seen on the PACE service. The medical director of the PACE service periodically reviewed each physician assistant's clinical experience, skills and knowledge base, and held semiannual feedback sessions.
Study Patients
All general medicine patients admitted to the PACE service from July 1, 2005 to June 30, 2006 comprised the study population. The comparison group consisted of general medicine patients admitted to the 8 house staff general medicine teams; patients transferred from an intensive care unit (ICU) or another facility were excluded in order to match the admission criteria for the PACE service and improve comparability between the 2 study arms.
Data Collection and Study Outcomes
We obtained all patient data from the hospital's administrative databases. We identified patients assigned to the PACE service or to the comparison group based on the admitting service, team, and attending. We obtained patient demographics, insurance, admission source and discharge destination, admission and discharge times, dates, diagnoses, and diagnosis‐related groups (DRGs), as well as dates and times of transfers to other services, including to the intensive care unit. We also obtained the Medicare case‐mix index (CMI, based on DRG weight), and calculated a Charlson score based on billing diagnoses coded in the year prior to the index admission.14 Outcomes included length of stay (LOS) to the nearest hour, in‐hospital mortality, transfers to the intensive care unit, readmissions to the study hospital within 72 hours, 14 days, and 30 days, and total costs as derived from the hospital's cost accounting system (Transition Systems Inc., Boston, MA). Other outcomes included patient satisfaction as measured by responses to the Press‐Ganey survey routinely administered to a randomly selected 70% of recently discharged patients and effect on self‐reported resident work hours.
Statistical Analysis
Patient demographics, clinical characteristics, and study outcomes are presented using proportions, means with standard deviations, and medians with inter‐quartile ranges as appropriate. Unadjusted differences in outcomes between the two services were calculated using univariable regression techniques with service as the independent variable and each outcome as the dependent variable. We used logistic regression for dichotomous outcomes (readmissions, ICU transfers, and inpatient mortality), and linear regression for log‐transformed LOS and log‐transformed total costs of care. To adjust each outcome for potential confounders, we then built multivariable regression models. Each potential confounder was entered into the model one at a time as the independent variable. All variables found to be significant predictors of the outcome at the P < 0.10 level were then retained in the final model along with service as the predictor of interest. We used general estimating equations in all multivariable models to adjust for clustering of patients by attending physician. For logistic regression models, the effect size is presented as an odds ratio (OR); for log‐transformed linear regression models, the effect size is presented as the percent difference between groups. We also performed 2 subgroup analyses, limited to (1) the patients with the 10 most common discharge DRGs, and (2) patients admitted between the hours of 7:00 AM and 7:00 PM to remove the effects of moonlighters performing the initial admission. Except as noted above, 2‐sided P values < 0.05 were considered significant. SAS 9.1 (SAS Institute, Cary, NC) was used for all analyses.
RESULTS
Patient Demographics
Table 1 shows patient demographics and clinical characteristics of the PACE service and the comparison group. Patients in the comparison group were slightly older and tended to have slightly higher CMI and Charlson scores. Patients on the PACE service were more likely to be admitted at night (10:00 PM to 7:00 AM; 43.8% versus 30.3%; P < 0.0001). There were no significant differences in sex, race, insurance, or percentage of patients discharged to home. The 10 most common DRGs in the comparison group accounted for 37.0% of discharges, and these same DRGs accounted for 37.5% of discharges on the PACE service (Table 2).
| Characteristic | PACE Service (n = 992) | House Staff Services (n = 4,202) | P value |
|---|---|---|---|
| |||
| Age (years) | |||
| 1844 | 19.1 | 18.2 | |
| 4564 | 35.5 | 31.9 | 0.04 |
| 65+ | 45.5 | 49.9 | |
| Sex (% female) | 57.7 | 60.0 | NS |
| Race/ethnicity | |||
| White | 57.3 | 59.3 | |
| Black | 24.0 | 23.5 | NS |
| Hispanic | 14.1 | 13.3 | |
| Other | 4.6 | 3.9 | |
| Insurance | |||
| Medicare | 41.9 | 43.8 | |
| Commercial | 34.9 | 35.9 | |
| Medicaid | 14.4 | 11.7 | NS |
| Free care | 4.5 | 3.9 | |
| Self pay | 1.1 | 0.8 | |
| Median income by zip code of residence, USD (IQR) | 45,517 (32,49362,932) | 45,517 (35,88963,275) | NS |
| Case‐mix index, median (IQR) | 1.1 (0.81.5) | 1.2 (0.91.8) | 0.001 |
| Charlson score | |||
| 0 | 27.2 | 24.9 | |
| 1 | 22.6 | 21.1 | 0.02 |
| 2 | 16.2 | 16.5 | |
| 3+ | 34.0 | 37.6 | |
| Admissions between 10:00 PM and 7:00 AM | 43.8 | 30.3 | <0.0001 |
| Discharged to home | 81.1 | 80.5 | NS |
| Diagnosis‐Related Group at Discharge | PACE Service (n = 992)* | House Staff Services (n = 4,202)* |
|---|---|---|
| ||
| Chest pain | 5.4 | 6.4 |
| Esophagitis, gastroenteritis, and miscellaneous digestive disorders | 4.5 | 4.4 |
| Heart failure and shock | 3.4 | 4.6 |
| Simple pneumonia and pleurisy | 2.7 | 4.4 |
| Kidney and urinary tract infections | 4.7 | 3.2 |
| Chronic obstructive pulmonary disease | 4.0 | 3.3 |
| Renal failure | 2.7 | 3.5 |
| Gastrointestinal hemorrhage | 3.7 | 2.7 |
| Nutritional and miscellaneous metabolic disorders | 3.3 | 2.4 |
| Disorders of the pancreas except malignancy | 3.1 | 2.1 |
| Cumulative percent | 37.5 | 37.0 |
Efficiency and Quality of Care
Table 3 compares the performance of the PACE service and the comparison group on several efficiency and quality measures. Unadjusted LOS was not significantly different, and adjusted LOS was slightly but not statistically significantly higher on the study service (adjusted LOS 5.0% higher; 95% confidence interval [CI], 0.4% to +10%). Unadjusted and adjusted total costs of care were marginally lower on the study service (adjusted total cost of care 3.9% lower; 95% CI, 7.5% to 0.3%).
| PACE Service | House Staff Services | Unadjusted % Difference (95%CI) | Adjusted % Difference (95%CI)* | |
|---|---|---|---|---|
| PACE Service | House Staff Services | Unadjusted OR (95% CI) | Adjusted OR (95% CI) | |
| ||||
| Efficiency measure | ||||
| Length of stay, days, median (IQR) | 2.6 (1.6, 4.4) | 2.6 (1.4, 4.6) | +0.1% (5.6% to +6.1%) | +5.0% (0.4% to +10.0%) |
| Total costs, USD, median (IQR) | 4,536 (2,848, 7,201) | 4,749 (3,046, 8,161) | 9.1% (14.0% to 3.8%) | 3.9% (7.5% to 0.3%)‖ |
| Quality measure | ||||
| 72‐hour readmissions/100 discharges | 0.8 | 1.3 | 0.6 (0.31.3) | 0.7 (0.21.8) |
| 14‐day readmissions/100 discharges | 5.4 | 5.4 | 1.0 (0.71.4) | 1.1 (0.81.4) |
| 30‐day readmissions/100 discharges | 8.0 | 8.1 | 1.0 (0.81.3) | 1.1 (0.91.3) |
| ICU transfers/100 discharges | 2.0 | 2.3 | 0.9 (0.51.4) | 1.4 (0.82.4)# |
| Inpatient mortality/100 discharges | 0.7 | 1.2 | 0.6 (0.31.3) | 0.8 (0.31.8)** |
We found no differences between the PACE service and comparison group in unadjusted rates of hospital readmissions within 72 hours, 14 days, and 30 days, transfer to the intensive care units, or inpatient mortality (Table 3). The associated ORs for each outcome were similar after adjusting for patient demographics and clinical characteristics including severity of illness, as well as for clustering by attending physician.
Subgroup Analyses
When the analysis was limited to the subset of patients with the 10 most common discharge DRGs, the difference in adjusted total cost of care was similar but lost statistical significance (4.0% lower on PACE service; 95% CI, 11.0% to +3.3%). In this subgroup, LOS, readmission rates, and ICU transfer rates were not different. ORs for mortality could not be calculated because there were no deaths in this subgroup on the PACE service (data not shown). When analysis was limited to daytime admissions (to remove any potential effect of admitting by a moonlighter), the difference in total cost of care was attenuated and lost statistical significance (0.2% lower on PACE service; 95%CI, 5.9% to +5.5%). No differences were seen in LOS, mortality, and ICU transfers (data not shown). However, 14‐day readmissions (but not 72‐hour or 30‐day readmissions) were lower on the PACE service (OR, 0.49; 95% CI, 0.25‐0.93).
Patient Satisfaction
Patients were similarly satisfied with their care on the PACE service and on the house staff services. In specific areas and globally, percentages of patients satisfied with their physicians and with the discharge process were not different, as measured by the Press‐Ganey survey (Press‐Ganey Associates, South Bend, IN; Figures 1 and 2). The survey distinguishes between attendings and residents, but not physician assistants; therefore, Figure 1 only includes responses to the attending questions. Given the sampling procedure of the Press‐Ganey survey, exact response rates cannot be calculated, but Press‐Ganey reports a response rate of about 40% for the English survey and about 20% for the Spanish survey.
Resident Duty Hours
Comparing the same month 1 year prior to implementation of the PACE service, mean self‐reported resident duty hours on the general medicine service were unchanged; however, self‐reported data were incomplete, and multiple changes took place in the residency program during the study period. For example, implementation of the PACE service allowed for the dissolution of one full house staff general medicine team and redistribution of these house staff to night float positions and an expanded medical intensive care unit.
Costs of Implementation
The costs associated with implementing the PACE service included physician and physician assistant salaries (2.5 full‐time physicians, 5 full‐time physician assistants, plus fringe) and night coverage by resident and fellow moonlighters (without fringe, and estimated at 50% effort given other moonlighter coverage responsibilities on subspecialty services). We estimated these costs at $257.50/patient‐day ($115/patient‐day for attending physician compensation, $110/patient‐day for physician assistant compensation, and $32.50/patient‐day for moonlighting coverage).
DISCUSSION
As academic centers struggle with developing a workforce to provide patient care no longer provided by residents, questions about the ideal structure of nonhouse staff inpatient services abound. Although solutions to this problem will be determined to some extent by local factors such as institutional culture and resources, some lessons learned in developing such services will be more widely applicable. We found that by implementing a geographically localized, physician assistant‐staffed hospitalist service, we were able to provide care of similar quality and efficiency to that of traditional house staff services, despite inexperienced hospitalists staffing the service and a medical residency program commonly recognized as one of the best in the country. Adjusted total costs were slightly lower on the PACE service, but this difference was small and of borderline statistical significance. Likewise, no significant differences were seen in any of several quality measures or in patient satisfaction.
Our findings add to the available evidence supporting the use of physician assistants on academic general medicine services, and are germane to academic centers facing reductions in house staff availability and seeking alternative models of care for inpatients. Several specific characteristics of the PACE service and the implications of these should be considered:
The service accepted all patients, regardless of diagnosis, acuity, or complexity of illness. This was unlike many previously described nonhouse staff services which were more limited in scope, and allowed more flexibility with patient flow. However, in the end, patients on the PACE service did have a modestly lower case mix index and Charlson score, suggesting that, despite a lack of triage guidelines, there was some bias in the triage of admissions, possibly due to a perception that physician assistants should take care of lower complexity patients. If it is desirable to have a similar distribution of higher complexity patients across house staff and nonhouse staff services, extra efforts may be necessary to overcome this perception.
The service was geographically regionalized. Geographic regionalization offered many important advantages, especially with regards to communication among staff, nursing, and consultants, and allowed for multidisciplinary rounds. However, it is possible that the modest, but not statistically significant, trend toward an increased LOS seen on the PACE service might be a reflection of geographic admitting (less incentive to discharge since discharging a patient means taking a new admission).
The education and professional development of the physician assistants was a priority. Physician assistants had considerable autonomy and responsibility, and rather than being assigned only lower level administrative tasks, performed all aspects of patient care. They also received regular teaching from the hospitalists, attended house staff teaching conferences, and developed nonclinical roles in education and quality improvement. The higher standards expected of the physician assistants were quite possibly a factor in the quality of care delivered, and almost certainly contributed to physician assistant satisfaction and retention.
Our findings contrast with those of Myers et al.,9 who found that a nonteaching service staffed by hospitalists and nurse practitioners had a significantly lower median LOS and hospital charges compared to similar patients on resident‐based services. However, unlike ours, their service cared for a select patient population, and only accepted patients with chest pain at low risk for acute coronary syndrome. Van Rhee et al.10 found that physician assistants on a general medicine service used fewer resources for patients with pneumonia, stroke, and congestive heart failure than resident physicians, and did not exceed the resources used by residents in other diagnoses. The authors did not find a difference in LOS, but did find a significantly higher mortality among patients with pneumonia cared for by physician assistants.
Several limitations should be noted. First, the study was a retrospective analysis of administrative data rather than a randomized trial, and although we employed a standard approach to adjust for a wide range of patient characteristics including severity of illness, there may have been undetected differences in the patient populations studied that may have confounded our results. Second, resident moonlighters admitted patients to the PACE service and, at other times, to the house staff services, and this may have diluted any differences between the groups. However, when we limited our analysis to the subgroup of patients admitted during the day, similar results were obtained, with the exception that the PACE service had a lower rate of 14‐day readmissions, an unexpected finding deserving of further study. Third, the study was conducted in a single academic institution and our findings may not be generalizable to others with different needs and resources; indeed, the costs associated with implementing such a service may be prohibitive for some institutions. Fourth, because of simultaneous changes that were taking place in our residency program, we are unable to accurately assess the impact of the PACE service on resident duty hours. However, resident duty hours did not increase over this time period on the general medicine service, and implementation of the service allowed for redistribution of house staff to other services and positions. Fifth, patient satisfaction data were obtained from responses to the mailed Press‐Ganey survey, to which there is a relatively low response rate. Also, we did not survey providers regarding their satisfaction with the service during the study period. Sixth, the study had limited power to detect clinically important differences in mortality and ICU transfers. Finally, this study is unable to compare this particular model of incorporating midlevel providers into general medical services with other models, only with traditional house staff services.
Future research should focus on determining the most effective and efficient ways to incorporate midlevel providers on academic general medicine services. One important question from the standpoint of house staff training is whether such services should be separate but equal, or should house staff gain experience during residency working with midlevel providers, since they are likely to encounter them in the future whether they stay in academics or not. Different models of care will likely have large implications for the quality and efficiency of patient care, house staff education and satisfaction, and physician assistant job satisfaction and turnover.
In summary, our study demonstrates that a geographically regionalized, multidisciplinary service staffed by hospitalists and physician assistants can be a safe alternative to house staff‐based services for the care of general medicine inpatients in an academic medical center.
- ,,,,,.The physician's assistant as resident on surgical service. An example of creative problem solving in surgical manpower.Arch Surg.1980;115:310–314.
- ,,,.Coronary arteriography performed by a physician assistant.Am J Cardiol.1987;60:784–787.
- .The specialized physician assistant: an alternative to the clinical cardiology trainee.Am J Cardiol.1987;60:901–902.
- ,,.One hospital's successful 20‐year experience with physician assistants in graduate medical education.Acad Med.1999;74:641–645.
- ,.Physicians assistants in cardiothoracic surgery: a 30‐year experience in a university center.Ann Thorac Surg.2006;81:195–199; discussion 199–200.
- ,,.Comparative review of use of physician assistants in a level I trauma center.Am Surg.2004;70:272–279.
- ,,,,.Integrating midlevel practitioners into a teaching service.Am J Surg.2006;192:119–124.
- ,.Acute care nurse practitioners: creating and implementing a model of care for an inpatient general medical service.Am J Crit Care.2002;11:448–458.
- ,,,,.Improving resource utilization in a teaching hospital: development of a nonteaching service for chest pain admissions.Acad Med.2006;81:432–435.
- ,,.Resource use by physician assistant services versus teaching services.JAAPA.2002;15:33–38.
- ,,, for the ACGME Work Group on Resident Duty Hours, Accreditation Council for Graduate Medical Education.New requirements for resident duty hours.JAMA.2002;288:1112–1114.
- ,,.The substitution of physician assistants and nurse practitioners for physician residents in teaching hospitals.Health Aff.1995;14:181–191.
- ,,,,.Challenges of the 80‐hour resident work rules: collaboration between surgeons and nonphysician practitioners.Surg Clin North Am.2004;84:1573–1586.
- ,,.Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases.J Clin Epidemiol.1992;45:613–619.
Midlevel providers (physician assistants and nurse practitioners) have long been employed by academic medical centers, predominantly on surgical services, or on medical subspecialty services, where they have typically had a limited scope of practice, focused in a narrowly defined area or set of procedures.17 In contrast, there are relatively few reports of experiences deploying midlevel providers to replace house staff on inpatient general medicine services in academic centers,810 and few studies of the effect of midlevel providers on quality and efficiency of care in the academic setting. Despite this, reductions in house officer duty hours as mandated by the Accreditation Council on Graduate Medical Education (ACGME)11 have resulted in academic centers increasingly using midlevel providers to decrease house staff workload on inpatient services.12, 13 In general, midlevel practitioners on general medicine services have been deployed to: (1) care for a population of patients separate from and in parallel with house staff; this population may be narrowly defined (eg, patients with chest pain) or not; (2) assist with the management of patients cared for by house staff by performing certain tasks (eg, scheduling appointments, discharging patients). Even as midlevel providers become more prevalent on academic general medicine services, the best model of care incorporating them into clinical care remains unclear, and few studies have rigorously examined the care provided on services that use them.
We developed an inpatient general medicine service within a large academic medical center staffed by physician assistants and hospitalists to help our residency program meet ACGME duty hour requirements. We hypothesized that by creating a service that is geographically localized and supervised by full‐time hospitalists, by instituting multidisciplinary rounds, and by investing in the professional development of highly‐skilled physician assistants, we could provide care for medically complex, acutely ill general medicine inpatients with similar quality and efficiency as compared to house staff teams. We report our experience during the first year of implementing the service, and compare quality and efficiency of care on this service with that of our traditional house staff services. We also evaluate the effects of this service on patient satisfaction and self‐reported house staff workload.
PATIENTS AND METHODS
Study Setting
The study was conducted in a 747‐bed urban, academic medical center in the northeastern United States. The hospital's human research committee reviewed and approved the study design. The hospital has accredited residency and fellowship programs in all major specialties. Prior to July 2005, physician assistants were employed only on surgical and medical subspecialty services (ie, bone marrow transplant, interventional cardiology); none were employed on the inpatient general medicine service. There were approximately 44,000 inpatient admissions during the year of the study, with approximately 6500 of these to the general medicine service.
Description of the General Medicine Service
The General Medicine Service consisted of 8 traditional house staff teams, with 1 attending, 1 junior or senior resident, 2 interns, and 1 or 2 medical students. These teams admitted patients on a rotating basis every fourth day. On 4 of these teams, the attending was a hospitalist, with clinical responsibility for the majority of the patients admitted to the team. On the remaining 4 teams, the teaching attending was a primary care physician or medical subspecialist, responsible for the direct care of a small number of the team's patients, with the remainder cared for by private primary care physicians or subspecialists.
Description of the Physician Assistant/Hospitalist Service
The Physician Assistant/Clinician Educator (PACE) service opened in July 2005, and consisted of 15 beds localized to 2 adjacent inpatient pods, staffed by a single cadre of nurses and medically staffed by 1 hospitalist and 2 physician assistants from 7:00 AM to 7:00 PM on weekdays and by 1 hospitalist, 1 physician assistant, and 1 moonlighter (usually a senior medical resident or fellow) from 7:00 AM to 7:00 PM on weekends. A moonlighter, typically a senior resident or medical subspecialty fellow, admitted patients and covered nights on the service from 7:00 PM to 7:00 AM 7 days a week. The daily census goal for the service was 15 patients, limited by the number of available beds on the 2 pods, and the service accepted admissions 24 hours per day, 7 days per week, whenever beds were available. Daily morning rounds occurred at 8:00 AM and included the hospitalist, physician assistants, nurses, a care coordinator, and a pharmacist. The PACE service did not have triage guidelines related to diagnosis, complexity, or acuity, but only accepted patients via the emergency department or via a primary care physician's office, and did not accept patients transferred from outside hospitals or from the intensive care units.
Physician Assistants
All of the physician assistants on the PACE service had prior inpatient medicine experience, ranging from 6 months to 5 years. The physician assistants worked in 3‐day to 6‐day blocks of 12‐hour shifts. Their clinical responsibilities were similar to those of interns at the study hospital, and included taking histories and performing physical examinations, writing notes and orders, reviewing and assimilating data, creating and updating patient signouts, completing discharge summaries, consulting other services as needed, and communicating with nurses and family members.
Many physician assistants also had nonclinical responsibilities, taking on physician‐mentored roles in education, quality improvement, and administration. They were involved in several initiatives: (1) developing a physician assistant curriculum in hospital medicine, (2) presenting at hospital‐wide physician assistant grand rounds, (3) surveying and tracking patient and family satisfaction on the service, (4) reviewing all 72‐hour hospital readmissions, intensive care unit transfers, and deaths on the service, and (5) maintaining the service's compliance with state regulations regarding physician assistant scope of practice and prescribing.
Hospitalists
The 3 hospitalists on the PACE service worked in 7‐day blocks of 12‐hour shifts (7:00 AM to 7:00 PM). They directly supervised the physician assistants and had no competing responsibilities. The hospitalists were all recent graduates of the study hospital's internal medicine residency, with no prior clinical experience beyond residency. All were planning to work on the service for 1 to 2 years before beginning a subspecialty fellowship. In addition to supervising the clinical work of the physician assistants, the hospitalists were responsible for teaching the physician assistants on rounds and in weekly didactic sessions, guided by a curriculum in hospital medicine that focused on the most common general medicine diagnoses seen on the PACE service. The medical director of the PACE service periodically reviewed each physician assistant's clinical experience, skills and knowledge base, and held semiannual feedback sessions.
Study Patients
All general medicine patients admitted to the PACE service from July 1, 2005 to June 30, 2006 comprised the study population. The comparison group consisted of general medicine patients admitted to the 8 house staff general medicine teams; patients transferred from an intensive care unit (ICU) or another facility were excluded in order to match the admission criteria for the PACE service and improve comparability between the 2 study arms.
Data Collection and Study Outcomes
We obtained all patient data from the hospital's administrative databases. We identified patients assigned to the PACE service or to the comparison group based on the admitting service, team, and attending. We obtained patient demographics, insurance, admission source and discharge destination, admission and discharge times, dates, diagnoses, and diagnosis‐related groups (DRGs), as well as dates and times of transfers to other services, including to the intensive care unit. We also obtained the Medicare case‐mix index (CMI, based on DRG weight), and calculated a Charlson score based on billing diagnoses coded in the year prior to the index admission.14 Outcomes included length of stay (LOS) to the nearest hour, in‐hospital mortality, transfers to the intensive care unit, readmissions to the study hospital within 72 hours, 14 days, and 30 days, and total costs as derived from the hospital's cost accounting system (Transition Systems Inc., Boston, MA). Other outcomes included patient satisfaction as measured by responses to the Press‐Ganey survey routinely administered to a randomly selected 70% of recently discharged patients and effect on self‐reported resident work hours.
Statistical Analysis
Patient demographics, clinical characteristics, and study outcomes are presented using proportions, means with standard deviations, and medians with inter‐quartile ranges as appropriate. Unadjusted differences in outcomes between the two services were calculated using univariable regression techniques with service as the independent variable and each outcome as the dependent variable. We used logistic regression for dichotomous outcomes (readmissions, ICU transfers, and inpatient mortality), and linear regression for log‐transformed LOS and log‐transformed total costs of care. To adjust each outcome for potential confounders, we then built multivariable regression models. Each potential confounder was entered into the model one at a time as the independent variable. All variables found to be significant predictors of the outcome at the P < 0.10 level were then retained in the final model along with service as the predictor of interest. We used general estimating equations in all multivariable models to adjust for clustering of patients by attending physician. For logistic regression models, the effect size is presented as an odds ratio (OR); for log‐transformed linear regression models, the effect size is presented as the percent difference between groups. We also performed 2 subgroup analyses, limited to (1) the patients with the 10 most common discharge DRGs, and (2) patients admitted between the hours of 7:00 AM and 7:00 PM to remove the effects of moonlighters performing the initial admission. Except as noted above, 2‐sided P values < 0.05 were considered significant. SAS 9.1 (SAS Institute, Cary, NC) was used for all analyses.
RESULTS
Patient Demographics
Table 1 shows patient demographics and clinical characteristics of the PACE service and the comparison group. Patients in the comparison group were slightly older and tended to have slightly higher CMI and Charlson scores. Patients on the PACE service were more likely to be admitted at night (10:00 PM to 7:00 AM; 43.8% versus 30.3%; P < 0.0001). There were no significant differences in sex, race, insurance, or percentage of patients discharged to home. The 10 most common DRGs in the comparison group accounted for 37.0% of discharges, and these same DRGs accounted for 37.5% of discharges on the PACE service (Table 2).
| Characteristic | PACE Service (n = 992) | House Staff Services (n = 4,202) | P value |
|---|---|---|---|
| |||
| Age (years) | |||
| 1844 | 19.1 | 18.2 | |
| 4564 | 35.5 | 31.9 | 0.04 |
| 65+ | 45.5 | 49.9 | |
| Sex (% female) | 57.7 | 60.0 | NS |
| Race/ethnicity | |||
| White | 57.3 | 59.3 | |
| Black | 24.0 | 23.5 | NS |
| Hispanic | 14.1 | 13.3 | |
| Other | 4.6 | 3.9 | |
| Insurance | |||
| Medicare | 41.9 | 43.8 | |
| Commercial | 34.9 | 35.9 | |
| Medicaid | 14.4 | 11.7 | NS |
| Free care | 4.5 | 3.9 | |
| Self pay | 1.1 | 0.8 | |
| Median income by zip code of residence, USD (IQR) | 45,517 (32,49362,932) | 45,517 (35,88963,275) | NS |
| Case‐mix index, median (IQR) | 1.1 (0.81.5) | 1.2 (0.91.8) | 0.001 |
| Charlson score | |||
| 0 | 27.2 | 24.9 | |
| 1 | 22.6 | 21.1 | 0.02 |
| 2 | 16.2 | 16.5 | |
| 3+ | 34.0 | 37.6 | |
| Admissions between 10:00 PM and 7:00 AM | 43.8 | 30.3 | <0.0001 |
| Discharged to home | 81.1 | 80.5 | NS |
| Diagnosis‐Related Group at Discharge | PACE Service (n = 992)* | House Staff Services (n = 4,202)* |
|---|---|---|
| ||
| Chest pain | 5.4 | 6.4 |
| Esophagitis, gastroenteritis, and miscellaneous digestive disorders | 4.5 | 4.4 |
| Heart failure and shock | 3.4 | 4.6 |
| Simple pneumonia and pleurisy | 2.7 | 4.4 |
| Kidney and urinary tract infections | 4.7 | 3.2 |
| Chronic obstructive pulmonary disease | 4.0 | 3.3 |
| Renal failure | 2.7 | 3.5 |
| Gastrointestinal hemorrhage | 3.7 | 2.7 |
| Nutritional and miscellaneous metabolic disorders | 3.3 | 2.4 |
| Disorders of the pancreas except malignancy | 3.1 | 2.1 |
| Cumulative percent | 37.5 | 37.0 |
Efficiency and Quality of Care
Table 3 compares the performance of the PACE service and the comparison group on several efficiency and quality measures. Unadjusted LOS was not significantly different, and adjusted LOS was slightly but not statistically significantly higher on the study service (adjusted LOS 5.0% higher; 95% confidence interval [CI], 0.4% to +10%). Unadjusted and adjusted total costs of care were marginally lower on the study service (adjusted total cost of care 3.9% lower; 95% CI, 7.5% to 0.3%).
| PACE Service | House Staff Services | Unadjusted % Difference (95%CI) | Adjusted % Difference (95%CI)* | |
|---|---|---|---|---|
| PACE Service | House Staff Services | Unadjusted OR (95% CI) | Adjusted OR (95% CI) | |
| ||||
| Efficiency measure | ||||
| Length of stay, days, median (IQR) | 2.6 (1.6, 4.4) | 2.6 (1.4, 4.6) | +0.1% (5.6% to +6.1%) | +5.0% (0.4% to +10.0%) |
| Total costs, USD, median (IQR) | 4,536 (2,848, 7,201) | 4,749 (3,046, 8,161) | 9.1% (14.0% to 3.8%) | 3.9% (7.5% to 0.3%)‖ |
| Quality measure | ||||
| 72‐hour readmissions/100 discharges | 0.8 | 1.3 | 0.6 (0.31.3) | 0.7 (0.21.8) |
| 14‐day readmissions/100 discharges | 5.4 | 5.4 | 1.0 (0.71.4) | 1.1 (0.81.4) |
| 30‐day readmissions/100 discharges | 8.0 | 8.1 | 1.0 (0.81.3) | 1.1 (0.91.3) |
| ICU transfers/100 discharges | 2.0 | 2.3 | 0.9 (0.51.4) | 1.4 (0.82.4)# |
| Inpatient mortality/100 discharges | 0.7 | 1.2 | 0.6 (0.31.3) | 0.8 (0.31.8)** |
We found no differences between the PACE service and comparison group in unadjusted rates of hospital readmissions within 72 hours, 14 days, and 30 days, transfer to the intensive care units, or inpatient mortality (Table 3). The associated ORs for each outcome were similar after adjusting for patient demographics and clinical characteristics including severity of illness, as well as for clustering by attending physician.
Subgroup Analyses
When the analysis was limited to the subset of patients with the 10 most common discharge DRGs, the difference in adjusted total cost of care was similar but lost statistical significance (4.0% lower on PACE service; 95% CI, 11.0% to +3.3%). In this subgroup, LOS, readmission rates, and ICU transfer rates were not different. ORs for mortality could not be calculated because there were no deaths in this subgroup on the PACE service (data not shown). When analysis was limited to daytime admissions (to remove any potential effect of admitting by a moonlighter), the difference in total cost of care was attenuated and lost statistical significance (0.2% lower on PACE service; 95%CI, 5.9% to +5.5%). No differences were seen in LOS, mortality, and ICU transfers (data not shown). However, 14‐day readmissions (but not 72‐hour or 30‐day readmissions) were lower on the PACE service (OR, 0.49; 95% CI, 0.25‐0.93).
Patient Satisfaction
Patients were similarly satisfied with their care on the PACE service and on the house staff services. In specific areas and globally, percentages of patients satisfied with their physicians and with the discharge process were not different, as measured by the Press‐Ganey survey (Press‐Ganey Associates, South Bend, IN; Figures 1 and 2). The survey distinguishes between attendings and residents, but not physician assistants; therefore, Figure 1 only includes responses to the attending questions. Given the sampling procedure of the Press‐Ganey survey, exact response rates cannot be calculated, but Press‐Ganey reports a response rate of about 40% for the English survey and about 20% for the Spanish survey.
Resident Duty Hours
Comparing the same month 1 year prior to implementation of the PACE service, mean self‐reported resident duty hours on the general medicine service were unchanged; however, self‐reported data were incomplete, and multiple changes took place in the residency program during the study period. For example, implementation of the PACE service allowed for the dissolution of one full house staff general medicine team and redistribution of these house staff to night float positions and an expanded medical intensive care unit.
Costs of Implementation
The costs associated with implementing the PACE service included physician and physician assistant salaries (2.5 full‐time physicians, 5 full‐time physician assistants, plus fringe) and night coverage by resident and fellow moonlighters (without fringe, and estimated at 50% effort given other moonlighter coverage responsibilities on subspecialty services). We estimated these costs at $257.50/patient‐day ($115/patient‐day for attending physician compensation, $110/patient‐day for physician assistant compensation, and $32.50/patient‐day for moonlighting coverage).
DISCUSSION
As academic centers struggle with developing a workforce to provide patient care no longer provided by residents, questions about the ideal structure of nonhouse staff inpatient services abound. Although solutions to this problem will be determined to some extent by local factors such as institutional culture and resources, some lessons learned in developing such services will be more widely applicable. We found that by implementing a geographically localized, physician assistant‐staffed hospitalist service, we were able to provide care of similar quality and efficiency to that of traditional house staff services, despite inexperienced hospitalists staffing the service and a medical residency program commonly recognized as one of the best in the country. Adjusted total costs were slightly lower on the PACE service, but this difference was small and of borderline statistical significance. Likewise, no significant differences were seen in any of several quality measures or in patient satisfaction.
Our findings add to the available evidence supporting the use of physician assistants on academic general medicine services, and are germane to academic centers facing reductions in house staff availability and seeking alternative models of care for inpatients. Several specific characteristics of the PACE service and the implications of these should be considered:
The service accepted all patients, regardless of diagnosis, acuity, or complexity of illness. This was unlike many previously described nonhouse staff services which were more limited in scope, and allowed more flexibility with patient flow. However, in the end, patients on the PACE service did have a modestly lower case mix index and Charlson score, suggesting that, despite a lack of triage guidelines, there was some bias in the triage of admissions, possibly due to a perception that physician assistants should take care of lower complexity patients. If it is desirable to have a similar distribution of higher complexity patients across house staff and nonhouse staff services, extra efforts may be necessary to overcome this perception.
The service was geographically regionalized. Geographic regionalization offered many important advantages, especially with regards to communication among staff, nursing, and consultants, and allowed for multidisciplinary rounds. However, it is possible that the modest, but not statistically significant, trend toward an increased LOS seen on the PACE service might be a reflection of geographic admitting (less incentive to discharge since discharging a patient means taking a new admission).
The education and professional development of the physician assistants was a priority. Physician assistants had considerable autonomy and responsibility, and rather than being assigned only lower level administrative tasks, performed all aspects of patient care. They also received regular teaching from the hospitalists, attended house staff teaching conferences, and developed nonclinical roles in education and quality improvement. The higher standards expected of the physician assistants were quite possibly a factor in the quality of care delivered, and almost certainly contributed to physician assistant satisfaction and retention.
Our findings contrast with those of Myers et al.,9 who found that a nonteaching service staffed by hospitalists and nurse practitioners had a significantly lower median LOS and hospital charges compared to similar patients on resident‐based services. However, unlike ours, their service cared for a select patient population, and only accepted patients with chest pain at low risk for acute coronary syndrome. Van Rhee et al.10 found that physician assistants on a general medicine service used fewer resources for patients with pneumonia, stroke, and congestive heart failure than resident physicians, and did not exceed the resources used by residents in other diagnoses. The authors did not find a difference in LOS, but did find a significantly higher mortality among patients with pneumonia cared for by physician assistants.
Several limitations should be noted. First, the study was a retrospective analysis of administrative data rather than a randomized trial, and although we employed a standard approach to adjust for a wide range of patient characteristics including severity of illness, there may have been undetected differences in the patient populations studied that may have confounded our results. Second, resident moonlighters admitted patients to the PACE service and, at other times, to the house staff services, and this may have diluted any differences between the groups. However, when we limited our analysis to the subgroup of patients admitted during the day, similar results were obtained, with the exception that the PACE service had a lower rate of 14‐day readmissions, an unexpected finding deserving of further study. Third, the study was conducted in a single academic institution and our findings may not be generalizable to others with different needs and resources; indeed, the costs associated with implementing such a service may be prohibitive for some institutions. Fourth, because of simultaneous changes that were taking place in our residency program, we are unable to accurately assess the impact of the PACE service on resident duty hours. However, resident duty hours did not increase over this time period on the general medicine service, and implementation of the service allowed for redistribution of house staff to other services and positions. Fifth, patient satisfaction data were obtained from responses to the mailed Press‐Ganey survey, to which there is a relatively low response rate. Also, we did not survey providers regarding their satisfaction with the service during the study period. Sixth, the study had limited power to detect clinically important differences in mortality and ICU transfers. Finally, this study is unable to compare this particular model of incorporating midlevel providers into general medical services with other models, only with traditional house staff services.
Future research should focus on determining the most effective and efficient ways to incorporate midlevel providers on academic general medicine services. One important question from the standpoint of house staff training is whether such services should be separate but equal, or should house staff gain experience during residency working with midlevel providers, since they are likely to encounter them in the future whether they stay in academics or not. Different models of care will likely have large implications for the quality and efficiency of patient care, house staff education and satisfaction, and physician assistant job satisfaction and turnover.
In summary, our study demonstrates that a geographically regionalized, multidisciplinary service staffed by hospitalists and physician assistants can be a safe alternative to house staff‐based services for the care of general medicine inpatients in an academic medical center.
Midlevel providers (physician assistants and nurse practitioners) have long been employed by academic medical centers, predominantly on surgical services, or on medical subspecialty services, where they have typically had a limited scope of practice, focused in a narrowly defined area or set of procedures.17 In contrast, there are relatively few reports of experiences deploying midlevel providers to replace house staff on inpatient general medicine services in academic centers,810 and few studies of the effect of midlevel providers on quality and efficiency of care in the academic setting. Despite this, reductions in house officer duty hours as mandated by the Accreditation Council on Graduate Medical Education (ACGME)11 have resulted in academic centers increasingly using midlevel providers to decrease house staff workload on inpatient services.12, 13 In general, midlevel practitioners on general medicine services have been deployed to: (1) care for a population of patients separate from and in parallel with house staff; this population may be narrowly defined (eg, patients with chest pain) or not; (2) assist with the management of patients cared for by house staff by performing certain tasks (eg, scheduling appointments, discharging patients). Even as midlevel providers become more prevalent on academic general medicine services, the best model of care incorporating them into clinical care remains unclear, and few studies have rigorously examined the care provided on services that use them.
We developed an inpatient general medicine service within a large academic medical center staffed by physician assistants and hospitalists to help our residency program meet ACGME duty hour requirements. We hypothesized that by creating a service that is geographically localized and supervised by full‐time hospitalists, by instituting multidisciplinary rounds, and by investing in the professional development of highly‐skilled physician assistants, we could provide care for medically complex, acutely ill general medicine inpatients with similar quality and efficiency as compared to house staff teams. We report our experience during the first year of implementing the service, and compare quality and efficiency of care on this service with that of our traditional house staff services. We also evaluate the effects of this service on patient satisfaction and self‐reported house staff workload.
PATIENTS AND METHODS
Study Setting
The study was conducted in a 747‐bed urban, academic medical center in the northeastern United States. The hospital's human research committee reviewed and approved the study design. The hospital has accredited residency and fellowship programs in all major specialties. Prior to July 2005, physician assistants were employed only on surgical and medical subspecialty services (ie, bone marrow transplant, interventional cardiology); none were employed on the inpatient general medicine service. There were approximately 44,000 inpatient admissions during the year of the study, with approximately 6500 of these to the general medicine service.
Description of the General Medicine Service
The General Medicine Service consisted of 8 traditional house staff teams, with 1 attending, 1 junior or senior resident, 2 interns, and 1 or 2 medical students. These teams admitted patients on a rotating basis every fourth day. On 4 of these teams, the attending was a hospitalist, with clinical responsibility for the majority of the patients admitted to the team. On the remaining 4 teams, the teaching attending was a primary care physician or medical subspecialist, responsible for the direct care of a small number of the team's patients, with the remainder cared for by private primary care physicians or subspecialists.
Description of the Physician Assistant/Hospitalist Service
The Physician Assistant/Clinician Educator (PACE) service opened in July 2005, and consisted of 15 beds localized to 2 adjacent inpatient pods, staffed by a single cadre of nurses and medically staffed by 1 hospitalist and 2 physician assistants from 7:00 AM to 7:00 PM on weekdays and by 1 hospitalist, 1 physician assistant, and 1 moonlighter (usually a senior medical resident or fellow) from 7:00 AM to 7:00 PM on weekends. A moonlighter, typically a senior resident or medical subspecialty fellow, admitted patients and covered nights on the service from 7:00 PM to 7:00 AM 7 days a week. The daily census goal for the service was 15 patients, limited by the number of available beds on the 2 pods, and the service accepted admissions 24 hours per day, 7 days per week, whenever beds were available. Daily morning rounds occurred at 8:00 AM and included the hospitalist, physician assistants, nurses, a care coordinator, and a pharmacist. The PACE service did not have triage guidelines related to diagnosis, complexity, or acuity, but only accepted patients via the emergency department or via a primary care physician's office, and did not accept patients transferred from outside hospitals or from the intensive care units.
Physician Assistants
All of the physician assistants on the PACE service had prior inpatient medicine experience, ranging from 6 months to 5 years. The physician assistants worked in 3‐day to 6‐day blocks of 12‐hour shifts. Their clinical responsibilities were similar to those of interns at the study hospital, and included taking histories and performing physical examinations, writing notes and orders, reviewing and assimilating data, creating and updating patient signouts, completing discharge summaries, consulting other services as needed, and communicating with nurses and family members.
Many physician assistants also had nonclinical responsibilities, taking on physician‐mentored roles in education, quality improvement, and administration. They were involved in several initiatives: (1) developing a physician assistant curriculum in hospital medicine, (2) presenting at hospital‐wide physician assistant grand rounds, (3) surveying and tracking patient and family satisfaction on the service, (4) reviewing all 72‐hour hospital readmissions, intensive care unit transfers, and deaths on the service, and (5) maintaining the service's compliance with state regulations regarding physician assistant scope of practice and prescribing.
Hospitalists
The 3 hospitalists on the PACE service worked in 7‐day blocks of 12‐hour shifts (7:00 AM to 7:00 PM). They directly supervised the physician assistants and had no competing responsibilities. The hospitalists were all recent graduates of the study hospital's internal medicine residency, with no prior clinical experience beyond residency. All were planning to work on the service for 1 to 2 years before beginning a subspecialty fellowship. In addition to supervising the clinical work of the physician assistants, the hospitalists were responsible for teaching the physician assistants on rounds and in weekly didactic sessions, guided by a curriculum in hospital medicine that focused on the most common general medicine diagnoses seen on the PACE service. The medical director of the PACE service periodically reviewed each physician assistant's clinical experience, skills and knowledge base, and held semiannual feedback sessions.
Study Patients
All general medicine patients admitted to the PACE service from July 1, 2005 to June 30, 2006 comprised the study population. The comparison group consisted of general medicine patients admitted to the 8 house staff general medicine teams; patients transferred from an intensive care unit (ICU) or another facility were excluded in order to match the admission criteria for the PACE service and improve comparability between the 2 study arms.
Data Collection and Study Outcomes
We obtained all patient data from the hospital's administrative databases. We identified patients assigned to the PACE service or to the comparison group based on the admitting service, team, and attending. We obtained patient demographics, insurance, admission source and discharge destination, admission and discharge times, dates, diagnoses, and diagnosis‐related groups (DRGs), as well as dates and times of transfers to other services, including to the intensive care unit. We also obtained the Medicare case‐mix index (CMI, based on DRG weight), and calculated a Charlson score based on billing diagnoses coded in the year prior to the index admission.14 Outcomes included length of stay (LOS) to the nearest hour, in‐hospital mortality, transfers to the intensive care unit, readmissions to the study hospital within 72 hours, 14 days, and 30 days, and total costs as derived from the hospital's cost accounting system (Transition Systems Inc., Boston, MA). Other outcomes included patient satisfaction as measured by responses to the Press‐Ganey survey routinely administered to a randomly selected 70% of recently discharged patients and effect on self‐reported resident work hours.
Statistical Analysis
Patient demographics, clinical characteristics, and study outcomes are presented using proportions, means with standard deviations, and medians with inter‐quartile ranges as appropriate. Unadjusted differences in outcomes between the two services were calculated using univariable regression techniques with service as the independent variable and each outcome as the dependent variable. We used logistic regression for dichotomous outcomes (readmissions, ICU transfers, and inpatient mortality), and linear regression for log‐transformed LOS and log‐transformed total costs of care. To adjust each outcome for potential confounders, we then built multivariable regression models. Each potential confounder was entered into the model one at a time as the independent variable. All variables found to be significant predictors of the outcome at the P < 0.10 level were then retained in the final model along with service as the predictor of interest. We used general estimating equations in all multivariable models to adjust for clustering of patients by attending physician. For logistic regression models, the effect size is presented as an odds ratio (OR); for log‐transformed linear regression models, the effect size is presented as the percent difference between groups. We also performed 2 subgroup analyses, limited to (1) the patients with the 10 most common discharge DRGs, and (2) patients admitted between the hours of 7:00 AM and 7:00 PM to remove the effects of moonlighters performing the initial admission. Except as noted above, 2‐sided P values < 0.05 were considered significant. SAS 9.1 (SAS Institute, Cary, NC) was used for all analyses.
RESULTS
Patient Demographics
Table 1 shows patient demographics and clinical characteristics of the PACE service and the comparison group. Patients in the comparison group were slightly older and tended to have slightly higher CMI and Charlson scores. Patients on the PACE service were more likely to be admitted at night (10:00 PM to 7:00 AM; 43.8% versus 30.3%; P < 0.0001). There were no significant differences in sex, race, insurance, or percentage of patients discharged to home. The 10 most common DRGs in the comparison group accounted for 37.0% of discharges, and these same DRGs accounted for 37.5% of discharges on the PACE service (Table 2).
| Characteristic | PACE Service (n = 992) | House Staff Services (n = 4,202) | P value |
|---|---|---|---|
| |||
| Age (years) | |||
| 1844 | 19.1 | 18.2 | |
| 4564 | 35.5 | 31.9 | 0.04 |
| 65+ | 45.5 | 49.9 | |
| Sex (% female) | 57.7 | 60.0 | NS |
| Race/ethnicity | |||
| White | 57.3 | 59.3 | |
| Black | 24.0 | 23.5 | NS |
| Hispanic | 14.1 | 13.3 | |
| Other | 4.6 | 3.9 | |
| Insurance | |||
| Medicare | 41.9 | 43.8 | |
| Commercial | 34.9 | 35.9 | |
| Medicaid | 14.4 | 11.7 | NS |
| Free care | 4.5 | 3.9 | |
| Self pay | 1.1 | 0.8 | |
| Median income by zip code of residence, USD (IQR) | 45,517 (32,49362,932) | 45,517 (35,88963,275) | NS |
| Case‐mix index, median (IQR) | 1.1 (0.81.5) | 1.2 (0.91.8) | 0.001 |
| Charlson score | |||
| 0 | 27.2 | 24.9 | |
| 1 | 22.6 | 21.1 | 0.02 |
| 2 | 16.2 | 16.5 | |
| 3+ | 34.0 | 37.6 | |
| Admissions between 10:00 PM and 7:00 AM | 43.8 | 30.3 | <0.0001 |
| Discharged to home | 81.1 | 80.5 | NS |
| Diagnosis‐Related Group at Discharge | PACE Service (n = 992)* | House Staff Services (n = 4,202)* |
|---|---|---|
| ||
| Chest pain | 5.4 | 6.4 |
| Esophagitis, gastroenteritis, and miscellaneous digestive disorders | 4.5 | 4.4 |
| Heart failure and shock | 3.4 | 4.6 |
| Simple pneumonia and pleurisy | 2.7 | 4.4 |
| Kidney and urinary tract infections | 4.7 | 3.2 |
| Chronic obstructive pulmonary disease | 4.0 | 3.3 |
| Renal failure | 2.7 | 3.5 |
| Gastrointestinal hemorrhage | 3.7 | 2.7 |
| Nutritional and miscellaneous metabolic disorders | 3.3 | 2.4 |
| Disorders of the pancreas except malignancy | 3.1 | 2.1 |
| Cumulative percent | 37.5 | 37.0 |
Efficiency and Quality of Care
Table 3 compares the performance of the PACE service and the comparison group on several efficiency and quality measures. Unadjusted LOS was not significantly different, and adjusted LOS was slightly but not statistically significantly higher on the study service (adjusted LOS 5.0% higher; 95% confidence interval [CI], 0.4% to +10%). Unadjusted and adjusted total costs of care were marginally lower on the study service (adjusted total cost of care 3.9% lower; 95% CI, 7.5% to 0.3%).
| PACE Service | House Staff Services | Unadjusted % Difference (95%CI) | Adjusted % Difference (95%CI)* | |
|---|---|---|---|---|
| PACE Service | House Staff Services | Unadjusted OR (95% CI) | Adjusted OR (95% CI) | |
| ||||
| Efficiency measure | ||||
| Length of stay, days, median (IQR) | 2.6 (1.6, 4.4) | 2.6 (1.4, 4.6) | +0.1% (5.6% to +6.1%) | +5.0% (0.4% to +10.0%) |
| Total costs, USD, median (IQR) | 4,536 (2,848, 7,201) | 4,749 (3,046, 8,161) | 9.1% (14.0% to 3.8%) | 3.9% (7.5% to 0.3%)‖ |
| Quality measure | ||||
| 72‐hour readmissions/100 discharges | 0.8 | 1.3 | 0.6 (0.31.3) | 0.7 (0.21.8) |
| 14‐day readmissions/100 discharges | 5.4 | 5.4 | 1.0 (0.71.4) | 1.1 (0.81.4) |
| 30‐day readmissions/100 discharges | 8.0 | 8.1 | 1.0 (0.81.3) | 1.1 (0.91.3) |
| ICU transfers/100 discharges | 2.0 | 2.3 | 0.9 (0.51.4) | 1.4 (0.82.4)# |
| Inpatient mortality/100 discharges | 0.7 | 1.2 | 0.6 (0.31.3) | 0.8 (0.31.8)** |
We found no differences between the PACE service and comparison group in unadjusted rates of hospital readmissions within 72 hours, 14 days, and 30 days, transfer to the intensive care units, or inpatient mortality (Table 3). The associated ORs for each outcome were similar after adjusting for patient demographics and clinical characteristics including severity of illness, as well as for clustering by attending physician.
Subgroup Analyses
When the analysis was limited to the subset of patients with the 10 most common discharge DRGs, the difference in adjusted total cost of care was similar but lost statistical significance (4.0% lower on PACE service; 95% CI, 11.0% to +3.3%). In this subgroup, LOS, readmission rates, and ICU transfer rates were not different. ORs for mortality could not be calculated because there were no deaths in this subgroup on the PACE service (data not shown). When analysis was limited to daytime admissions (to remove any potential effect of admitting by a moonlighter), the difference in total cost of care was attenuated and lost statistical significance (0.2% lower on PACE service; 95%CI, 5.9% to +5.5%). No differences were seen in LOS, mortality, and ICU transfers (data not shown). However, 14‐day readmissions (but not 72‐hour or 30‐day readmissions) were lower on the PACE service (OR, 0.49; 95% CI, 0.25‐0.93).
Patient Satisfaction
Patients were similarly satisfied with their care on the PACE service and on the house staff services. In specific areas and globally, percentages of patients satisfied with their physicians and with the discharge process were not different, as measured by the Press‐Ganey survey (Press‐Ganey Associates, South Bend, IN; Figures 1 and 2). The survey distinguishes between attendings and residents, but not physician assistants; therefore, Figure 1 only includes responses to the attending questions. Given the sampling procedure of the Press‐Ganey survey, exact response rates cannot be calculated, but Press‐Ganey reports a response rate of about 40% for the English survey and about 20% for the Spanish survey.
Resident Duty Hours
Comparing the same month 1 year prior to implementation of the PACE service, mean self‐reported resident duty hours on the general medicine service were unchanged; however, self‐reported data were incomplete, and multiple changes took place in the residency program during the study period. For example, implementation of the PACE service allowed for the dissolution of one full house staff general medicine team and redistribution of these house staff to night float positions and an expanded medical intensive care unit.
Costs of Implementation
The costs associated with implementing the PACE service included physician and physician assistant salaries (2.5 full‐time physicians, 5 full‐time physician assistants, plus fringe) and night coverage by resident and fellow moonlighters (without fringe, and estimated at 50% effort given other moonlighter coverage responsibilities on subspecialty services). We estimated these costs at $257.50/patient‐day ($115/patient‐day for attending physician compensation, $110/patient‐day for physician assistant compensation, and $32.50/patient‐day for moonlighting coverage).
DISCUSSION
As academic centers struggle with developing a workforce to provide patient care no longer provided by residents, questions about the ideal structure of nonhouse staff inpatient services abound. Although solutions to this problem will be determined to some extent by local factors such as institutional culture and resources, some lessons learned in developing such services will be more widely applicable. We found that by implementing a geographically localized, physician assistant‐staffed hospitalist service, we were able to provide care of similar quality and efficiency to that of traditional house staff services, despite inexperienced hospitalists staffing the service and a medical residency program commonly recognized as one of the best in the country. Adjusted total costs were slightly lower on the PACE service, but this difference was small and of borderline statistical significance. Likewise, no significant differences were seen in any of several quality measures or in patient satisfaction.
Our findings add to the available evidence supporting the use of physician assistants on academic general medicine services, and are germane to academic centers facing reductions in house staff availability and seeking alternative models of care for inpatients. Several specific characteristics of the PACE service and the implications of these should be considered:
The service accepted all patients, regardless of diagnosis, acuity, or complexity of illness. This was unlike many previously described nonhouse staff services which were more limited in scope, and allowed more flexibility with patient flow. However, in the end, patients on the PACE service did have a modestly lower case mix index and Charlson score, suggesting that, despite a lack of triage guidelines, there was some bias in the triage of admissions, possibly due to a perception that physician assistants should take care of lower complexity patients. If it is desirable to have a similar distribution of higher complexity patients across house staff and nonhouse staff services, extra efforts may be necessary to overcome this perception.
The service was geographically regionalized. Geographic regionalization offered many important advantages, especially with regards to communication among staff, nursing, and consultants, and allowed for multidisciplinary rounds. However, it is possible that the modest, but not statistically significant, trend toward an increased LOS seen on the PACE service might be a reflection of geographic admitting (less incentive to discharge since discharging a patient means taking a new admission).
The education and professional development of the physician assistants was a priority. Physician assistants had considerable autonomy and responsibility, and rather than being assigned only lower level administrative tasks, performed all aspects of patient care. They also received regular teaching from the hospitalists, attended house staff teaching conferences, and developed nonclinical roles in education and quality improvement. The higher standards expected of the physician assistants were quite possibly a factor in the quality of care delivered, and almost certainly contributed to physician assistant satisfaction and retention.
Our findings contrast with those of Myers et al.,9 who found that a nonteaching service staffed by hospitalists and nurse practitioners had a significantly lower median LOS and hospital charges compared to similar patients on resident‐based services. However, unlike ours, their service cared for a select patient population, and only accepted patients with chest pain at low risk for acute coronary syndrome. Van Rhee et al.10 found that physician assistants on a general medicine service used fewer resources for patients with pneumonia, stroke, and congestive heart failure than resident physicians, and did not exceed the resources used by residents in other diagnoses. The authors did not find a difference in LOS, but did find a significantly higher mortality among patients with pneumonia cared for by physician assistants.
Several limitations should be noted. First, the study was a retrospective analysis of administrative data rather than a randomized trial, and although we employed a standard approach to adjust for a wide range of patient characteristics including severity of illness, there may have been undetected differences in the patient populations studied that may have confounded our results. Second, resident moonlighters admitted patients to the PACE service and, at other times, to the house staff services, and this may have diluted any differences between the groups. However, when we limited our analysis to the subgroup of patients admitted during the day, similar results were obtained, with the exception that the PACE service had a lower rate of 14‐day readmissions, an unexpected finding deserving of further study. Third, the study was conducted in a single academic institution and our findings may not be generalizable to others with different needs and resources; indeed, the costs associated with implementing such a service may be prohibitive for some institutions. Fourth, because of simultaneous changes that were taking place in our residency program, we are unable to accurately assess the impact of the PACE service on resident duty hours. However, resident duty hours did not increase over this time period on the general medicine service, and implementation of the service allowed for redistribution of house staff to other services and positions. Fifth, patient satisfaction data were obtained from responses to the mailed Press‐Ganey survey, to which there is a relatively low response rate. Also, we did not survey providers regarding their satisfaction with the service during the study period. Sixth, the study had limited power to detect clinically important differences in mortality and ICU transfers. Finally, this study is unable to compare this particular model of incorporating midlevel providers into general medical services with other models, only with traditional house staff services.
Future research should focus on determining the most effective and efficient ways to incorporate midlevel providers on academic general medicine services. One important question from the standpoint of house staff training is whether such services should be separate but equal, or should house staff gain experience during residency working with midlevel providers, since they are likely to encounter them in the future whether they stay in academics or not. Different models of care will likely have large implications for the quality and efficiency of patient care, house staff education and satisfaction, and physician assistant job satisfaction and turnover.
In summary, our study demonstrates that a geographically regionalized, multidisciplinary service staffed by hospitalists and physician assistants can be a safe alternative to house staff‐based services for the care of general medicine inpatients in an academic medical center.
- ,,,,,.The physician's assistant as resident on surgical service. An example of creative problem solving in surgical manpower.Arch Surg.1980;115:310–314.
- ,,,.Coronary arteriography performed by a physician assistant.Am J Cardiol.1987;60:784–787.
- .The specialized physician assistant: an alternative to the clinical cardiology trainee.Am J Cardiol.1987;60:901–902.
- ,,.One hospital's successful 20‐year experience with physician assistants in graduate medical education.Acad Med.1999;74:641–645.
- ,.Physicians assistants in cardiothoracic surgery: a 30‐year experience in a university center.Ann Thorac Surg.2006;81:195–199; discussion 199–200.
- ,,.Comparative review of use of physician assistants in a level I trauma center.Am Surg.2004;70:272–279.
- ,,,,.Integrating midlevel practitioners into a teaching service.Am J Surg.2006;192:119–124.
- ,.Acute care nurse practitioners: creating and implementing a model of care for an inpatient general medical service.Am J Crit Care.2002;11:448–458.
- ,,,,.Improving resource utilization in a teaching hospital: development of a nonteaching service for chest pain admissions.Acad Med.2006;81:432–435.
- ,,.Resource use by physician assistant services versus teaching services.JAAPA.2002;15:33–38.
- ,,, for the ACGME Work Group on Resident Duty Hours, Accreditation Council for Graduate Medical Education.New requirements for resident duty hours.JAMA.2002;288:1112–1114.
- ,,.The substitution of physician assistants and nurse practitioners for physician residents in teaching hospitals.Health Aff.1995;14:181–191.
- ,,,,.Challenges of the 80‐hour resident work rules: collaboration between surgeons and nonphysician practitioners.Surg Clin North Am.2004;84:1573–1586.
- ,,.Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases.J Clin Epidemiol.1992;45:613–619.
- ,,,,,.The physician's assistant as resident on surgical service. An example of creative problem solving in surgical manpower.Arch Surg.1980;115:310–314.
- ,,,.Coronary arteriography performed by a physician assistant.Am J Cardiol.1987;60:784–787.
- .The specialized physician assistant: an alternative to the clinical cardiology trainee.Am J Cardiol.1987;60:901–902.
- ,,.One hospital's successful 20‐year experience with physician assistants in graduate medical education.Acad Med.1999;74:641–645.
- ,.Physicians assistants in cardiothoracic surgery: a 30‐year experience in a university center.Ann Thorac Surg.2006;81:195–199; discussion 199–200.
- ,,.Comparative review of use of physician assistants in a level I trauma center.Am Surg.2004;70:272–279.
- ,,,,.Integrating midlevel practitioners into a teaching service.Am J Surg.2006;192:119–124.
- ,.Acute care nurse practitioners: creating and implementing a model of care for an inpatient general medical service.Am J Crit Care.2002;11:448–458.
- ,,,,.Improving resource utilization in a teaching hospital: development of a nonteaching service for chest pain admissions.Acad Med.2006;81:432–435.
- ,,.Resource use by physician assistant services versus teaching services.JAAPA.2002;15:33–38.
- ,,, for the ACGME Work Group on Resident Duty Hours, Accreditation Council for Graduate Medical Education.New requirements for resident duty hours.JAMA.2002;288:1112–1114.
- ,,.The substitution of physician assistants and nurse practitioners for physician residents in teaching hospitals.Health Aff.1995;14:181–191.
- ,,,,.Challenges of the 80‐hour resident work rules: collaboration between surgeons and nonphysician practitioners.Surg Clin North Am.2004;84:1573–1586.
- ,,.Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases.J Clin Epidemiol.1992;45:613–619.
Copyright © 2008 Society of Hospital Medicine
