David O. Meltzer, MD, PhD

Recently released Accreditation Council for Graduate Medical Education (ACGME) recommendations, set to take effect in 2011, place further limits on resident duty hours, which are expected to result in additional handoffs of patients between inpatient providers.1 The increase in these handoffs following the prior set of ACGME recommendations in 2003 has been cited as a potential etiology for the underwhelming effects of the duty hour restrictions on patient outcomes, whereby the benefits of well‐rested physicians are theorized to be offset by increased harm associated with discontinuous care, especially in high‐risk patients.2 In 2007, an Institute of Medicine committee on the topic recommended improving handovers to make the transfer of patient responsibility and information more effective and less error prone.3

Several strategies have been proposed, but an ideal way to quickly transfer complex medical information on numerous patients remains to be identified. A standardized metric of a patient's risk level, if accurate, has the potential to summarize how stable or unstable a patient might be. We hypothesized that clinicians would be able to quantify their judgments regarding the stability of their inpatients and that this measure would correlate with impending clinical deterioration as determined by cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours.

METHODS

Study Design

We developed the Patient Acuity Rating (PAR), a 7‐point Likert scale to quantify clinician judgment regarding the stability of inpatients outside the ICU, and conducted a prospective study of its diagnostic accuracy for predicting impending clinical deterioration in an academic tertiary care hospital. Providers were prospectively surveyed once per call‐cycle, on the day after patient admission, and asked to rate each of their patients on their likelihood of suffering a cardiac arrest or being transferred to the ICU. The scale was anchored at both ends, with a PAR of 1 corresponding to extreme unlikelihood of suffering a cardiac arrest or requiring emergent ICU transfer within the next 24 hours, and a PAR of 7 corresponding with extreme likelihood (Figure 1). A score of 4 suggested neither likely nor unlikely to experience an event. No further anchors were provided.

Figure 1

RESULTS

During the study period, 140/159 (88.1%) eligible clinicians consented to participate. Of these clinicians, 45 (32.1%) were intern physicians, 40 (28.6%) were resident physicians, 51 (36.4%) were attending physicians, and 4 (2.9%) were midlevel providers. They provided PAR scores on 1663 distinct patients over the course of 2019 separate admissions. Table 1 shows the patient and admission demographics grouped by the type of medical service: general medicine teaching or multispecialty non‐teaching. Severity of illness assignments were determined using All Patient Refined Diagnosis Related Group (APR‐DRG) methodology, which incorporates features such as principle diagnosis at admission, co‐morbidities, complications during admission, age, and gender.4, 5 The multispecialty patients were more likely to be male, have a higher severity of illness, and die during the hospitalization, when compared to general medicine patients.

A total of 6034 individual PAR scores from 3419 patient‐days were obtained, which represented a response rate of 74.3%. The average PAR was 2.9 1.4. Table 2 shows the inter‐rater reliability between providers. Weighted kappa statistics ranged from 0.32 (for interns and attendings) to 0.43 (for midlevels and attendings), representing moderate inter‐rater reliability. No comparison was made between midlevel providers and interns or residents, as these participants never cared for the same patients on the same day.

Seventy‐four of the 3419 patient‐days (2.2%) ended in cardiac arrest or unplanned ICU transfer. The distribution of clinical deterioration by average PAR, along with sensitivity and specificity values, are shown in Table 3. Using a cutoff value of 5 yielded a sensitivity of 62.2% and a specificity of 84.6%. Lowering the threshold to 4 increased the sensitivity to 82.4% but decreased the specificity to 68.3%. This corresponded with a combined AUC of 0.82 [95% CI 0.77, 0.87] (Table 4). Provider‐specific AUC values ranged from a low of 0.69 [95% CI 0.59, 0.78] for residents to a high of 0.84 [95% CI 0.78, 0.90] for attendings on general medicine (P = 0.01). The remaining values were not statistically different from one another. Figure 2 shows the provider‐specific percentage of patients deteriorating by PAR. The risk of clinical deterioration decreased in logarithmic fashion as the PAR decreased for all provider types (P < 0.001). These results were confirmed using multivariate logistic regression adjusting for provider, service, and individual patient (data not shown). In addition, we found no significant differences in AUC values between attendings in terms of years in practice or specialty, however, the study was not powered to detect such differences.

Figure 2

DISCUSSION

Physicians frequently depend on subjective judgments in their decision making.6 However, these judgments are difficult to communicate succinctly and hard to compare among clinicians. We have developed a simple tool for quantifying provider judgment, which yields moderate inter‐rater reliability, and good accuracy in predicting which floor patients may suffer cardiac arrest or emergent ICU transfer in the next 24 hours at an academic medical center.

Physicians routinely use written sign‐outs to convey important information to covering physicians during the handoff process, with the result being loss of information and decreased communication.7, 8 A recent study found that sign‐outs are frequently lacking comprehensive data, with the least commonly conveyed information being the patient's current clinical condition.9 The PAR has the potential to improve clinician handoffs by succinctly summarizing a patient's risk level. This need is made even more pressing by the ACGME's new resident duty hour restrictions and impending further increase in handoffs, a known correlate with inpatient morbidity and mortality.10 The PAR could be added to the sign‐out and updated nightly to readily summarize the judgments of the primary inpatient providers for the covering physician who has little, if any, personal knowledge of the patient at hand.

While ours is the first to examine the correlation between physician judgment and clinical deterioration on the floors, several studies have evaluated the accuracy of clinical judgment in predicting mortality of critically ill hospitalized patients. In the study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT), for example, physicians accurately predicted 180‐day mortality of ICU patients with an AUC of 0.78.11 Similar studies in the neonatal intensive care setting revealed greater than 80% predictive ability to identify those patients who would be intubated or fail to survive.12 These intuitive judgments have faired well when compared to previously validated ICU‐based physiologic scoring systems, such as the Acute Physiology, Age, Chronic Health Evaluation (APACHE) system. A meta‐analysis of studies which compared physician intuition to various physiologic scoring systems found intuition to be significantly better at predicting ICU mortality with an AUC of 0.85 compared to 0.63, P = 0.002.13

Physiology‐based scoring systems, relying on routine vital signs, have been developed for non‐ICU inpatients. Smith and colleagues14 recently conducted a systematic review and identified 33 distinct scoring systems, which they independently validated on a single data set for the ability of the admission score to predict overall hospital mortality. The resulting AUC values ranged from 0.66 to 0.78.14 In a prospective study, Kho and colleagues used an electronic medical record (EMR) to generate real‐time risk scores for use in surveillance rounds by their Rapid Response Team (RRT).15 Their scoring system relied on systolic blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, age, and body mass index. The resulting score yielded an AUC of 0.72, with a sensitivity of 88% and specificity of 48% using a cutoff of 3, or 34% and 86% using a cutoff of 5, for predicting code call, cardiopulmonary arrest, or transfer to an ICU. Similar to the latter study, using the PAR for RRT surveillance would allow an institution to set its threshold according to available resources. For example, the team could first evaluate all the patients with a PAR score of 7, followed by those who received a score of 6 and so on. Using the data from the current study, evaluating all the patients with scores of 5, 6, or 7 would require assessing 16% of the patients in order to identify 62% of the events. Adding patients with a score of 4 would require assessing one‐third of the floor patients, but would identify 82% of subsequent deteriorations.

Although the objective nature of physiology‐based scoring systems makes them very appealing, they have two significant limitations. The first is that these scoring systems either require manual vital sign data entry and score calculation, which can be labor intensive and impractical, or technological solutions such as an EMR, which are costly and therefore cannot be applied broadly to all hospitalized patients. In fact, in a recent survey of U.S. hospitals, only 1.5% were found to have a comprehensive EMR on all units.16 Additionally, they are limited by the quality of the data input. This is particularly true for the case of respiratory rate and mental status, which are frequently unreliably measured and documented in current practice.17, 18 The PAR score has the benefit of being readily generated in minimal time by a broad range of providers, as we have demonstrated.

Furthermore, it is well known that vital signs do not capture the full clinical picture, which is why most RRT activation criteria include a vague catch‐all trigger for provider worry or concern.19, 20 Interestingly, this trigger is frequently one of the top cited reasons for activating the RRT,21, 22 and is missed by any automated track‐and‐trigger scoring system which relies only on quantitative clinical assessments such as vital signs. The PAR allows this concern to be quantified, either for addition to a physiology‐based track‐and‐trigger system, or for use on its own, as we have done here.

It is interesting to note that, in this study, attending physician judgment was most predictive and resident judgment the least. One explanation may be that clinical judgment optimally requires both experience and at‐the‐bedside data. While attendings have the most experience, the amount of time interns spend at the bedside collecting data may offset their relative inexperience.23, 24 In contrast, residents generally spend less time at the bedside than interns and have only marginally more experience,25 suggesting that either strong clinical experience or a good amount of time at the bedside are required for the best assessments of risk. This is supported by the close agreement between the attendings and midlevels, who likely spend a comparable amount of time at the bedside as interns.

There are several imitations to this study. First, there may be respondent bias in those who chose to participate and the days in which they provided scores. We would expect this bias to work against the null hypothesis if providers with better clinical judgment were more inclined to participate, and were less likely to provide scores when they were very busy, and thus may have had less time to assess patients. However, the enrollment and response rates were quite good (88% and 74%, respectively) which likely mitigates against this bias. Another limitation is that the study was conducted at a single institution, and only on medical patients, which may limit its generalizability to other institutions and patient populations. Also, intern performance during this January through June period may not reflect their performance earlier in their training year. In addition, we did not have physiologic data available for the patients, and thus were not able to compare the PAR directly to a physiology‐based scoring system. Finally, it is theoretically possible that a provider could decide on the PAR and then transfer the patient to the ICU based on their score. However, we carefully surveyed physicians and nurse practitioners at the time of sign‐out, when they had finished their clinical work for the day, to minimize this risk. We also instructed providers not to share their PAR score with the covering physicians to avoid introduction of bias on the part of the cross‐covering physician.

This was a pilot study designed to measure the correlation between PAR scores and patient outcomes. The PAR has the potential to be added to any handoff system as a way to convey individual severity of illness for patients. In addition, it has the potential for use in risk stratifying patients for interventions, such as increased vital sign monitoring or heightened surveillance by cross‐covering physicians or Rapid Response Teams. One could imagine instructing interns to have a low threshold of concern for patients with high PAR scores, and even formalizing procedures for rounding on such patients a second time during the day or overnight, when on call. Future studies will be required to test its use in clinical practice, which would ideally include a randomized‐controlled trial.

We conclude that clinical judgment regarding floor patient stability is quantifiable in a readily obtained, low‐technology score that has moderate inter‐rater reliability and a good ability to distinguish patients who will suffer a cardiac arrest or require ICU transfer within the next 24 hours. Due to its simple and easy to administer nature, the PAR has the potential to be a useful tool for efficiently conveying complex assessments from one member of the healthcare team to another, thereby improving handoffs and identifying patients at risk of clinical deterioration who might benefit from earlier intervention.

Recently released Accreditation Council for Graduate Medical Education (ACGME) recommendations, set to take effect in 2011, place further limits on resident duty hours, which are expected to result in additional handoffs of patients between inpatient providers.1 The increase in these handoffs following the prior set of ACGME recommendations in 2003 has been cited as a potential etiology for the underwhelming effects of the duty hour restrictions on patient outcomes, whereby the benefits of well‐rested physicians are theorized to be offset by increased harm associated with discontinuous care, especially in high‐risk patients.2 In 2007, an Institute of Medicine committee on the topic recommended improving handovers to make the transfer of patient responsibility and information more effective and less error prone.3

Several strategies have been proposed, but an ideal way to quickly transfer complex medical information on numerous patients remains to be identified. A standardized metric of a patient's risk level, if accurate, has the potential to summarize how stable or unstable a patient might be. We hypothesized that clinicians would be able to quantify their judgments regarding the stability of their inpatients and that this measure would correlate with impending clinical deterioration as determined by cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours.

METHODS

Study Design

We developed the Patient Acuity Rating (PAR), a 7‐point Likert scale to quantify clinician judgment regarding the stability of inpatients outside the ICU, and conducted a prospective study of its diagnostic accuracy for predicting impending clinical deterioration in an academic tertiary care hospital. Providers were prospectively surveyed once per call‐cycle, on the day after patient admission, and asked to rate each of their patients on their likelihood of suffering a cardiac arrest or being transferred to the ICU. The scale was anchored at both ends, with a PAR of 1 corresponding to extreme unlikelihood of suffering a cardiac arrest or requiring emergent ICU transfer within the next 24 hours, and a PAR of 7 corresponding with extreme likelihood (Figure 1). A score of 4 suggested neither likely nor unlikely to experience an event. No further anchors were provided.

Figure 1

RESULTS

During the study period, 140/159 (88.1%) eligible clinicians consented to participate. Of these clinicians, 45 (32.1%) were intern physicians, 40 (28.6%) were resident physicians, 51 (36.4%) were attending physicians, and 4 (2.9%) were midlevel providers. They provided PAR scores on 1663 distinct patients over the course of 2019 separate admissions. Table 1 shows the patient and admission demographics grouped by the type of medical service: general medicine teaching or multispecialty non‐teaching. Severity of illness assignments were determined using All Patient Refined Diagnosis Related Group (APR‐DRG) methodology, which incorporates features such as principle diagnosis at admission, co‐morbidities, complications during admission, age, and gender.4, 5 The multispecialty patients were more likely to be male, have a higher severity of illness, and die during the hospitalization, when compared to general medicine patients.

A total of 6034 individual PAR scores from 3419 patient‐days were obtained, which represented a response rate of 74.3%. The average PAR was 2.9 1.4. Table 2 shows the inter‐rater reliability between providers. Weighted kappa statistics ranged from 0.32 (for interns and attendings) to 0.43 (for midlevels and attendings), representing moderate inter‐rater reliability. No comparison was made between midlevel providers and interns or residents, as these participants never cared for the same patients on the same day.

Seventy‐four of the 3419 patient‐days (2.2%) ended in cardiac arrest or unplanned ICU transfer. The distribution of clinical deterioration by average PAR, along with sensitivity and specificity values, are shown in Table 3. Using a cutoff value of 5 yielded a sensitivity of 62.2% and a specificity of 84.6%. Lowering the threshold to 4 increased the sensitivity to 82.4% but decreased the specificity to 68.3%. This corresponded with a combined AUC of 0.82 [95% CI 0.77, 0.87] (Table 4). Provider‐specific AUC values ranged from a low of 0.69 [95% CI 0.59, 0.78] for residents to a high of 0.84 [95% CI 0.78, 0.90] for attendings on general medicine (P = 0.01). The remaining values were not statistically different from one another. Figure 2 shows the provider‐specific percentage of patients deteriorating by PAR. The risk of clinical deterioration decreased in logarithmic fashion as the PAR decreased for all provider types (P < 0.001). These results were confirmed using multivariate logistic regression adjusting for provider, service, and individual patient (data not shown). In addition, we found no significant differences in AUC values between attendings in terms of years in practice or specialty, however, the study was not powered to detect such differences.

Figure 2

DISCUSSION

Physicians frequently depend on subjective judgments in their decision making.6 However, these judgments are difficult to communicate succinctly and hard to compare among clinicians. We have developed a simple tool for quantifying provider judgment, which yields moderate inter‐rater reliability, and good accuracy in predicting which floor patients may suffer cardiac arrest or emergent ICU transfer in the next 24 hours at an academic medical center.

Physicians routinely use written sign‐outs to convey important information to covering physicians during the handoff process, with the result being loss of information and decreased communication.7, 8 A recent study found that sign‐outs are frequently lacking comprehensive data, with the least commonly conveyed information being the patient's current clinical condition.9 The PAR has the potential to improve clinician handoffs by succinctly summarizing a patient's risk level. This need is made even more pressing by the ACGME's new resident duty hour restrictions and impending further increase in handoffs, a known correlate with inpatient morbidity and mortality.10 The PAR could be added to the sign‐out and updated nightly to readily summarize the judgments of the primary inpatient providers for the covering physician who has little, if any, personal knowledge of the patient at hand.

While ours is the first to examine the correlation between physician judgment and clinical deterioration on the floors, several studies have evaluated the accuracy of clinical judgment in predicting mortality of critically ill hospitalized patients. In the study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT), for example, physicians accurately predicted 180‐day mortality of ICU patients with an AUC of 0.78.11 Similar studies in the neonatal intensive care setting revealed greater than 80% predictive ability to identify those patients who would be intubated or fail to survive.12 These intuitive judgments have faired well when compared to previously validated ICU‐based physiologic scoring systems, such as the Acute Physiology, Age, Chronic Health Evaluation (APACHE) system. A meta‐analysis of studies which compared physician intuition to various physiologic scoring systems found intuition to be significantly better at predicting ICU mortality with an AUC of 0.85 compared to 0.63, P = 0.002.13

Physiology‐based scoring systems, relying on routine vital signs, have been developed for non‐ICU inpatients. Smith and colleagues14 recently conducted a systematic review and identified 33 distinct scoring systems, which they independently validated on a single data set for the ability of the admission score to predict overall hospital mortality. The resulting AUC values ranged from 0.66 to 0.78.14 In a prospective study, Kho and colleagues used an electronic medical record (EMR) to generate real‐time risk scores for use in surveillance rounds by their Rapid Response Team (RRT).15 Their scoring system relied on systolic blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, age, and body mass index. The resulting score yielded an AUC of 0.72, with a sensitivity of 88% and specificity of 48% using a cutoff of 3, or 34% and 86% using a cutoff of 5, for predicting code call, cardiopulmonary arrest, or transfer to an ICU. Similar to the latter study, using the PAR for RRT surveillance would allow an institution to set its threshold according to available resources. For example, the team could first evaluate all the patients with a PAR score of 7, followed by those who received a score of 6 and so on. Using the data from the current study, evaluating all the patients with scores of 5, 6, or 7 would require assessing 16% of the patients in order to identify 62% of the events. Adding patients with a score of 4 would require assessing one‐third of the floor patients, but would identify 82% of subsequent deteriorations.

Although the objective nature of physiology‐based scoring systems makes them very appealing, they have two significant limitations. The first is that these scoring systems either require manual vital sign data entry and score calculation, which can be labor intensive and impractical, or technological solutions such as an EMR, which are costly and therefore cannot be applied broadly to all hospitalized patients. In fact, in a recent survey of U.S. hospitals, only 1.5% were found to have a comprehensive EMR on all units.16 Additionally, they are limited by the quality of the data input. This is particularly true for the case of respiratory rate and mental status, which are frequently unreliably measured and documented in current practice.17, 18 The PAR score has the benefit of being readily generated in minimal time by a broad range of providers, as we have demonstrated.

Furthermore, it is well known that vital signs do not capture the full clinical picture, which is why most RRT activation criteria include a vague catch‐all trigger for provider worry or concern.19, 20 Interestingly, this trigger is frequently one of the top cited reasons for activating the RRT,21, 22 and is missed by any automated track‐and‐trigger scoring system which relies only on quantitative clinical assessments such as vital signs. The PAR allows this concern to be quantified, either for addition to a physiology‐based track‐and‐trigger system, or for use on its own, as we have done here.

It is interesting to note that, in this study, attending physician judgment was most predictive and resident judgment the least. One explanation may be that clinical judgment optimally requires both experience and at‐the‐bedside data. While attendings have the most experience, the amount of time interns spend at the bedside collecting data may offset their relative inexperience.23, 24 In contrast, residents generally spend less time at the bedside than interns and have only marginally more experience,25 suggesting that either strong clinical experience or a good amount of time at the bedside are required for the best assessments of risk. This is supported by the close agreement between the attendings and midlevels, who likely spend a comparable amount of time at the bedside as interns.

There are several imitations to this study. First, there may be respondent bias in those who chose to participate and the days in which they provided scores. We would expect this bias to work against the null hypothesis if providers with better clinical judgment were more inclined to participate, and were less likely to provide scores when they were very busy, and thus may have had less time to assess patients. However, the enrollment and response rates were quite good (88% and 74%, respectively) which likely mitigates against this bias. Another limitation is that the study was conducted at a single institution, and only on medical patients, which may limit its generalizability to other institutions and patient populations. Also, intern performance during this January through June period may not reflect their performance earlier in their training year. In addition, we did not have physiologic data available for the patients, and thus were not able to compare the PAR directly to a physiology‐based scoring system. Finally, it is theoretically possible that a provider could decide on the PAR and then transfer the patient to the ICU based on their score. However, we carefully surveyed physicians and nurse practitioners at the time of sign‐out, when they had finished their clinical work for the day, to minimize this risk. We also instructed providers not to share their PAR score with the covering physicians to avoid introduction of bias on the part of the cross‐covering physician.

This was a pilot study designed to measure the correlation between PAR scores and patient outcomes. The PAR has the potential to be added to any handoff system as a way to convey individual severity of illness for patients. In addition, it has the potential for use in risk stratifying patients for interventions, such as increased vital sign monitoring or heightened surveillance by cross‐covering physicians or Rapid Response Teams. One could imagine instructing interns to have a low threshold of concern for patients with high PAR scores, and even formalizing procedures for rounding on such patients a second time during the day or overnight, when on call. Future studies will be required to test its use in clinical practice, which would ideally include a randomized‐controlled trial.

We conclude that clinical judgment regarding floor patient stability is quantifiable in a readily obtained, low‐technology score that has moderate inter‐rater reliability and a good ability to distinguish patients who will suffer a cardiac arrest or require ICU transfer within the next 24 hours. Due to its simple and easy to administer nature, the PAR has the potential to be a useful tool for efficiently conveying complex assessments from one member of the healthcare team to another, thereby improving handoffs and identifying patients at risk of clinical deterioration who might benefit from earlier intervention.

Recently released Accreditation Council for Graduate Medical Education (ACGME) recommendations, set to take effect in 2011, place further limits on resident duty hours, which are expected to result in additional handoffs of patients between inpatient providers.1 The increase in these handoffs following the prior set of ACGME recommendations in 2003 has been cited as a potential etiology for the underwhelming effects of the duty hour restrictions on patient outcomes, whereby the benefits of well‐rested physicians are theorized to be offset by increased harm associated with discontinuous care, especially in high‐risk patients.2 In 2007, an Institute of Medicine committee on the topic recommended improving handovers to make the transfer of patient responsibility and information more effective and less error prone.3

Several strategies have been proposed, but an ideal way to quickly transfer complex medical information on numerous patients remains to be identified. A standardized metric of a patient's risk level, if accurate, has the potential to summarize how stable or unstable a patient might be. We hypothesized that clinicians would be able to quantify their judgments regarding the stability of their inpatients and that this measure would correlate with impending clinical deterioration as determined by cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours.

METHODS

Study Design

We developed the Patient Acuity Rating (PAR), a 7‐point Likert scale to quantify clinician judgment regarding the stability of inpatients outside the ICU, and conducted a prospective study of its diagnostic accuracy for predicting impending clinical deterioration in an academic tertiary care hospital. Providers were prospectively surveyed once per call‐cycle, on the day after patient admission, and asked to rate each of their patients on their likelihood of suffering a cardiac arrest or being transferred to the ICU. The scale was anchored at both ends, with a PAR of 1 corresponding to extreme unlikelihood of suffering a cardiac arrest or requiring emergent ICU transfer within the next 24 hours, and a PAR of 7 corresponding with extreme likelihood (Figure 1). A score of 4 suggested neither likely nor unlikely to experience an event. No further anchors were provided.

Figure 1

RESULTS

During the study period, 140/159 (88.1%) eligible clinicians consented to participate. Of these clinicians, 45 (32.1%) were intern physicians, 40 (28.6%) were resident physicians, 51 (36.4%) were attending physicians, and 4 (2.9%) were midlevel providers. They provided PAR scores on 1663 distinct patients over the course of 2019 separate admissions. Table 1 shows the patient and admission demographics grouped by the type of medical service: general medicine teaching or multispecialty non‐teaching. Severity of illness assignments were determined using All Patient Refined Diagnosis Related Group (APR‐DRG) methodology, which incorporates features such as principle diagnosis at admission, co‐morbidities, complications during admission, age, and gender.4, 5 The multispecialty patients were more likely to be male, have a higher severity of illness, and die during the hospitalization, when compared to general medicine patients.

A total of 6034 individual PAR scores from 3419 patient‐days were obtained, which represented a response rate of 74.3%. The average PAR was 2.9 1.4. Table 2 shows the inter‐rater reliability between providers. Weighted kappa statistics ranged from 0.32 (for interns and attendings) to 0.43 (for midlevels and attendings), representing moderate inter‐rater reliability. No comparison was made between midlevel providers and interns or residents, as these participants never cared for the same patients on the same day.

Seventy‐four of the 3419 patient‐days (2.2%) ended in cardiac arrest or unplanned ICU transfer. The distribution of clinical deterioration by average PAR, along with sensitivity and specificity values, are shown in Table 3. Using a cutoff value of 5 yielded a sensitivity of 62.2% and a specificity of 84.6%. Lowering the threshold to 4 increased the sensitivity to 82.4% but decreased the specificity to 68.3%. This corresponded with a combined AUC of 0.82 [95% CI 0.77, 0.87] (Table 4). Provider‐specific AUC values ranged from a low of 0.69 [95% CI 0.59, 0.78] for residents to a high of 0.84 [95% CI 0.78, 0.90] for attendings on general medicine (P = 0.01). The remaining values were not statistically different from one another. Figure 2 shows the provider‐specific percentage of patients deteriorating by PAR. The risk of clinical deterioration decreased in logarithmic fashion as the PAR decreased for all provider types (P < 0.001). These results were confirmed using multivariate logistic regression adjusting for provider, service, and individual patient (data not shown). In addition, we found no significant differences in AUC values between attendings in terms of years in practice or specialty, however, the study was not powered to detect such differences.

Figure 2

DISCUSSION

Physicians frequently depend on subjective judgments in their decision making.6 However, these judgments are difficult to communicate succinctly and hard to compare among clinicians. We have developed a simple tool for quantifying provider judgment, which yields moderate inter‐rater reliability, and good accuracy in predicting which floor patients may suffer cardiac arrest or emergent ICU transfer in the next 24 hours at an academic medical center.

Physicians routinely use written sign‐outs to convey important information to covering physicians during the handoff process, with the result being loss of information and decreased communication.7, 8 A recent study found that sign‐outs are frequently lacking comprehensive data, with the least commonly conveyed information being the patient's current clinical condition.9 The PAR has the potential to improve clinician handoffs by succinctly summarizing a patient's risk level. This need is made even more pressing by the ACGME's new resident duty hour restrictions and impending further increase in handoffs, a known correlate with inpatient morbidity and mortality.10 The PAR could be added to the sign‐out and updated nightly to readily summarize the judgments of the primary inpatient providers for the covering physician who has little, if any, personal knowledge of the patient at hand.

While ours is the first to examine the correlation between physician judgment and clinical deterioration on the floors, several studies have evaluated the accuracy of clinical judgment in predicting mortality of critically ill hospitalized patients. In the study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT), for example, physicians accurately predicted 180‐day mortality of ICU patients with an AUC of 0.78.11 Similar studies in the neonatal intensive care setting revealed greater than 80% predictive ability to identify those patients who would be intubated or fail to survive.12 These intuitive judgments have faired well when compared to previously validated ICU‐based physiologic scoring systems, such as the Acute Physiology, Age, Chronic Health Evaluation (APACHE) system. A meta‐analysis of studies which compared physician intuition to various physiologic scoring systems found intuition to be significantly better at predicting ICU mortality with an AUC of 0.85 compared to 0.63, P = 0.002.13

Physiology‐based scoring systems, relying on routine vital signs, have been developed for non‐ICU inpatients. Smith and colleagues14 recently conducted a systematic review and identified 33 distinct scoring systems, which they independently validated on a single data set for the ability of the admission score to predict overall hospital mortality. The resulting AUC values ranged from 0.66 to 0.78.14 In a prospective study, Kho and colleagues used an electronic medical record (EMR) to generate real‐time risk scores for use in surveillance rounds by their Rapid Response Team (RRT).15 Their scoring system relied on systolic blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, age, and body mass index. The resulting score yielded an AUC of 0.72, with a sensitivity of 88% and specificity of 48% using a cutoff of 3, or 34% and 86% using a cutoff of 5, for predicting code call, cardiopulmonary arrest, or transfer to an ICU. Similar to the latter study, using the PAR for RRT surveillance would allow an institution to set its threshold according to available resources. For example, the team could first evaluate all the patients with a PAR score of 7, followed by those who received a score of 6 and so on. Using the data from the current study, evaluating all the patients with scores of 5, 6, or 7 would require assessing 16% of the patients in order to identify 62% of the events. Adding patients with a score of 4 would require assessing one‐third of the floor patients, but would identify 82% of subsequent deteriorations.

Although the objective nature of physiology‐based scoring systems makes them very appealing, they have two significant limitations. The first is that these scoring systems either require manual vital sign data entry and score calculation, which can be labor intensive and impractical, or technological solutions such as an EMR, which are costly and therefore cannot be applied broadly to all hospitalized patients. In fact, in a recent survey of U.S. hospitals, only 1.5% were found to have a comprehensive EMR on all units.16 Additionally, they are limited by the quality of the data input. This is particularly true for the case of respiratory rate and mental status, which are frequently unreliably measured and documented in current practice.17, 18 The PAR score has the benefit of being readily generated in minimal time by a broad range of providers, as we have demonstrated.

Furthermore, it is well known that vital signs do not capture the full clinical picture, which is why most RRT activation criteria include a vague catch‐all trigger for provider worry or concern.19, 20 Interestingly, this trigger is frequently one of the top cited reasons for activating the RRT,21, 22 and is missed by any automated track‐and‐trigger scoring system which relies only on quantitative clinical assessments such as vital signs. The PAR allows this concern to be quantified, either for addition to a physiology‐based track‐and‐trigger system, or for use on its own, as we have done here.

It is interesting to note that, in this study, attending physician judgment was most predictive and resident judgment the least. One explanation may be that clinical judgment optimally requires both experience and at‐the‐bedside data. While attendings have the most experience, the amount of time interns spend at the bedside collecting data may offset their relative inexperience.23, 24 In contrast, residents generally spend less time at the bedside than interns and have only marginally more experience,25 suggesting that either strong clinical experience or a good amount of time at the bedside are required for the best assessments of risk. This is supported by the close agreement between the attendings and midlevels, who likely spend a comparable amount of time at the bedside as interns.

There are several imitations to this study. First, there may be respondent bias in those who chose to participate and the days in which they provided scores. We would expect this bias to work against the null hypothesis if providers with better clinical judgment were more inclined to participate, and were less likely to provide scores when they were very busy, and thus may have had less time to assess patients. However, the enrollment and response rates were quite good (88% and 74%, respectively) which likely mitigates against this bias. Another limitation is that the study was conducted at a single institution, and only on medical patients, which may limit its generalizability to other institutions and patient populations. Also, intern performance during this January through June period may not reflect their performance earlier in their training year. In addition, we did not have physiologic data available for the patients, and thus were not able to compare the PAR directly to a physiology‐based scoring system. Finally, it is theoretically possible that a provider could decide on the PAR and then transfer the patient to the ICU based on their score. However, we carefully surveyed physicians and nurse practitioners at the time of sign‐out, when they had finished their clinical work for the day, to minimize this risk. We also instructed providers not to share their PAR score with the covering physicians to avoid introduction of bias on the part of the cross‐covering physician.

This was a pilot study designed to measure the correlation between PAR scores and patient outcomes. The PAR has the potential to be added to any handoff system as a way to convey individual severity of illness for patients. In addition, it has the potential for use in risk stratifying patients for interventions, such as increased vital sign monitoring or heightened surveillance by cross‐covering physicians or Rapid Response Teams. One could imagine instructing interns to have a low threshold of concern for patients with high PAR scores, and even formalizing procedures for rounding on such patients a second time during the day or overnight, when on call. Future studies will be required to test its use in clinical practice, which would ideally include a randomized‐controlled trial.

We conclude that clinical judgment regarding floor patient stability is quantifiable in a readily obtained, low‐technology score that has moderate inter‐rater reliability and a good ability to distinguish patients who will suffer a cardiac arrest or require ICU transfer within the next 24 hours. Due to its simple and easy to administer nature, the PAR has the potential to be a useful tool for efficiently conveying complex assessments from one member of the healthcare team to another, thereby improving handoffs and identifying patients at risk of clinical deterioration who might benefit from earlier intervention.

Technological advances drive medical providers to specialize through the need for proficiency around increasingly focused areas of expertise.1 But the benefits of specialization are attained only by balancing the advantages of increasing expertise and the costs of coordinating care that must be borne as specialization increases.2 Integrating experts into modern medical delivery systems requires attention to the coordinating mechanisms that govern team‐based care.3

Coordination, defined as the management of task interdependencies,4 is a central component and a useful measure of teamwork.5 Several studies demonstrate the patient‐level impact of coordination among providers.69 Gittell et al.8 demonstrated that orthopedic hospitals whose staff had better relational coordination (RC) measures had shorter lengths of stay and better post‐operative pain control for patients undergoing surgery. In medical intensive care units (ICUs), Wheelan et al.9 showed that staff members of units with lower mortality rates perceived their teams as functioning at higher stages of group development and perceived their team members as less dependent and more trusting.

Communication is the cornerstone of effective team coordination.10, 11 As such, practice model interventions that facilitate frequent communication of higher quality are associated with lower error rates10 and better teamwork.11 The use of hospitalists, for example, is shown to capitalize on this advantage by improving coordination through physician availability that facilitates communication and relational interactions among hospital‐based staff.12 While system‐level interventions such as this have received significant attention from experts in organizations, empirical studies that explore the contribution of team member characteristics to overall coordination are lacking.13

Inpatient comanagement services offer a unique model for studying teamwork. While the label is used to describe a variety of arrangements,1416 comanagement broadly describes a practice model wherein providers of various specialties deliver direct care to patients, in contrast to the traditional generalist‐consultant model in which specialists lend expertise.17 Many recent comanagement practices involve hospitalists in partnership with surgeons in the care of patients with concurrent medical and surgical needs,18 but similar arrangements between hospitalists and medical subspecialists are being adopted in some medical centers for the care of complex patients with conditions such as heart failure, cancer, stroke, and solid organ transplantations. Coordination among providers has not been studied in this context.

The goals of this study are: (1) to measure the input of individual providers to the overall coordination of care on a highly interdependent medical comanagement service, (2) to characterize high and low coordinators, and (3) to explore the relationship between coordination and patient outcomes. The main hypothesis is that the quality of team coordination is determined partly by the attributes of its members such that their individual contributions to the coordination of care affect the outcomes of vulnerable hospitalized patients.

Materials and Methods

Results

All 32 providers (100%) completed the Baseline Survey and participated in the Repeated Surveys of which 177/224 (79%) were completed. The median number of surveys that contributed to the calculation of individual RC and the mean RC by provider type are summarized in Table 1.

Of the 119 patients managed on the service, the mean age (standard deviation [SD]) was 55 (14) years and 48% were women. Of the 201 hospitalizations, there were 13 floor‐to‐ICU transfers and 5 in‐hospital deaths, however, we excluded from the analysis 1 death of a patient who was admitted under inpatient hospice status.

Discussion

By adapting Gittell's RC instrument to focus on individual providers, we found that their characteristic attributes such as preference for particular management styles, leadership quality, and patient ownership are associated with their externally perceived contributions to the overall coordination of care. In an unadjusted analysis, we also observed an intriguing trend towards more frequent major hospital complications when the worst coordinators of each physician type were on service.

Existing evidence22, 23 mostly summarized in a recent RAND Health report shows a weak association between clinical teamwork quality and patient mortality. While our data also support this association, it does so with limitations. Most importantly, the small sample size limited our ability to rigorously account for potential confounders that may have contributed to this apparent association. Further studies may better address whether or not bad outcomes are indeed associated with poor coordinators in highly interdependent clinical teams. In addition to confounding, the small sample size of providers makes the analysis vulnerable to type 1 errors. We addressed this issue by intensively surveying providers repeatedly to achieve a high resolution of the coordination and management structure measures from each comanaged team. The potential for omitted variables and reverse causality in that the coordination scores may be negatively influenced by particularly complex patients and bad outcomes remains a valid concern. We addressed this by confirming the stability of provider RC over time and excluding the RC data from rotations with a bad outcome, but the negative perception of an individual tied to past bad outcomes may persist beyond a particular rotation. Survey responses are subject to recall and hindsight biases, which we attempted to minimize by surveying respondents immediately after each team rotation. Finally, all of our findings may be not be generalizable to other comanagement settings. However, the important correlations between coordination and quality have been observed in other contexts.24, 25

In our study, in‐hospital deaths and ICU transfers are treated as consequences of uncoordinated care. This interpretation may be problematic for circumstances when death is inevitable no matter how well coordinated the care, or when transfer to a higher level of care is appropriate. The rationale for grouping the 2 events into 1 composite bad outcome is based on the assumption that both death and the escalation of care can be delayed to an extent, if not wholly prevented, with the coordinated utilization of a modern hospital's resources. The attribution of these events to poor coordinators may indicate the unraveling of coordination that normally must be maintained to help patients overcome decompensating events that are particularly common in the course of patients with severe liver diseases. Due to the exploratory nature of this analysis, additional studies are necessary to fully characterize the relationship between care coordination and care transfers.

An important implication of this study is that the communication skill and ethical disposition of each individual provider is relevant to the coordination that is sought in multi‐provider teams. Training medical professionals to be better team members may have direct impact on the patients they serve. Our finding about patient ownership suggests that commitment to patients in the framework of care is not merely tradition but a characteristic of competent physicians. Moreover, physicians' commitment to patients is a possible factor, not just in achieving patients' satisfaction, but in securing better outcomes. To that end, the teaching of this and other humanistic principles must remain a vital part of medical education at all levels of training.

Several implications about team leadership and hierarchy are apparent from the data. Findings around the perceived assignment of responsibilities show that high coordinator hepatologists acknowledge the advantages of overlapping task boundaries to prevent critical tasks from being missed and risking bad outcomes. High RC hepatologists in our study adopted a more participatory than supervisory role which presumably facilitated better coordination by transmitting organizational goals to other team members. The function of a comanaged team is likely to be enhanced by a fluid assignment of roles to better handle tasks with high uncertainty. Accordingly, comanagement models of care may not be appropriate in settings where tasks are not interdependent.26 Inherent hierarchy appears to be a feature of well coordinated teams. One possible interpretation of our data is that hospitalists who yield the leadership role to the hepatologist are perceived to be better coordinators and that those who insist on exerting more influence in team decisions are perceived to be poor coordinators.

Existing evidence around care coordination predicts that comanagement designs improve provider coordination through stage‐based and site‐based specialization.12 However, the mechanisms that mediate coordination and patient outcomes are not clear. Moreover, the mechanisms of coordinating multi‐disciplinary teams may be specific to each clinical setting. The role of individual provider characteristics on coordination deserves more attention. Similarly, the impact of organizational culture under which favorable provider characteristics thrive is unknown. Finally, a detailed exposition of patient ownership and the role patients play in affecting the coordination of healthcare resources needs further exploration.

Technological advances drive medical providers to specialize through the need for proficiency around increasingly focused areas of expertise.1 But the benefits of specialization are attained only by balancing the advantages of increasing expertise and the costs of coordinating care that must be borne as specialization increases.2 Integrating experts into modern medical delivery systems requires attention to the coordinating mechanisms that govern team‐based care.3

Coordination, defined as the management of task interdependencies,4 is a central component and a useful measure of teamwork.5 Several studies demonstrate the patient‐level impact of coordination among providers.69 Gittell et al.8 demonstrated that orthopedic hospitals whose staff had better relational coordination (RC) measures had shorter lengths of stay and better post‐operative pain control for patients undergoing surgery. In medical intensive care units (ICUs), Wheelan et al.9 showed that staff members of units with lower mortality rates perceived their teams as functioning at higher stages of group development and perceived their team members as less dependent and more trusting.

Communication is the cornerstone of effective team coordination.10, 11 As such, practice model interventions that facilitate frequent communication of higher quality are associated with lower error rates10 and better teamwork.11 The use of hospitalists, for example, is shown to capitalize on this advantage by improving coordination through physician availability that facilitates communication and relational interactions among hospital‐based staff.12 While system‐level interventions such as this have received significant attention from experts in organizations, empirical studies that explore the contribution of team member characteristics to overall coordination are lacking.13

Inpatient comanagement services offer a unique model for studying teamwork. While the label is used to describe a variety of arrangements,1416 comanagement broadly describes a practice model wherein providers of various specialties deliver direct care to patients, in contrast to the traditional generalist‐consultant model in which specialists lend expertise.17 Many recent comanagement practices involve hospitalists in partnership with surgeons in the care of patients with concurrent medical and surgical needs,18 but similar arrangements between hospitalists and medical subspecialists are being adopted in some medical centers for the care of complex patients with conditions such as heart failure, cancer, stroke, and solid organ transplantations. Coordination among providers has not been studied in this context.

The goals of this study are: (1) to measure the input of individual providers to the overall coordination of care on a highly interdependent medical comanagement service, (2) to characterize high and low coordinators, and (3) to explore the relationship between coordination and patient outcomes. The main hypothesis is that the quality of team coordination is determined partly by the attributes of its members such that their individual contributions to the coordination of care affect the outcomes of vulnerable hospitalized patients.

Materials and Methods

Results

All 32 providers (100%) completed the Baseline Survey and participated in the Repeated Surveys of which 177/224 (79%) were completed. The median number of surveys that contributed to the calculation of individual RC and the mean RC by provider type are summarized in Table 1.

Of the 119 patients managed on the service, the mean age (standard deviation [SD]) was 55 (14) years and 48% were women. Of the 201 hospitalizations, there were 13 floor‐to‐ICU transfers and 5 in‐hospital deaths, however, we excluded from the analysis 1 death of a patient who was admitted under inpatient hospice status.

Discussion

By adapting Gittell's RC instrument to focus on individual providers, we found that their characteristic attributes such as preference for particular management styles, leadership quality, and patient ownership are associated with their externally perceived contributions to the overall coordination of care. In an unadjusted analysis, we also observed an intriguing trend towards more frequent major hospital complications when the worst coordinators of each physician type were on service.

Existing evidence22, 23 mostly summarized in a recent RAND Health report shows a weak association between clinical teamwork quality and patient mortality. While our data also support this association, it does so with limitations. Most importantly, the small sample size limited our ability to rigorously account for potential confounders that may have contributed to this apparent association. Further studies may better address whether or not bad outcomes are indeed associated with poor coordinators in highly interdependent clinical teams. In addition to confounding, the small sample size of providers makes the analysis vulnerable to type 1 errors. We addressed this issue by intensively surveying providers repeatedly to achieve a high resolution of the coordination and management structure measures from each comanaged team. The potential for omitted variables and reverse causality in that the coordination scores may be negatively influenced by particularly complex patients and bad outcomes remains a valid concern. We addressed this by confirming the stability of provider RC over time and excluding the RC data from rotations with a bad outcome, but the negative perception of an individual tied to past bad outcomes may persist beyond a particular rotation. Survey responses are subject to recall and hindsight biases, which we attempted to minimize by surveying respondents immediately after each team rotation. Finally, all of our findings may be not be generalizable to other comanagement settings. However, the important correlations between coordination and quality have been observed in other contexts.24, 25

In our study, in‐hospital deaths and ICU transfers are treated as consequences of uncoordinated care. This interpretation may be problematic for circumstances when death is inevitable no matter how well coordinated the care, or when transfer to a higher level of care is appropriate. The rationale for grouping the 2 events into 1 composite bad outcome is based on the assumption that both death and the escalation of care can be delayed to an extent, if not wholly prevented, with the coordinated utilization of a modern hospital's resources. The attribution of these events to poor coordinators may indicate the unraveling of coordination that normally must be maintained to help patients overcome decompensating events that are particularly common in the course of patients with severe liver diseases. Due to the exploratory nature of this analysis, additional studies are necessary to fully characterize the relationship between care coordination and care transfers.

An important implication of this study is that the communication skill and ethical disposition of each individual provider is relevant to the coordination that is sought in multi‐provider teams. Training medical professionals to be better team members may have direct impact on the patients they serve. Our finding about patient ownership suggests that commitment to patients in the framework of care is not merely tradition but a characteristic of competent physicians. Moreover, physicians' commitment to patients is a possible factor, not just in achieving patients' satisfaction, but in securing better outcomes. To that end, the teaching of this and other humanistic principles must remain a vital part of medical education at all levels of training.

Several implications about team leadership and hierarchy are apparent from the data. Findings around the perceived assignment of responsibilities show that high coordinator hepatologists acknowledge the advantages of overlapping task boundaries to prevent critical tasks from being missed and risking bad outcomes. High RC hepatologists in our study adopted a more participatory than supervisory role which presumably facilitated better coordination by transmitting organizational goals to other team members. The function of a comanaged team is likely to be enhanced by a fluid assignment of roles to better handle tasks with high uncertainty. Accordingly, comanagement models of care may not be appropriate in settings where tasks are not interdependent.26 Inherent hierarchy appears to be a feature of well coordinated teams. One possible interpretation of our data is that hospitalists who yield the leadership role to the hepatologist are perceived to be better coordinators and that those who insist on exerting more influence in team decisions are perceived to be poor coordinators.

Existing evidence around care coordination predicts that comanagement designs improve provider coordination through stage‐based and site‐based specialization.12 However, the mechanisms that mediate coordination and patient outcomes are not clear. Moreover, the mechanisms of coordinating multi‐disciplinary teams may be specific to each clinical setting. The role of individual provider characteristics on coordination deserves more attention. Similarly, the impact of organizational culture under which favorable provider characteristics thrive is unknown. Finally, a detailed exposition of patient ownership and the role patients play in affecting the coordination of healthcare resources needs further exploration.

Technological advances drive medical providers to specialize through the need for proficiency around increasingly focused areas of expertise.1 But the benefits of specialization are attained only by balancing the advantages of increasing expertise and the costs of coordinating care that must be borne as specialization increases.2 Integrating experts into modern medical delivery systems requires attention to the coordinating mechanisms that govern team‐based care.3

Coordination, defined as the management of task interdependencies,4 is a central component and a useful measure of teamwork.5 Several studies demonstrate the patient‐level impact of coordination among providers.69 Gittell et al.8 demonstrated that orthopedic hospitals whose staff had better relational coordination (RC) measures had shorter lengths of stay and better post‐operative pain control for patients undergoing surgery. In medical intensive care units (ICUs), Wheelan et al.9 showed that staff members of units with lower mortality rates perceived their teams as functioning at higher stages of group development and perceived their team members as less dependent and more trusting.

Communication is the cornerstone of effective team coordination.10, 11 As such, practice model interventions that facilitate frequent communication of higher quality are associated with lower error rates10 and better teamwork.11 The use of hospitalists, for example, is shown to capitalize on this advantage by improving coordination through physician availability that facilitates communication and relational interactions among hospital‐based staff.12 While system‐level interventions such as this have received significant attention from experts in organizations, empirical studies that explore the contribution of team member characteristics to overall coordination are lacking.13

Inpatient comanagement services offer a unique model for studying teamwork. While the label is used to describe a variety of arrangements,1416 comanagement broadly describes a practice model wherein providers of various specialties deliver direct care to patients, in contrast to the traditional generalist‐consultant model in which specialists lend expertise.17 Many recent comanagement practices involve hospitalists in partnership with surgeons in the care of patients with concurrent medical and surgical needs,18 but similar arrangements between hospitalists and medical subspecialists are being adopted in some medical centers for the care of complex patients with conditions such as heart failure, cancer, stroke, and solid organ transplantations. Coordination among providers has not been studied in this context.

The goals of this study are: (1) to measure the input of individual providers to the overall coordination of care on a highly interdependent medical comanagement service, (2) to characterize high and low coordinators, and (3) to explore the relationship between coordination and patient outcomes. The main hypothesis is that the quality of team coordination is determined partly by the attributes of its members such that their individual contributions to the coordination of care affect the outcomes of vulnerable hospitalized patients.

Materials and Methods

Results

All 32 providers (100%) completed the Baseline Survey and participated in the Repeated Surveys of which 177/224 (79%) were completed. The median number of surveys that contributed to the calculation of individual RC and the mean RC by provider type are summarized in Table 1.

Of the 119 patients managed on the service, the mean age (standard deviation [SD]) was 55 (14) years and 48% were women. Of the 201 hospitalizations, there were 13 floor‐to‐ICU transfers and 5 in‐hospital deaths, however, we excluded from the analysis 1 death of a patient who was admitted under inpatient hospice status.

Discussion

By adapting Gittell's RC instrument to focus on individual providers, we found that their characteristic attributes such as preference for particular management styles, leadership quality, and patient ownership are associated with their externally perceived contributions to the overall coordination of care. In an unadjusted analysis, we also observed an intriguing trend towards more frequent major hospital complications when the worst coordinators of each physician type were on service.

Existing evidence22, 23 mostly summarized in a recent RAND Health report shows a weak association between clinical teamwork quality and patient mortality. While our data also support this association, it does so with limitations. Most importantly, the small sample size limited our ability to rigorously account for potential confounders that may have contributed to this apparent association. Further studies may better address whether or not bad outcomes are indeed associated with poor coordinators in highly interdependent clinical teams. In addition to confounding, the small sample size of providers makes the analysis vulnerable to type 1 errors. We addressed this issue by intensively surveying providers repeatedly to achieve a high resolution of the coordination and management structure measures from each comanaged team. The potential for omitted variables and reverse causality in that the coordination scores may be negatively influenced by particularly complex patients and bad outcomes remains a valid concern. We addressed this by confirming the stability of provider RC over time and excluding the RC data from rotations with a bad outcome, but the negative perception of an individual tied to past bad outcomes may persist beyond a particular rotation. Survey responses are subject to recall and hindsight biases, which we attempted to minimize by surveying respondents immediately after each team rotation. Finally, all of our findings may be not be generalizable to other comanagement settings. However, the important correlations between coordination and quality have been observed in other contexts.24, 25

In our study, in‐hospital deaths and ICU transfers are treated as consequences of uncoordinated care. This interpretation may be problematic for circumstances when death is inevitable no matter how well coordinated the care, or when transfer to a higher level of care is appropriate. The rationale for grouping the 2 events into 1 composite bad outcome is based on the assumption that both death and the escalation of care can be delayed to an extent, if not wholly prevented, with the coordinated utilization of a modern hospital's resources. The attribution of these events to poor coordinators may indicate the unraveling of coordination that normally must be maintained to help patients overcome decompensating events that are particularly common in the course of patients with severe liver diseases. Due to the exploratory nature of this analysis, additional studies are necessary to fully characterize the relationship between care coordination and care transfers.

An important implication of this study is that the communication skill and ethical disposition of each individual provider is relevant to the coordination that is sought in multi‐provider teams. Training medical professionals to be better team members may have direct impact on the patients they serve. Our finding about patient ownership suggests that commitment to patients in the framework of care is not merely tradition but a characteristic of competent physicians. Moreover, physicians' commitment to patients is a possible factor, not just in achieving patients' satisfaction, but in securing better outcomes. To that end, the teaching of this and other humanistic principles must remain a vital part of medical education at all levels of training.

Several implications about team leadership and hierarchy are apparent from the data. Findings around the perceived assignment of responsibilities show that high coordinator hepatologists acknowledge the advantages of overlapping task boundaries to prevent critical tasks from being missed and risking bad outcomes. High RC hepatologists in our study adopted a more participatory than supervisory role which presumably facilitated better coordination by transmitting organizational goals to other team members. The function of a comanaged team is likely to be enhanced by a fluid assignment of roles to better handle tasks with high uncertainty. Accordingly, comanagement models of care may not be appropriate in settings where tasks are not interdependent.26 Inherent hierarchy appears to be a feature of well coordinated teams. One possible interpretation of our data is that hospitalists who yield the leadership role to the hepatologist are perceived to be better coordinators and that those who insist on exerting more influence in team decisions are perceived to be poor coordinators.

Existing evidence around care coordination predicts that comanagement designs improve provider coordination through stage‐based and site‐based specialization.12 However, the mechanisms that mediate coordination and patient outcomes are not clear. Moreover, the mechanisms of coordinating multi‐disciplinary teams may be specific to each clinical setting. The role of individual provider characteristics on coordination deserves more attention. Similarly, the impact of organizational culture under which favorable provider characteristics thrive is unknown. Finally, a detailed exposition of patient ownership and the role patients play in affecting the coordination of healthcare resources needs further exploration.

Recently, there has been an increased focus on improving communication during care transitions for older patients as they leave the hospital. One reason for this focus is the increasing utilization of hospitalists, or hospital‐based physicians, caring for patients in the United States.1 As a result, many primary care physicians (PCPs) no longer care for their patients while in the hospital and may not be informed of their patients' hospitalization.2 Additionally, with an emphasis on shorter lengths of hospital stay, more extensive post‐discharge follow‐up is often warranted for patients, which often becomes the responsibility of a patient's PCP. Recently 6 societies (American College of Physicians, Society of General Internal Medicine, Society of Hospital Medicine, American Geriatrics Society, American College of Emergency Physicians, and Society of Academic Emergency Medicine) have recommended that a patient's PCP is notified during all steps in care transitions and that patient‐centered approaches are employed.3 Despite the increased need for improved inpatient‐ambulatory care transitions, the communication between hospitalists and PCPs has been characterized as being poor and ineffective.4 Prior studies have shown that PCPs are not aware of test results that require follow‐up, may not receive timely or high quality discharge materials, and have an overall poor perception of the quality of communication.46 Ensuring adequate communication is considered important due to the increased risk of adverse events that patients experience after discharge from the hospital.79 Furthermore, recent studies have shown that patients are often able to identify and report adverse events that would not be detected by medical record review alone.10, 11 Eliciting patient perspectives on their experiences after discharge and their expectations of communication between PCPs and hospital physicians can help clinical teams design more patient‐centered solutions for care transitions.

The aim of this study is to report older patients' experiences with problems after hospital discharge and their understanding and expectation of communication between hospital physicians and their PCP. We also explored the relationship between patient experiences and whether their PCPs were aware of their hospitalization.

Methods

Results

Of the 114 eligible patients recruited between February and July 2008, 64 patient interviews were completed (56%). The average patient age was 73 years. Most patients were female (69%), African American (70%), live at home (75%), and have a PCP located at the University of Chicago (70%). There were also several who were low income (23% below a median yearly income of $15,000), and did not attend any college (52%). These patients had an average length of stay of 5.3 days, nearly half (48%) having been hospitalized in the past year, and 6 patients (9%) required a proxy to complete the interview (Table 1). There were no significant differences between responders and nonresponders with respect to race, gender, education, income, admission in the past 12 months, residence, PCP location, mental status, length of stay, or discharge status. Responders were more likely to be older than nonresponders (73 years [95% confidence interval {CI} 6976 years] vs. 63 years for nonresponders [95% CI 5769 years]; [P < 0.01]).

Forty‐two percent (27) of patients reported experiencing a post‐discharge problem. These 27 patients reported 42 distinct problems, each of which fell into 1 of 5 broad categories (Table 2). The most common of these were patients having difficulty obtaining follow‐up tests or appointments. These patients either had delay in getting, or were unable to get, follow‐up appointments, or follow‐up tests and test results. There were also many patients who needed reevaluation and thus, were either readmitted to the hospital or had to return to the Emergency Department. Another major category was those who had problems getting medication or therapy. For example, one of (the patients) treatment medswas very hard to find and it delayed us giving her her meds. Others reported they were not properly prepared for discharge. Most of these patients did not receive proper discharge materials which then caused other issues. As one proxy reported, The services were supposed to be provided for (the patient) through her social worker, no one has been informed to her being discharged or her being sent home. We have not gotten any services. Lastly, a few patients reported having hospital complications, such as post‐procedural complications, or questions, such as diagnosis questions.0

Patients were often uncertain of whether and how communication between the inpatient physician and PCP (Table 3) took place. One patient said, I don't know what the procedure is as far as giving him the message. Does she fax it to him? I don't know She told me that she was going to call and inform him on everything that happened. I don't know anything from there. The second most commonly expressed perception was from patients who assumed good communication had taken place between his or her physicians. This assumption was grounded in a belief that good communication naturally occurred between physicians. For example 1 patient expressed: (doctors) let the other doctors in too. That's the way to take care of stuff. Lastly, many patients expressed the feeling that their physicians were obligated to communicate with each other. As 1 patient reported, I think that they should have let (my PCP) know that I was in the hospital.

Two new themes emerged from the inductive analysis (Table 4). Forty‐five percent of patients reported experiencing negative emotions. These negative emotions were most often expressed as frustration or confusion. For example, 1 patient expressed confusion by saying, When I usually have lab work done I have prescription signedmaybe they changed the way of doing it. Now the pharmacy called me. But I'm supposed to have a note or something. Patients who reported a post‐discharge problem were more likely to report negative emotions (67% vs. 26%, P < 0.01). Feelings of empowerment were reported by 31% of patients. Empowerment was expressed most often as the patient being proactive in communicating with the PCP. One patient reported, We informed (my PCP) and we filled in all of the information that we wanted him to know about. Empowerment was also expressed as being proactive in advocating for communication between the inpatient team and the PCP (Table 3). Some patients expressed feeling empowered through the support of a third party, such as a home nurse. In addition, patients who have a third party advocate are more likely to report being empowered. Empowerment was expressed by 26% of patients with no third party advocate compared with 71% of patients with a third party advocate (P = 0.02).

From our sample of patients who completed a 2‐week post‐discharge interview, we were able to obtain PCP surveys for 40 (63%) of these patients (Figure 1). Thirty percent (12) of PCPs reported being unaware of the hospitalization. In all but 4 cases, PCPs had communicated with the medical team during hospitalization. Examining the association between PCP knowledge and patient reported post‐discharge problems showed that patients whose PCPs were not aware of the hospitalization were 2 times more likely to report a post‐discharge problem. A post‐discharge problem was reported by 67% of patients whose PCP was not aware of the hospitalization, while a post‐discharge problem was reported by 32% of patients whose PCP was aware (P < 0.05). Six patients reported returning to the ED or being readmitted. Four patients (33%) of PCPs who were unaware of hospitalization reported returning for reevaluation whereas 7% (n = 2) of patients whose PCP was aware of hospitalization reported returning for evaluation (P = 0.055). Interestingly, patients whose PCPs were not aware of the hospitalization reported feeling more empowered (58%) than those patients whose PCP were aware of the hospitalization (21%, P = 0.03). Because of possible confounding (patient report of problems post‐discharge problems may be affected by PCP awareness of hospitalization), we examined whether patients whose PCPs were aware of their hospitalization differed from those that did not. Patients whose PCPs were aware of their hospitalization were often older (75 vs. 69 years old), white (80% white vs. 65% nonwhite) and female (75% female vs. 54% male). While this small sample size prohibits examining for statistical significance, the magnitude of these differences suggests the need for a larger study to examine patient predictors of PCP awareness of hospitalization.

Figure 1

Discussion

In this sample of frail, older hospitalized patients, nearly half reported at least 1 post‐discharge problem. Most patients have perceptions of what communication did or did not take place between their physicians. While most do not understand the communication process, many expect good communication to occur, and feel that physicians are obligated to communicate with each other. However, patients' perceptions of communication highlight that patient expectations are far from the actual practice in some cases. Nearly half of patients reported feeling negative emotions, such as confusion and frustration, and patients were more likely to experience negative emotions when they also reported a post‐discharge problem. One‐third of patients reported feeling empowered. Empowerment was associated with having a third party who helped advocate for them. Paradoxically, patients whose PCP were not aware of their hospitalization were more likely to feel empowered. Lastly, more patients reported a post‐discharge problem when their PCP was not aware of the hospitalization.

Because this is predominantly a qualitative observational study, it is important to consider the mechanism for these findings since we cannot assume causal relationships. The association of negative emotions, like confusion and frustration, with post‐discharge problems could be explained due to additional stress of the problem itself or that a distressed frame of mind is associated with reporting more problems that may have been overlooked otherwise. In addition, the association between patient empowerment and lack of PCP awareness could be due to the fact that patients are forced to assume a more proactive role in contacting their PCP if they feel that their PCP was not aware. It is equally possible that PCP communication is selectively initiated by hospital physicians when the patients are least empowered. For example, our comparison of demographics for patients whose PCP was aware versus those that were not do suggest that patient characteristics might play a role in whether a patient's PCP is contacted. The association between a third party advocate and patient empowerment is likely explained as the third party is able to keep the patient informed and empowered.

This study has implications for efforts to design a more patient‐centered care transition for hospitalized older patients. First, patients and their proxies should be advocates for good communication to avoid the risks of care transitions. Prior interventions such as use of coaches to boost patient empowerment have had positive results for hospitalized older patients. Moreover, hospitals should keep in mind that problems after discharge are common and are linked to negative emotions, which may lower patient satisfaction or increase liability risk. Similarly, these findings also highlight the importance of keeping PCPs aware of patient hospitalization. For example, PCPs that are aware of hospitalization are better prepared to properly follow‐up on medications, tests, and appointments. The PCP can also help to better prepare the patient for discharge and ease the transition for the patient.

There are several limitations to our study. First and foremost, our small sample size limits our ability to examine statistical significance. This study was part of a short planning grant to design interventions to improve communication with PCPs during hospitalization. Efforts are currently underway to design a communication solution and educational intervention to highlight the importance of contacting PCPs during hospitalization. Because these patients were hospitalized on the teaching service, the resident with the guidance of the teaching attending is responsible for communicating with the PCP. The teaching attending was either a generalist, hospitalist, or specialist who routinely had no a priori relationship with patients prior to the hospitalization. Only 53% of patients were reached by telephone which raises the concern for nonresponse bias. Our low response rate highlights the challenge of doing this type of work with recently discharge patients in low income, underserved areas. In comparing responders and nonresponders, the only difference between the 2 groups was that responders were more likely to be older. One possible reason for this difference may be that older people are more likely to be at home and easier to contact over the phone. Similarly, since data were collected through interviews and adverse events were discussed, these results are subject to recall bias. Efforts were made to reduce this by calling within 2 to 3 weeks after discharge. Lastly, these findings are limited by generalizability. All the patients included in this study were from the University of Chicago Medical Center, which serves largely underserved, African American patients. The experiences of these patients may be unique to this site. In addition, we only studied patients who had a PCP, excluding a population of patients that are at inherent risk due to lack of a coordinating physician to guide ongoing care.

In conclusion, this study suggests that many frail, older patients reported experiencing a post‐discharge problem and patients whose PCPs did not know about their admission were more likely to report a post‐discharge problem. Systematic interventions to improve communications with PCPs during patient care transitions in and out of the hospital are needed.

Recently, there has been an increased focus on improving communication during care transitions for older patients as they leave the hospital. One reason for this focus is the increasing utilization of hospitalists, or hospital‐based physicians, caring for patients in the United States.1 As a result, many primary care physicians (PCPs) no longer care for their patients while in the hospital and may not be informed of their patients' hospitalization.2 Additionally, with an emphasis on shorter lengths of hospital stay, more extensive post‐discharge follow‐up is often warranted for patients, which often becomes the responsibility of a patient's PCP. Recently 6 societies (American College of Physicians, Society of General Internal Medicine, Society of Hospital Medicine, American Geriatrics Society, American College of Emergency Physicians, and Society of Academic Emergency Medicine) have recommended that a patient's PCP is notified during all steps in care transitions and that patient‐centered approaches are employed.3 Despite the increased need for improved inpatient‐ambulatory care transitions, the communication between hospitalists and PCPs has been characterized as being poor and ineffective.4 Prior studies have shown that PCPs are not aware of test results that require follow‐up, may not receive timely or high quality discharge materials, and have an overall poor perception of the quality of communication.46 Ensuring adequate communication is considered important due to the increased risk of adverse events that patients experience after discharge from the hospital.79 Furthermore, recent studies have shown that patients are often able to identify and report adverse events that would not be detected by medical record review alone.10, 11 Eliciting patient perspectives on their experiences after discharge and their expectations of communication between PCPs and hospital physicians can help clinical teams design more patient‐centered solutions for care transitions.

The aim of this study is to report older patients' experiences with problems after hospital discharge and their understanding and expectation of communication between hospital physicians and their PCP. We also explored the relationship between patient experiences and whether their PCPs were aware of their hospitalization.

Methods

Results

Of the 114 eligible patients recruited between February and July 2008, 64 patient interviews were completed (56%). The average patient age was 73 years. Most patients were female (69%), African American (70%), live at home (75%), and have a PCP located at the University of Chicago (70%). There were also several who were low income (23% below a median yearly income of $15,000), and did not attend any college (52%). These patients had an average length of stay of 5.3 days, nearly half (48%) having been hospitalized in the past year, and 6 patients (9%) required a proxy to complete the interview (Table 1). There were no significant differences between responders and nonresponders with respect to race, gender, education, income, admission in the past 12 months, residence, PCP location, mental status, length of stay, or discharge status. Responders were more likely to be older than nonresponders (73 years [95% confidence interval {CI} 6976 years] vs. 63 years for nonresponders [95% CI 5769 years]; [P < 0.01]).

Forty‐two percent (27) of patients reported experiencing a post‐discharge problem. These 27 patients reported 42 distinct problems, each of which fell into 1 of 5 broad categories (Table 2). The most common of these were patients having difficulty obtaining follow‐up tests or appointments. These patients either had delay in getting, or were unable to get, follow‐up appointments, or follow‐up tests and test results. There were also many patients who needed reevaluation and thus, were either readmitted to the hospital or had to return to the Emergency Department. Another major category was those who had problems getting medication or therapy. For example, one of (the patients) treatment medswas very hard to find and it delayed us giving her her meds. Others reported they were not properly prepared for discharge. Most of these patients did not receive proper discharge materials which then caused other issues. As one proxy reported, The services were supposed to be provided for (the patient) through her social worker, no one has been informed to her being discharged or her being sent home. We have not gotten any services. Lastly, a few patients reported having hospital complications, such as post‐procedural complications, or questions, such as diagnosis questions.0

Patients were often uncertain of whether and how communication between the inpatient physician and PCP (Table 3) took place. One patient said, I don't know what the procedure is as far as giving him the message. Does she fax it to him? I don't know She told me that she was going to call and inform him on everything that happened. I don't know anything from there. The second most commonly expressed perception was from patients who assumed good communication had taken place between his or her physicians. This assumption was grounded in a belief that good communication naturally occurred between physicians. For example 1 patient expressed: (doctors) let the other doctors in too. That's the way to take care of stuff. Lastly, many patients expressed the feeling that their physicians were obligated to communicate with each other. As 1 patient reported, I think that they should have let (my PCP) know that I was in the hospital.

Two new themes emerged from the inductive analysis (Table 4). Forty‐five percent of patients reported experiencing negative emotions. These negative emotions were most often expressed as frustration or confusion. For example, 1 patient expressed confusion by saying, When I usually have lab work done I have prescription signedmaybe they changed the way of doing it. Now the pharmacy called me. But I'm supposed to have a note or something. Patients who reported a post‐discharge problem were more likely to report negative emotions (67% vs. 26%, P < 0.01). Feelings of empowerment were reported by 31% of patients. Empowerment was expressed most often as the patient being proactive in communicating with the PCP. One patient reported, We informed (my PCP) and we filled in all of the information that we wanted him to know about. Empowerment was also expressed as being proactive in advocating for communication between the inpatient team and the PCP (Table 3). Some patients expressed feeling empowered through the support of a third party, such as a home nurse. In addition, patients who have a third party advocate are more likely to report being empowered. Empowerment was expressed by 26% of patients with no third party advocate compared with 71% of patients with a third party advocate (P = 0.02).

From our sample of patients who completed a 2‐week post‐discharge interview, we were able to obtain PCP surveys for 40 (63%) of these patients (Figure 1). Thirty percent (12) of PCPs reported being unaware of the hospitalization. In all but 4 cases, PCPs had communicated with the medical team during hospitalization. Examining the association between PCP knowledge and patient reported post‐discharge problems showed that patients whose PCPs were not aware of the hospitalization were 2 times more likely to report a post‐discharge problem. A post‐discharge problem was reported by 67% of patients whose PCP was not aware of the hospitalization, while a post‐discharge problem was reported by 32% of patients whose PCP was aware (P < 0.05). Six patients reported returning to the ED or being readmitted. Four patients (33%) of PCPs who were unaware of hospitalization reported returning for reevaluation whereas 7% (n = 2) of patients whose PCP was aware of hospitalization reported returning for evaluation (P = 0.055). Interestingly, patients whose PCPs were not aware of the hospitalization reported feeling more empowered (58%) than those patients whose PCP were aware of the hospitalization (21%, P = 0.03). Because of possible confounding (patient report of problems post‐discharge problems may be affected by PCP awareness of hospitalization), we examined whether patients whose PCPs were aware of their hospitalization differed from those that did not. Patients whose PCPs were aware of their hospitalization were often older (75 vs. 69 years old), white (80% white vs. 65% nonwhite) and female (75% female vs. 54% male). While this small sample size prohibits examining for statistical significance, the magnitude of these differences suggests the need for a larger study to examine patient predictors of PCP awareness of hospitalization.

Figure 1

Discussion

In this sample of frail, older hospitalized patients, nearly half reported at least 1 post‐discharge problem. Most patients have perceptions of what communication did or did not take place between their physicians. While most do not understand the communication process, many expect good communication to occur, and feel that physicians are obligated to communicate with each other. However, patients' perceptions of communication highlight that patient expectations are far from the actual practice in some cases. Nearly half of patients reported feeling negative emotions, such as confusion and frustration, and patients were more likely to experience negative emotions when they also reported a post‐discharge problem. One‐third of patients reported feeling empowered. Empowerment was associated with having a third party who helped advocate for them. Paradoxically, patients whose PCP were not aware of their hospitalization were more likely to feel empowered. Lastly, more patients reported a post‐discharge problem when their PCP was not aware of the hospitalization.

Because this is predominantly a qualitative observational study, it is important to consider the mechanism for these findings since we cannot assume causal relationships. The association of negative emotions, like confusion and frustration, with post‐discharge problems could be explained due to additional stress of the problem itself or that a distressed frame of mind is associated with reporting more problems that may have been overlooked otherwise. In addition, the association between patient empowerment and lack of PCP awareness could be due to the fact that patients are forced to assume a more proactive role in contacting their PCP if they feel that their PCP was not aware. It is equally possible that PCP communication is selectively initiated by hospital physicians when the patients are least empowered. For example, our comparison of demographics for patients whose PCP was aware versus those that were not do suggest that patient characteristics might play a role in whether a patient's PCP is contacted. The association between a third party advocate and patient empowerment is likely explained as the third party is able to keep the patient informed and empowered.

This study has implications for efforts to design a more patient‐centered care transition for hospitalized older patients. First, patients and their proxies should be advocates for good communication to avoid the risks of care transitions. Prior interventions such as use of coaches to boost patient empowerment have had positive results for hospitalized older patients. Moreover, hospitals should keep in mind that problems after discharge are common and are linked to negative emotions, which may lower patient satisfaction or increase liability risk. Similarly, these findings also highlight the importance of keeping PCPs aware of patient hospitalization. For example, PCPs that are aware of hospitalization are better prepared to properly follow‐up on medications, tests, and appointments. The PCP can also help to better prepare the patient for discharge and ease the transition for the patient.

There are several limitations to our study. First and foremost, our small sample size limits our ability to examine statistical significance. This study was part of a short planning grant to design interventions to improve communication with PCPs during hospitalization. Efforts are currently underway to design a communication solution and educational intervention to highlight the importance of contacting PCPs during hospitalization. Because these patients were hospitalized on the teaching service, the resident with the guidance of the teaching attending is responsible for communicating with the PCP. The teaching attending was either a generalist, hospitalist, or specialist who routinely had no a priori relationship with patients prior to the hospitalization. Only 53% of patients were reached by telephone which raises the concern for nonresponse bias. Our low response rate highlights the challenge of doing this type of work with recently discharge patients in low income, underserved areas. In comparing responders and nonresponders, the only difference between the 2 groups was that responders were more likely to be older. One possible reason for this difference may be that older people are more likely to be at home and easier to contact over the phone. Similarly, since data were collected through interviews and adverse events were discussed, these results are subject to recall bias. Efforts were made to reduce this by calling within 2 to 3 weeks after discharge. Lastly, these findings are limited by generalizability. All the patients included in this study were from the University of Chicago Medical Center, which serves largely underserved, African American patients. The experiences of these patients may be unique to this site. In addition, we only studied patients who had a PCP, excluding a population of patients that are at inherent risk due to lack of a coordinating physician to guide ongoing care.

In conclusion, this study suggests that many frail, older patients reported experiencing a post‐discharge problem and patients whose PCPs did not know about their admission were more likely to report a post‐discharge problem. Systematic interventions to improve communications with PCPs during patient care transitions in and out of the hospital are needed.

Recently, there has been an increased focus on improving communication during care transitions for older patients as they leave the hospital. One reason for this focus is the increasing utilization of hospitalists, or hospital‐based physicians, caring for patients in the United States.1 As a result, many primary care physicians (PCPs) no longer care for their patients while in the hospital and may not be informed of their patients' hospitalization.2 Additionally, with an emphasis on shorter lengths of hospital stay, more extensive post‐discharge follow‐up is often warranted for patients, which often becomes the responsibility of a patient's PCP. Recently 6 societies (American College of Physicians, Society of General Internal Medicine, Society of Hospital Medicine, American Geriatrics Society, American College of Emergency Physicians, and Society of Academic Emergency Medicine) have recommended that a patient's PCP is notified during all steps in care transitions and that patient‐centered approaches are employed.3 Despite the increased need for improved inpatient‐ambulatory care transitions, the communication between hospitalists and PCPs has been characterized as being poor and ineffective.4 Prior studies have shown that PCPs are not aware of test results that require follow‐up, may not receive timely or high quality discharge materials, and have an overall poor perception of the quality of communication.46 Ensuring adequate communication is considered important due to the increased risk of adverse events that patients experience after discharge from the hospital.79 Furthermore, recent studies have shown that patients are often able to identify and report adverse events that would not be detected by medical record review alone.10, 11 Eliciting patient perspectives on their experiences after discharge and their expectations of communication between PCPs and hospital physicians can help clinical teams design more patient‐centered solutions for care transitions.

The aim of this study is to report older patients' experiences with problems after hospital discharge and their understanding and expectation of communication between hospital physicians and their PCP. We also explored the relationship between patient experiences and whether their PCPs were aware of their hospitalization.

Methods

Results

Of the 114 eligible patients recruited between February and July 2008, 64 patient interviews were completed (56%). The average patient age was 73 years. Most patients were female (69%), African American (70%), live at home (75%), and have a PCP located at the University of Chicago (70%). There were also several who were low income (23% below a median yearly income of $15,000), and did not attend any college (52%). These patients had an average length of stay of 5.3 days, nearly half (48%) having been hospitalized in the past year, and 6 patients (9%) required a proxy to complete the interview (Table 1). There were no significant differences between responders and nonresponders with respect to race, gender, education, income, admission in the past 12 months, residence, PCP location, mental status, length of stay, or discharge status. Responders were more likely to be older than nonresponders (73 years [95% confidence interval {CI} 6976 years] vs. 63 years for nonresponders [95% CI 5769 years]; [P < 0.01]).

Forty‐two percent (27) of patients reported experiencing a post‐discharge problem. These 27 patients reported 42 distinct problems, each of which fell into 1 of 5 broad categories (Table 2). The most common of these were patients having difficulty obtaining follow‐up tests or appointments. These patients either had delay in getting, or were unable to get, follow‐up appointments, or follow‐up tests and test results. There were also many patients who needed reevaluation and thus, were either readmitted to the hospital or had to return to the Emergency Department. Another major category was those who had problems getting medication or therapy. For example, one of (the patients) treatment medswas very hard to find and it delayed us giving her her meds. Others reported they were not properly prepared for discharge. Most of these patients did not receive proper discharge materials which then caused other issues. As one proxy reported, The services were supposed to be provided for (the patient) through her social worker, no one has been informed to her being discharged or her being sent home. We have not gotten any services. Lastly, a few patients reported having hospital complications, such as post‐procedural complications, or questions, such as diagnosis questions.0

Patients were often uncertain of whether and how communication between the inpatient physician and PCP (Table 3) took place. One patient said, I don't know what the procedure is as far as giving him the message. Does she fax it to him? I don't know She told me that she was going to call and inform him on everything that happened. I don't know anything from there. The second most commonly expressed perception was from patients who assumed good communication had taken place between his or her physicians. This assumption was grounded in a belief that good communication naturally occurred between physicians. For example 1 patient expressed: (doctors) let the other doctors in too. That's the way to take care of stuff. Lastly, many patients expressed the feeling that their physicians were obligated to communicate with each other. As 1 patient reported, I think that they should have let (my PCP) know that I was in the hospital.

Two new themes emerged from the inductive analysis (Table 4). Forty‐five percent of patients reported experiencing negative emotions. These negative emotions were most often expressed as frustration or confusion. For example, 1 patient expressed confusion by saying, When I usually have lab work done I have prescription signedmaybe they changed the way of doing it. Now the pharmacy called me. But I'm supposed to have a note or something. Patients who reported a post‐discharge problem were more likely to report negative emotions (67% vs. 26%, P < 0.01). Feelings of empowerment were reported by 31% of patients. Empowerment was expressed most often as the patient being proactive in communicating with the PCP. One patient reported, We informed (my PCP) and we filled in all of the information that we wanted him to know about. Empowerment was also expressed as being proactive in advocating for communication between the inpatient team and the PCP (Table 3). Some patients expressed feeling empowered through the support of a third party, such as a home nurse. In addition, patients who have a third party advocate are more likely to report being empowered. Empowerment was expressed by 26% of patients with no third party advocate compared with 71% of patients with a third party advocate (P = 0.02).

From our sample of patients who completed a 2‐week post‐discharge interview, we were able to obtain PCP surveys for 40 (63%) of these patients (Figure 1). Thirty percent (12) of PCPs reported being unaware of the hospitalization. In all but 4 cases, PCPs had communicated with the medical team during hospitalization. Examining the association between PCP knowledge and patient reported post‐discharge problems showed that patients whose PCPs were not aware of the hospitalization were 2 times more likely to report a post‐discharge problem. A post‐discharge problem was reported by 67% of patients whose PCP was not aware of the hospitalization, while a post‐discharge problem was reported by 32% of patients whose PCP was aware (P < 0.05). Six patients reported returning to the ED or being readmitted. Four patients (33%) of PCPs who were unaware of hospitalization reported returning for reevaluation whereas 7% (n = 2) of patients whose PCP was aware of hospitalization reported returning for evaluation (P = 0.055). Interestingly, patients whose PCPs were not aware of the hospitalization reported feeling more empowered (58%) than those patients whose PCP were aware of the hospitalization (21%, P = 0.03). Because of possible confounding (patient report of problems post‐discharge problems may be affected by PCP awareness of hospitalization), we examined whether patients whose PCPs were aware of their hospitalization differed from those that did not. Patients whose PCPs were aware of their hospitalization were often older (75 vs. 69 years old), white (80% white vs. 65% nonwhite) and female (75% female vs. 54% male). While this small sample size prohibits examining for statistical significance, the magnitude of these differences suggests the need for a larger study to examine patient predictors of PCP awareness of hospitalization.

Figure 1

Discussion

In this sample of frail, older hospitalized patients, nearly half reported at least 1 post‐discharge problem. Most patients have perceptions of what communication did or did not take place between their physicians. While most do not understand the communication process, many expect good communication to occur, and feel that physicians are obligated to communicate with each other. However, patients' perceptions of communication highlight that patient expectations are far from the actual practice in some cases. Nearly half of patients reported feeling negative emotions, such as confusion and frustration, and patients were more likely to experience negative emotions when they also reported a post‐discharge problem. One‐third of patients reported feeling empowered. Empowerment was associated with having a third party who helped advocate for them. Paradoxically, patients whose PCP were not aware of their hospitalization were more likely to feel empowered. Lastly, more patients reported a post‐discharge problem when their PCP was not aware of the hospitalization.

Because this is predominantly a qualitative observational study, it is important to consider the mechanism for these findings since we cannot assume causal relationships. The association of negative emotions, like confusion and frustration, with post‐discharge problems could be explained due to additional stress of the problem itself or that a distressed frame of mind is associated with reporting more problems that may have been overlooked otherwise. In addition, the association between patient empowerment and lack of PCP awareness could be due to the fact that patients are forced to assume a more proactive role in contacting their PCP if they feel that their PCP was not aware. It is equally possible that PCP communication is selectively initiated by hospital physicians when the patients are least empowered. For example, our comparison of demographics for patients whose PCP was aware versus those that were not do suggest that patient characteristics might play a role in whether a patient's PCP is contacted. The association between a third party advocate and patient empowerment is likely explained as the third party is able to keep the patient informed and empowered.

This study has implications for efforts to design a more patient‐centered care transition for hospitalized older patients. First, patients and their proxies should be advocates for good communication to avoid the risks of care transitions. Prior interventions such as use of coaches to boost patient empowerment have had positive results for hospitalized older patients. Moreover, hospitals should keep in mind that problems after discharge are common and are linked to negative emotions, which may lower patient satisfaction or increase liability risk. Similarly, these findings also highlight the importance of keeping PCPs aware of patient hospitalization. For example, PCPs that are aware of hospitalization are better prepared to properly follow‐up on medications, tests, and appointments. The PCP can also help to better prepare the patient for discharge and ease the transition for the patient.

There are several limitations to our study. First and foremost, our small sample size limits our ability to examine statistical significance. This study was part of a short planning grant to design interventions to improve communication with PCPs during hospitalization. Efforts are currently underway to design a communication solution and educational intervention to highlight the importance of contacting PCPs during hospitalization. Because these patients were hospitalized on the teaching service, the resident with the guidance of the teaching attending is responsible for communicating with the PCP. The teaching attending was either a generalist, hospitalist, or specialist who routinely had no a priori relationship with patients prior to the hospitalization. Only 53% of patients were reached by telephone which raises the concern for nonresponse bias. Our low response rate highlights the challenge of doing this type of work with recently discharge patients in low income, underserved areas. In comparing responders and nonresponders, the only difference between the 2 groups was that responders were more likely to be older. One possible reason for this difference may be that older people are more likely to be at home and easier to contact over the phone. Similarly, since data were collected through interviews and adverse events were discussed, these results are subject to recall bias. Efforts were made to reduce this by calling within 2 to 3 weeks after discharge. Lastly, these findings are limited by generalizability. All the patients included in this study were from the University of Chicago Medical Center, which serves largely underserved, African American patients. The experiences of these patients may be unique to this site. In addition, we only studied patients who had a PCP, excluding a population of patients that are at inherent risk due to lack of a coordinating physician to guide ongoing care.

In conclusion, this study suggests that many frail, older patients reported experiencing a post‐discharge problem and patients whose PCPs did not know about their admission were more likely to report a post‐discharge problem. Systematic interventions to improve communications with PCPs during patient care transitions in and out of the hospital are needed.

Forty‐five‐million Americans speak a language other than English and more than 19 million of these speak English less than very wellor are limited English proficient (LEP).1 The number of non‐English‐speaking and LEP people in the US has risen in recent decades, presenting a challenge to healthcare systems to provide high‐quality, patient‐centered care for these patients.2

For outpatients, language barriers are a fundamental contributor to gaps in health care. In the clinic setting, patients who do not speak English well have less access to a usual source of care and lower rates of physician visits and preventive services.36 Even when patients with language barriers do have access to care, they have poorer adherence, decreased comprehension of their diagnoses, decreased satisfaction with care, and increased medication complications.710

Few studies, however, have examined how language influences outcomes of hospital care. Compared to English‐speakers, patients who do not speak English well may experience longer lengths of stay,11 and have more adverse events while in the hospital.12 However, these previous studies have not investigated outcomes immediately post‐hospitalization, such as readmission rates and mortality, nor have they directly addressed the interaction between ethnicity and language.

To understand these questions, we analyzed data collected from a university‐based teaching hospital which cares for patients of diverse cultural and language backgrounds. Using these data, we examined how patients' primary language influenced hospital costs, length of stay (LOS), 30‐day readmission, and 30‐day mortality risk.

Patients and Methods

Results

Conclusion/Discussion

Our results indicate that language barriers may contribute to higher readmission rates for non‐English speakers, but that they have less impact on care efficiency or mortality. This finding of an association between language and readmission, without a similar association with efficiency, suggests a potentially communication‐critical step in care.20, 21 Patients with language barriers are more likely to experience adverse events, and those events are often caused by errors in communication.12 It is conceivable that higher readmission risk for Chinese and Spanish speakers in our study was, at least in part, due to gaps in communication that are present in all patient groups, and exacerbated by the presence of a language barrier. This barrier is likely present during hospitalization but magnified at discharge, limiting caregivers' ability to understand patients' needs for home care, while simultaneously limiting patients' understanding of the discharge plan. After discharge, it is also possible that non‐English speakers are less able to communicate their needs as they arise, ormore subtlyfeel less supported by a primarily English speaking healthcare system. As in other clinical arenas,22 it is quite possible that increased access to professional interpreters in the hospital setting, and particularly at the time of discharge, would enhance communication and outcomes for LEP patients. Our interpreter services data showed that the patients in our study had quite limited access to staff professional interpreters.

Our findings differ somewhat from those of John‐Baptiste et al.,11 who found that language barriers contributed to increased LOS for patients with cardiac and major surgical diagnoses. Our study's findings are akin to recent research suggesting that being a monolingual Spanish speaker or receiving interpreter services may not significantly impact LOS or cost of hospitalization,23 and that LOS and in‐hospital mortality do not differ for non‐English speakers and English speakers after acute myocardial infarction.24 These studies, along with our results, suggest that that care efficiency in the hospital may be driven much more by clinical acuity (eg, the need to respond rapidly to urgent clinical signs such as hypotension, fever and respiratory distress) than by adequacy of communication. For example, elderly LEP patients may be even more likely than English speakers to have vigilant family members at the bedside throughout their hospitalization due to their need for communication assistance; these family members can quickly alert hospital staff to concerning changes in the patient's condition.

Our results also suggest the possibility that language and ethnicity are not monolithic concepts, and that even within language and ethnic groups there are potential differences in care pattern. For example, not speaking English may be a surrogate marker for unmeasured factors such as social supports and access to care. Language is intimately associated with culture; it remains plausible that cultural differences between highly acculturated and less acculturated members of a given ethno‐cultural group may have contributed to our observed differences in readmission rates. Differences in culture and associated factors, such as social support or use of multiple hospital systems, may account for lack of higher readmission risk in Russian speakers, while Chinese and Spanish speakers had higher readmission risk.

In addition, our finding that English‐speaking Latinos had lower readmission risk than any other group may be more consistent with their clinical characteristicseg, younger age, fewer comorbiditiesthan with cultural factors. Our finding that African American patients had the highest readmission risk in our hospital was both surprising and concerning. Some of this increased risk may be explained by clinical characteristics, such as higher comorbidities and higher rates of diagnoses leading to frequent admissions (eg, sickle cell disease); however, the reasons for this disparity deserves further investigation.

Our study has limitations. First, our data are administrative, and lack information about patients' educational attainment, social support, acculturation, utilization of other hospital systems, and usual source of care. Despite this, we were able to account for many significant covariates that might contribute to readmission rates, including age, insurance status, gender, comorbidities, and admission to the intensive care unit.2528 Second, our information about patients' English language proficiency is limited. While direct assessments of English proficiency are more accurate ways to determine a patient's ability to communicate with health care providers in English,29 our language validation work conducted in preparation for this study suggests that most of our patients recorded as having a non‐English primary language (87%) also have a low score on a language acculturation scale.

Third, only 14% of our non‐English speaking subjects utilized professional staff interpreters, and we had no information on the use of professional telephonic interpreters, or ad hoc interpretersfamily members, non‐interpreter staff membersand their impact on our results. It is well‐documented that ad hoc interpreters are used frequently in healthcare, particularly in the hospital setting, and thus we can assume this to be true in our study.30, 31 As noted above, it is likely that the advocacy of family members and friends at the bedside helped to minimize potential differences in care efficiency for patients with language barriers. Finally, our study was performed at a single university based hospital and may not produce results which are applicable to other care settings.

Our findings point to several avenues for future research on language barriers and hospitalized patients. First, the field would benefit from an examination of the impact of easy access to professional interpreters during hospitalization on outcomes of hospital care, in particular on readmission rates. Second, there is need for development and assessment of best practices for creating a culture of professional interpreter utilization in the hospital among physicians and nursing staff. Third, investigation of the role of caregiver presence in the hospital room and how this might differ by patient culture, age and language ability may further elucidate some of the differences across language groups observed in our study. Lastly, a more granular investigation of clinician‐patient communication and the importance of interpersonal processes of care on both patient satisfaction and understanding of and adherence to discharge instructions could lead to the development of detailed interventions to enhance this communication and these outcomes as it has done for communication‐sensitive outcomes in the outpatient arena.3234

In summary, our study suggests that higher risk for readmission can be added to the unfortunate list of outcomes which are worsened due to language barriers, pointing to transition from the hospital as a potentially communication‐critical step in care which may be amenable to intervention. Our findings also suggest that this risk can vary even between groups of patients who do not speak English primarily. Whether and to what degree language and communication barriers aloneincluding access to professional interpreters and patient‐centered communicationduring hospitalization, or differences in caregiver social support both during and after hospitalization as well as access to care post‐hospitalization contribute to these findings is a worthy subject of future research.

Forty‐five‐million Americans speak a language other than English and more than 19 million of these speak English less than very wellor are limited English proficient (LEP).1 The number of non‐English‐speaking and LEP people in the US has risen in recent decades, presenting a challenge to healthcare systems to provide high‐quality, patient‐centered care for these patients.2

For outpatients, language barriers are a fundamental contributor to gaps in health care. In the clinic setting, patients who do not speak English well have less access to a usual source of care and lower rates of physician visits and preventive services.36 Even when patients with language barriers do have access to care, they have poorer adherence, decreased comprehension of their diagnoses, decreased satisfaction with care, and increased medication complications.710

Few studies, however, have examined how language influences outcomes of hospital care. Compared to English‐speakers, patients who do not speak English well may experience longer lengths of stay,11 and have more adverse events while in the hospital.12 However, these previous studies have not investigated outcomes immediately post‐hospitalization, such as readmission rates and mortality, nor have they directly addressed the interaction between ethnicity and language.

To understand these questions, we analyzed data collected from a university‐based teaching hospital which cares for patients of diverse cultural and language backgrounds. Using these data, we examined how patients' primary language influenced hospital costs, length of stay (LOS), 30‐day readmission, and 30‐day mortality risk.

Patients and Methods

Results

Conclusion/Discussion

Our results indicate that language barriers may contribute to higher readmission rates for non‐English speakers, but that they have less impact on care efficiency or mortality. This finding of an association between language and readmission, without a similar association with efficiency, suggests a potentially communication‐critical step in care.20, 21 Patients with language barriers are more likely to experience adverse events, and those events are often caused by errors in communication.12 It is conceivable that higher readmission risk for Chinese and Spanish speakers in our study was, at least in part, due to gaps in communication that are present in all patient groups, and exacerbated by the presence of a language barrier. This barrier is likely present during hospitalization but magnified at discharge, limiting caregivers' ability to understand patients' needs for home care, while simultaneously limiting patients' understanding of the discharge plan. After discharge, it is also possible that non‐English speakers are less able to communicate their needs as they arise, ormore subtlyfeel less supported by a primarily English speaking healthcare system. As in other clinical arenas,22 it is quite possible that increased access to professional interpreters in the hospital setting, and particularly at the time of discharge, would enhance communication and outcomes for LEP patients. Our interpreter services data showed that the patients in our study had quite limited access to staff professional interpreters.

Our findings differ somewhat from those of John‐Baptiste et al.,11 who found that language barriers contributed to increased LOS for patients with cardiac and major surgical diagnoses. Our study's findings are akin to recent research suggesting that being a monolingual Spanish speaker or receiving interpreter services may not significantly impact LOS or cost of hospitalization,23 and that LOS and in‐hospital mortality do not differ for non‐English speakers and English speakers after acute myocardial infarction.24 These studies, along with our results, suggest that that care efficiency in the hospital may be driven much more by clinical acuity (eg, the need to respond rapidly to urgent clinical signs such as hypotension, fever and respiratory distress) than by adequacy of communication. For example, elderly LEP patients may be even more likely than English speakers to have vigilant family members at the bedside throughout their hospitalization due to their need for communication assistance; these family members can quickly alert hospital staff to concerning changes in the patient's condition.

Our results also suggest the possibility that language and ethnicity are not monolithic concepts, and that even within language and ethnic groups there are potential differences in care pattern. For example, not speaking English may be a surrogate marker for unmeasured factors such as social supports and access to care. Language is intimately associated with culture; it remains plausible that cultural differences between highly acculturated and less acculturated members of a given ethno‐cultural group may have contributed to our observed differences in readmission rates. Differences in culture and associated factors, such as social support or use of multiple hospital systems, may account for lack of higher readmission risk in Russian speakers, while Chinese and Spanish speakers had higher readmission risk.

In addition, our finding that English‐speaking Latinos had lower readmission risk than any other group may be more consistent with their clinical characteristicseg, younger age, fewer comorbiditiesthan with cultural factors. Our finding that African American patients had the highest readmission risk in our hospital was both surprising and concerning. Some of this increased risk may be explained by clinical characteristics, such as higher comorbidities and higher rates of diagnoses leading to frequent admissions (eg, sickle cell disease); however, the reasons for this disparity deserves further investigation.

Our study has limitations. First, our data are administrative, and lack information about patients' educational attainment, social support, acculturation, utilization of other hospital systems, and usual source of care. Despite this, we were able to account for many significant covariates that might contribute to readmission rates, including age, insurance status, gender, comorbidities, and admission to the intensive care unit.2528 Second, our information about patients' English language proficiency is limited. While direct assessments of English proficiency are more accurate ways to determine a patient's ability to communicate with health care providers in English,29 our language validation work conducted in preparation for this study suggests that most of our patients recorded as having a non‐English primary language (87%) also have a low score on a language acculturation scale.

Third, only 14% of our non‐English speaking subjects utilized professional staff interpreters, and we had no information on the use of professional telephonic interpreters, or ad hoc interpretersfamily members, non‐interpreter staff membersand their impact on our results. It is well‐documented that ad hoc interpreters are used frequently in healthcare, particularly in the hospital setting, and thus we can assume this to be true in our study.30, 31 As noted above, it is likely that the advocacy of family members and friends at the bedside helped to minimize potential differences in care efficiency for patients with language barriers. Finally, our study was performed at a single university based hospital and may not produce results which are applicable to other care settings.

Our findings point to several avenues for future research on language barriers and hospitalized patients. First, the field would benefit from an examination of the impact of easy access to professional interpreters during hospitalization on outcomes of hospital care, in particular on readmission rates. Second, there is need for development and assessment of best practices for creating a culture of professional interpreter utilization in the hospital among physicians and nursing staff. Third, investigation of the role of caregiver presence in the hospital room and how this might differ by patient culture, age and language ability may further elucidate some of the differences across language groups observed in our study. Lastly, a more granular investigation of clinician‐patient communication and the importance of interpersonal processes of care on both patient satisfaction and understanding of and adherence to discharge instructions could lead to the development of detailed interventions to enhance this communication and these outcomes as it has done for communication‐sensitive outcomes in the outpatient arena.3234

In summary, our study suggests that higher risk for readmission can be added to the unfortunate list of outcomes which are worsened due to language barriers, pointing to transition from the hospital as a potentially communication‐critical step in care which may be amenable to intervention. Our findings also suggest that this risk can vary even between groups of patients who do not speak English primarily. Whether and to what degree language and communication barriers aloneincluding access to professional interpreters and patient‐centered communicationduring hospitalization, or differences in caregiver social support both during and after hospitalization as well as access to care post‐hospitalization contribute to these findings is a worthy subject of future research.

Forty‐five‐million Americans speak a language other than English and more than 19 million of these speak English less than very wellor are limited English proficient (LEP).1 The number of non‐English‐speaking and LEP people in the US has risen in recent decades, presenting a challenge to healthcare systems to provide high‐quality, patient‐centered care for these patients.2

For outpatients, language barriers are a fundamental contributor to gaps in health care. In the clinic setting, patients who do not speak English well have less access to a usual source of care and lower rates of physician visits and preventive services.36 Even when patients with language barriers do have access to care, they have poorer adherence, decreased comprehension of their diagnoses, decreased satisfaction with care, and increased medication complications.710

Few studies, however, have examined how language influences outcomes of hospital care. Compared to English‐speakers, patients who do not speak English well may experience longer lengths of stay,11 and have more adverse events while in the hospital.12 However, these previous studies have not investigated outcomes immediately post‐hospitalization, such as readmission rates and mortality, nor have they directly addressed the interaction between ethnicity and language.

To understand these questions, we analyzed data collected from a university‐based teaching hospital which cares for patients of diverse cultural and language backgrounds. Using these data, we examined how patients' primary language influenced hospital costs, length of stay (LOS), 30‐day readmission, and 30‐day mortality risk.

Patients and Methods

Results