William E. Trick, MD

METHODS

RESULTS

DISCUSSION

METHODS

RESULTS

DISCUSSION

METHODS

RESULTS

DISCUSSION

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?

METHODS

We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.

RESULTS

A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

DISCUSSION

In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?

METHODS

We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.

RESULTS

A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

DISCUSSION

In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.

Despite widespread efforts to predict 30‐day rehospitalizations among discharged general medical patients,[1, 2, 3] not many strategies have incorporated patient‐reported outcome (PRO) measures in predictive models.[4] This despite the many longitudinal studies of the ambulatory population that demonstrate the higher likelihood of hospitalizations among those who score poorly on General Self‐Rated Health (GSRH),[5, 6, 7] baseline or declining Health‐Related Quality of Life,[8, 9, 10, 11, 12] psychological symptoms,[13, 14] and physical symptoms assessments.[15] One of the few existing studies that included PRO measures in 30‐day readmission models showed the predictive value of the 12‐item short form (SF12) Physical Component Score.[16] Others showed that persistent symptoms were associated with readmissions in patients with heart disease.[17, 18]

The paucity of efforts to connect PRO measures to utilization may be due to the limited availability of these measures in routine clinical records and the incomplete knowledge about how various PRO measures may fluctuate during episodes of acute illnesses and their treatments during hospitalizations. Health perception measures reflect both enduring features like self‐concept as well as dynamic features like a person's immediate health status.[19] As such, GSRH reflects the presence of chronic illnesses but is also responsive to acute events.[20, 21] Similarly, Health‐Related Quality of Life measures are dynamic as they decline around episodes of acute illness but are stable over a longer time window in their tendency to recover.[22] We do not know how fluctuations in measures of symptom burden, perceived health, and quality of life around the hospital‐to‐home transition may differentially inform readmission risk. Using a longitudinal cohort study, we addressed 2 questions: (1) How do PRO measures change when measured serially during the hospital‐to‐home transition? (2) How does the relative timing of each PRO measure variably inform the risk of subsequent utilization events including hospital readmissions?

METHODS

We conducted a longitudinal cohort study using data originally collected for a trial (ClinicalTrials.gov Identifier NCT01391026) of an intervention that was shown to have no associations with variables evaluated in this study. Patients were recruited from the John H. Stroger Hospital of Cook County, an urban safety‐net hospital that serves 128 municipalities in northeastern Illinois including the City of Chicago. Patients were eligible if they (1) were admitted to the general medical wards, the medical intensive care unit, or the cardiac care unit between May 2011 and February 2012; (2) had a clinic appointment in the Hospital's general medicine clinic (GMC) in the prior 12 months to facilitate follow‐up; and (3) were able to communicate independently in English or Spanish. Randomly selected patients were approached during their hospitalization and consenting subjects completed an in‐person questionnaire on the day of discharge. Subjects were contacted by telephone around 30, 90, and 180 days thereafter to complete follow‐up questionnaires; we began calling patients around 2 weeks prior to the target day anticipating noncontact on the first attempts. All telephone interviews were conducted by research assistants who had no clinical training and who did not give care‐related advice to patients based on their survey response. A few patients whose follow‐up survey window straddled the date of a scheduled clinic appointment were invited to complete the questionnaire in the GMC's waiting area using computer kiosks enabled with audio computer‐assisted self‐interview technology described elsewhere.[23] The Charlson Comorbidity Index was calculated inclusive of diagnostic codes assigned over 3 months preceding the index hospitalization.[24]

The following instruments were administered at each interview. The physical symptom severity portion of the Memorial Symptom Assessment Scale (MSAS) solicited the severity rank (none/a little bit/somewhat/quite a bit/very much) of 17 physical symptoms in the last week; the score was calculated by averaging the severity rank of the 12 most common symptom in the sample.[25, 26] The Patient Reported Outcomes Measurement Information System (PROMIS) Global Health Short Form is an instrument assessing GSRH (1 item), Social Activities (1 item), Global Physical Health (4 items), and Global Mental Health (4 items including a single‐item quality‐of‐life measure). Fatigue and pain for Global Physical Health, and emotional health for Global Mental Health were assessed over the past 7 days. Each of the 2 Global Health scores was standardized to a national mean of 50 and standard deviation of 10.[27]

The rate of survey completion at each follow‐up was calculated. Characteristics of participants were tabulated. Characteristics of patients censored prior to study completion were compared with patients with complete data. Box plots for MSAS physical symptom severity, and Global Physical and Mental Health scores were constructed to illustrate the comparisons of the mean scores between each consecutive survey period using t tests assuming unequal variance. A similar box plot of GSRH illustrated the comparison of the median score between consecutive surveys using the rank sum test. Hospital‐based utilization events were defined as either an emergency department visit or hospitalization at 1 of the 2 hospitals of the Cook County Health & Hospitals System (CCHHS). After accounting for patient data censored due to death (date reported by family) or withdrawal from study, Kaplan‐Meier curves showing time to first hospital‐based utilization event during each interval between surveys were drawn separately for above‐ and below‐median MSAS, Global Physical and Mental Health scores, and for poor or fair versus good, very good, or excellent GSRH assessment. The null hypothesis that the survivor functions were equal between the better and worse median quantiles or GSRH categories was tested using the log‐rank test at 14 and 30 days from survey completion. Hazard ratios for time to first utilization event within 14 days of each survey were calculated for the MSAS score, Global Physical and Mental Health as continuous variables, and GSRH response categories relative to poor using bivariate and multivariate Cox proportional hazard equations. Multivariate models incorporated the following 5 covariates: at least 1 utilization event during the study period prior to the survey, Charlson score, age, gender, and race/ethnicity category. Likelihood ratio statistics were calculated to test the hypothesis that the model including the PRO measure and covariates predicted the outcome equally well compared to the nested model with only covariates. We used the traditional threshold of .05 when reporting significance. All analyses were performed in Stata 13 (StataCorp, College Station, TX). The methods for patient consent, data collection, analyses, and reporting were reviewed and approved by the CCHHS institutional review board.

RESULTS

A total of 196 patients completed the initial survey. The completion rates were 98%, 90%, and 88% for the 30‐, 90‐, and 180‐day follow‐up surveys, respectively. As shown in Table 1, participants average age was 52 years, and about half were women. The majority was non‐Hispanic black, and 21% preferred to complete the survey in Spanish. Diabetes, congestive heart failure, cancer, and chronic pulmonary disease were each prevalent in at least one‐fifth of our patient cohort. Demographic characteristics were similar between the 160 patients who completed all 3 follow‐up surveys and the 36 who missed at least 1 follow‐up survey. Among the latter group, 1 withdrew at 30 days, 1 withdrew and 4 had died at 90 days, and 1 withdrew and 9 had died at 180 days.

Figure 1 shows a timeline of the follow‐up surveys and utilization events in the form of overlapping histograms. The majority of 30‐day follow‐up questionnaires were completed earlier than targeted, at a median of 17 (interquartile range [IQR] 16, 20) days after discharge. Similarly, questionnaires targeted for 90 and 180 days were completed at medians of 78 (IQR 7684) and 167 (IQR 166169) days from discharge. Fifty‐four (28%) patients experienced a first utilization event in the first 30 days following discharge. During the 60‐, 90‐, and 30‐day intervals after the first, second, and third follow‐up surveys, respectively, 63 (33%), 54 (31%), and 16 (9%) patients experienced a first utilization event.

Figure 1

A significant improvement in MSAS physical symptom severity was detected between the hospitalization and the 30‐day follow‐up (Figure 2A). Although the mean Global Physical Health score was below the national mean of 50 at every survey period, a similar improvement in the measure was noted between the hospitalization and the 30‐day follow‐up (Figure 2B). The mean Global Mental Health score was also below the national mean but remained stable throughout the study (Figure 2C). The median GSRH was stable at 2 (IQR 23) at every survey wave (Figure 2D). Of note, compared to patients who completed all 3 follow‐up surveys, patients who missed at least 1 follow‐up reported higher MSAS score (1.5 vs 1.8, P=0.03), lower Global Physical Health (36.1 vs 33.5, P=0.09), and lower Global Mental Health (44.7 vs 41.0, P=0.03) during their hospitalization. In addition, patients with complete data experienced an average of 1.2 utilization events during the study, whereas those with missing data experienced an average of 2.1 utilization events (P=0.03).

Figure 2

The MSAS physical symptom severity and Global Physical Health scores from the index hospitalizations did not identify patients with a first utilization event within 30 days. However, patients with poor Global Mental Health and GSRH in the hospital were more likely to experience a utilization event within 14 days of discharge (Figure 3). During the postdischarge period, patients scoring poorly on each of the PRO measures trended toward a greater risk of an early utilization event, but the association between utilization and MSAS was most consistently significant (Figure 3A). In general, the associations with MSAS, Global Physical Health, and GSRH were stronger with the risk of utilization events within 14 days than within 30 days (Figure 3A,B,D). The Global Mental Health score was not associated with a subsequent utilization when measured during the 180‐day postdischarge period.

Figure 3

As shown in Table 2, Cox proportional hazard models incorporating covariates preserved most of the significant associations seen in the unadjusted analyses. Global Mental Health and good relative to poor GSRH obtained during the hospitalization remained significant. MSAS obtained at each postdischarge follow‐up trended positively with utilization and was statistically significant at 90 and 180 days. Global Physical Health obtained at each postdischarge follow‐up similarly trended negatively with utilization and was significant at 180 days. Each multivariate model incorporating a PRO measure with a significant coefficient contributed to better fit of the predictive model compared to the nested model without the PRO measure.

DISCUSSION

In this longitudinal cohort study, patients, on average, reported relatively severe symptoms, low PROMIS Global Physical and Mental Health scores, and poor GSRH during the inpatient stay in an urban safety‐net hospital. Symptom severity and Global Physical Health improved, on average, by 30 days before stabilizing, but their poor levels in the hospital did not predict 30‐day hospital‐based utilization events. On the other hand, Global Mental Health and GSRH were stable through hospitalizations, and patients scoring poorly on these measures were at greater risk of utilization events within 14 days of discharge. PRO measures obtained during the 180‐day postdischarge period trended toward distinguishing populations with greater baseline risk of proximate utilization events. However, MSAS physical symptom severity and Global Physical Health were more consistently predictive of these events at statistically significant levels compared to Global Mental Health and GSRH in our relatively small sample of patients. Each of these measures selectively improved the fit‐of‐risk prediction models for hospital‐based utilization.

Some of the heterogeneity in readmission risk is explained by differences in PRO measures. Although the MSAS score and Global Physical Health assessment were reliable predictors of utilization when measured in ambulatory settings, they were less discriminating during acute hospitalizations when everyone, on average, reported severe symptoms and poor function. Our results were consistent with other studies that demonstrated the fairly rapid recovery in symptoms that follow hospitalizations,[28, 29] and these measures may become informative of utilization risk as early as 2 weeks postdischarge. GSRH and Global Mental Health (a measure of health‐related quality of life) only predicted utilizations immediately at hospital discharge. As multidimensional measures that reflect physical, social, and emotional capacity, these measures may indicate vulnerabilities in patients least able to handle the stresses of the early postdischarge period.

There is growing momentum around collecting PRO measures in routine clinical care as quality indicators that capture patient‐centered concerns.[30] Our study explored a novel application of these measures whose routine collection will likely proliferate, not solely for the purpose of helping healthcare systems identify patients at risk of unplanned resource utilization. Although multidimensional PRO measures seldom reflect conditions directly modifiable by simple interventions, we believe that the association between physical symptom burden and utilization in our data reveals a possible target for practice improvement. Hospitalists have contributed enormously to shorter lengths of stay that risk sicker and quicker discharges.[31] To mitigate its potential side effects on symptom management, a discharge plan that acknowledges physical symptoms that sometimes persist or recur beyond the hospitalization may be appropriate. This may be accomplished by ensuring that acute symptoms are resolving, giving clear instructions for symptom management at home, as now the standard of care for conditions like asthma,[32] and explicitly communicating the presence of residual symptoms to providers entrusted with continuity care. As an effective feedback measure that can drive continuous quality improvement, we believe that a technology‐based surveillance strategy that spans both the inpatient and outpatient domains is necessary.[23]

There are some notable similarities and differences between the results of our study and a recent hospital‐based study of PRO measures that used data from the Multi‐Center Hospitalist Project.[16] The Physical Component Score of the SF12 is similar to the PROMIS Global Physical Health score in that both incorporate measures of physical function, perceived health, pain, and energy level. Curiously, the SF12 Physical Component Score, but not the PROMIS Global Physical Health score, was associated with 30‐day rehospitalizations. An important difference between the measures is where the SF12 asks about limitations during the past 4 weeks the PROMIS instrument inquires about physical function in general and levels of fatigue and pain in the past 7 days. Considering most hospitalizations last <7 days, the PROMIS instrument may better reflect the declines associated with the acute illness related to the hospitalization than the SF12 score. Additionally, the discrepancy between the association between hospital‐based GSRH and utilization in our study and the absence, thereof, in Hasan et al. is noteworthy. The difference here may be explained by their use of a 0‐ to 100‐point response scale in contrast to our study's verbally labeled 5‐point scale in the PROMIS instrument. The range of rating scales for survey questions is traditionally governed by the tension between the difficulty with mapping respondents judgment on an excessively large scale on one hand, and the failure of insufficient response options to discriminate between respondents with different underlying judgment on the other.[33] We suspect the former to be a drawback of the unlabeled 100‐point response scale, and conjecture that an association might be found in the Multi‐Center Hospitalist Study data if the responses were grouped into summative categories.

We recognize several limitations in our study. The first is the generalizability of our patient population to others, not insignificantly because of the high proportion of the uninsured (around 70% during the study period) and racial/ethnic minorities among them. Although utilization patterns are clearly affected by socioeconomic status,[34] there may also be differences in the way validated PRO measures are calibrated between patients of public and private healthcare systems.[35] Another limitation is our inability to count utilization events at institutions outside of the CCHHS during our study. However, because the study was conducted prior to Cook County's Medicaid expansion demonstration program as part of the Affordable Care Act,[36] many patients established in our system faced barriers to receiving nonemergency care outside of the CCHHS supporting our assumption that few of our patients were discharged from other hospitals. Causality cannot be established in observational studies. Consequently, high prior‐symptom burden may be associated with utilizations through unmeasured variables. Measures of symptom burden are vulnerable to overendorsement and amplification.[37, 38] Inferences based on statistical significance are affected by sample size, and our conclusions may change if conducted with a larger number of participants. Our response rates were excellent through the survey waves, but we did not achieve perfect follow‐up. Worse levels of PRO responses and higher levels of utilization among censored patients biased our results toward the null. Finally, although we did not find any predominant comorbidities associated with hospital‐based utilizations in our sample, our analyses may be vulnerable to inadequate control for illness severity, which may also have biased our results.

PRO measures are likely to be useful in clinical medicine.[39] But to fully apply the powers of PROs in informing clinically and operationally relevant outcomes, we must actively develop a system for obtaining these measures in routine clinical care. The availability of patient downtime makes hospitalizations conducive to gathering patient‐generated data, and may further enhance patient‐provider communication if survey output was readily available in electronic medical records. Exploring innovative strategies for collecting PROs in the hospital and beyond remains our future work.

At times central venous catheters are essential to the delivery of appropriate medical care. Because catheter‐related complications are associated with limited operator experience,1 insertion technique,2 and venous site of insertion (eg, femoral, internal jugular, or subclavian vein),3 house staff training programs strive to provide their residents with appropriate training and oversight for this skill. Most quality improvement initiatives directed at reducing complications associated with central venous catheters have focused on patients in the intensive care unit (ICU).4, 5 However, in some hospitals more central venous catheters are inserted in patients not in the ICU,6 and practices that increase the risk of complications may be more common on wards.7

In our hospital, most catheters are placed in the femoral vein. Because femoral venous placement likely increases a patient's risk of thrombosis, hematoma, and bloodstream infection,8 we developed a program to change residents' choice of venous insertion site and improve their infection‐control practices during their general medicine ward rotation. The program provided simulated hands‐on experience in a simulation laboratory. We evaluated our intervention through a firm‐based clinical trial that compared the usual practice to our intervention. We compared infection‐control practices and resident choice of venous insertion site between the intervention and control groups; we also assessed residents' knowledge about catheter‐related complications, and we monitored patients for catheter‐related complications.

METHODS

RESULTS

DISCUSSION

An educational session designed to teach residents appropriate central venous catheter insertion practices that included simulated hands‐on training increased knowledge about insertion‐related complications and improved certain infection‐control practices. Although residents' confidence in inserting subclavian or internal jugular catheters initially improved, our training session did not change the choice of venous insertion site from femoral to subclavian or internal jugular veins, possibly because there were few opportunities for residents to insert a catheter during the 4‐week general medical ward rotations. Thus, although an active educational intervention improved the knowledge and confidence of residents, it had a minimal effect on behavior (only improved certain infection‐control practices). Catheter‐associated complications were infrequent and similar in the intervention and control groups.

Central venous catheter insertion is a skill that many general internists do not perform10; however, until recently the American Board of Internal Medicine considered it a requisite skill for internal medicine residents, and most residents at our hospital reported a desire to learn this skill. Although in our study complications were infrequent, suggesting that a change in venous insertion site is unlikely to dramatically improve patient safety, we believe that residents should become skilled at inserting catheters in internal jugular or subclavian veins, the currently recommended optimal venous insertion.8

There is evidence that single educational interventions are unlikely to result in substantial, sustained behavioral change, especially passive educational programs.11 However, a previous study documented a change in provider behavior and possibly a reduction in bloodstream infections after a single hands‐on training session.12 Our hands‐on educational format was very popular and likely improved some infection‐control practices but did not change provider behavior about choice of venous insertion site. In other institutions, mentoring residents on appropriate catheter insertion technique has been accomplished by establishing a procedure service13 or by resident rotation in a high‐volume location (eg, cardiac catheterization laboratory).14 Another option to facilitate behavioral change would be to provide a portable ultrasound machine, as requested by our residents, which may reduce complication rates.15, 16 At our hospital, we decided to supplement hands‐on training with expert bedside supervision during catheter insertion; the expert is provided through a procedure service that is led by hospitalists. The procedure service has a dedicated portable ultrasound machine to assist with internal jugular vein cannulation.

By the end of our study period, residents' confidence in subclavian or internal jugular catheter insertions had returned to presession levels; however, they reported increased confidence in femoral venous catheter insertions. These findings suggest that the session increased residents' confidence with catheter insertions in general, but not specifically for venous sites for which they had no previous experience. For subclavian or internal jugular catheter insertions, their confidence decayed to the presession baseline, likely because of few opportunities to insert catheters in patients; on average, each resident inserts 1 central venous catheter on the general medicine wards approximately every 4 months.

Our survey found that our intervention changed residents' attitudes about infection‐control practices. In particular, intervention‐group residents reported that they were more likely to remove unnecessary catheters and that they had used a mask and large drape during catheter insertion. Use of full‐barrier precautions (ie, sterile gloves and gown, large sterile drape, cap, and mask) has been shown to reduce the risk of bloodstream infection2 and is included in national guidelines.17 Adherence to these guidelines has been included in successful quality improvement initiatives.4, 5, 18 Compared to internists' adherence to recommendations for infection control reported in another survey,10 residents who attended our educational session reported more use of large sterile drapes (94% vs. 35%) or masks (75% vs. 66%); however, they were less likely to use a sterile gown (50% vs. 72%). Use of a large sterile drape is common in our hospital, likely because the drape is included in the central venous catheter package. We suspect that at our hospital, poor adherence to certain recommendations (eg, using a sterile gown) was due in part to difficulty accessing supplies. Another possibility is that use of a cap, compared to use of large drapes, is perceived as not giving the patient much additional protection. In fact, there is no evidence that using a cap provides benefit beyond that of other, more intuitively beneficial recommended infection‐control practices, such as using sterile gloves and a large sterile drape. The procedure service has addressed the supply problem by stocking hard‐to‐find items on a procedure cart.

Only 2 clinically evident complications associated with catheter insertion occurred (one patient with clinical sepsis and one with a hematoma). Although it is possible that we missed minor complications, our rates were similar to those reported by other investigators: clinically diagnosed venous thromboembolism, 0%2.2%3, 19, 20; pneumothorax, 1.4%21; catheter‐associated primary bloodstream infection, 1‐6/1000 catheter‐days.22, 23 Comparing complication rates was hindered by variability in definitions, methods of ascertainment, and populations evaluated. For example, the rate of venous thromboembolism was dramatically higher when routine diagnostic imaging was used, and detection of catheter‐associated infections likely increased when catheter‐tip cultures were routinely performed. We required clinical evidence of complications, and our study differs from others in that we evaluated general medicine ward patients.

This study had several limitations. Placement of central venous catheters on general medicine wards was less frequent than we anticipated based on a brief period of pilot data collection; therefore, our study was not powered to detect relatively small changes in venous insertion sites or differences in complications. Also, because direct observation was not possible, we relied on self‐reported adherence to infection‐control practices. However, intervention residents' self‐reported poor adherence to gown, glove, and cap use suggests that their responses were unbiased.

An educational session focused on central venous catheter insertion practices was well received by residents, increased their knowledge about complications, and improved infection‐control practices, but had no effect on increasing use of subclavian or internal jugular veins for catheter insertion. Despite continued frequent use of femoral venous catheters, clinically apparent complications were infrequent. However, we believe it is important to teach residents optimal catheter insertion techniques, including preferential placement of catheters in subclavian or internal jugular veins. Therefore, the section of hospital medicine at our hospital initiated a procedure service that provides expert bedside supervision, including use of a portable ultrasound machine, for catheter insertions.

At times central venous catheters are essential to the delivery of appropriate medical care. Because catheter‐related complications are associated with limited operator experience,1 insertion technique,2 and venous site of insertion (eg, femoral, internal jugular, or subclavian vein),3 house staff training programs strive to provide their residents with appropriate training and oversight for this skill. Most quality improvement initiatives directed at reducing complications associated with central venous catheters have focused on patients in the intensive care unit (ICU).4, 5 However, in some hospitals more central venous catheters are inserted in patients not in the ICU,6 and practices that increase the risk of complications may be more common on wards.7

In our hospital, most catheters are placed in the femoral vein. Because femoral venous placement likely increases a patient's risk of thrombosis, hematoma, and bloodstream infection,8 we developed a program to change residents' choice of venous insertion site and improve their infection‐control practices during their general medicine ward rotation. The program provided simulated hands‐on experience in a simulation laboratory. We evaluated our intervention through a firm‐based clinical trial that compared the usual practice to our intervention. We compared infection‐control practices and resident choice of venous insertion site between the intervention and control groups; we also assessed residents' knowledge about catheter‐related complications, and we monitored patients for catheter‐related complications.

METHODS

RESULTS

DISCUSSION

An educational session designed to teach residents appropriate central venous catheter insertion practices that included simulated hands‐on training increased knowledge about insertion‐related complications and improved certain infection‐control practices. Although residents' confidence in inserting subclavian or internal jugular catheters initially improved, our training session did not change the choice of venous insertion site from femoral to subclavian or internal jugular veins, possibly because there were few opportunities for residents to insert a catheter during the 4‐week general medical ward rotations. Thus, although an active educational intervention improved the knowledge and confidence of residents, it had a minimal effect on behavior (only improved certain infection‐control practices). Catheter‐associated complications were infrequent and similar in the intervention and control groups.

Central venous catheter insertion is a skill that many general internists do not perform10; however, until recently the American Board of Internal Medicine considered it a requisite skill for internal medicine residents, and most residents at our hospital reported a desire to learn this skill. Although in our study complications were infrequent, suggesting that a change in venous insertion site is unlikely to dramatically improve patient safety, we believe that residents should become skilled at inserting catheters in internal jugular or subclavian veins, the currently recommended optimal venous insertion.8

There is evidence that single educational interventions are unlikely to result in substantial, sustained behavioral change, especially passive educational programs.11 However, a previous study documented a change in provider behavior and possibly a reduction in bloodstream infections after a single hands‐on training session.12 Our hands‐on educational format was very popular and likely improved some infection‐control practices but did not change provider behavior about choice of venous insertion site. In other institutions, mentoring residents on appropriate catheter insertion technique has been accomplished by establishing a procedure service13 or by resident rotation in a high‐volume location (eg, cardiac catheterization laboratory).14 Another option to facilitate behavioral change would be to provide a portable ultrasound machine, as requested by our residents, which may reduce complication rates.15, 16 At our hospital, we decided to supplement hands‐on training with expert bedside supervision during catheter insertion; the expert is provided through a procedure service that is led by hospitalists. The procedure service has a dedicated portable ultrasound machine to assist with internal jugular vein cannulation.

By the end of our study period, residents' confidence in subclavian or internal jugular catheter insertions had returned to presession levels; however, they reported increased confidence in femoral venous catheter insertions. These findings suggest that the session increased residents' confidence with catheter insertions in general, but not specifically for venous sites for which they had no previous experience. For subclavian or internal jugular catheter insertions, their confidence decayed to the presession baseline, likely because of few opportunities to insert catheters in patients; on average, each resident inserts 1 central venous catheter on the general medicine wards approximately every 4 months.

Our survey found that our intervention changed residents' attitudes about infection‐control practices. In particular, intervention‐group residents reported that they were more likely to remove unnecessary catheters and that they had used a mask and large drape during catheter insertion. Use of full‐barrier precautions (ie, sterile gloves and gown, large sterile drape, cap, and mask) has been shown to reduce the risk of bloodstream infection2 and is included in national guidelines.17 Adherence to these guidelines has been included in successful quality improvement initiatives.4, 5, 18 Compared to internists' adherence to recommendations for infection control reported in another survey,10 residents who attended our educational session reported more use of large sterile drapes (94% vs. 35%) or masks (75% vs. 66%); however, they were less likely to use a sterile gown (50% vs. 72%). Use of a large sterile drape is common in our hospital, likely because the drape is included in the central venous catheter package. We suspect that at our hospital, poor adherence to certain recommendations (eg, using a sterile gown) was due in part to difficulty accessing supplies. Another possibility is that use of a cap, compared to use of large drapes, is perceived as not giving the patient much additional protection. In fact, there is no evidence that using a cap provides benefit beyond that of other, more intuitively beneficial recommended infection‐control practices, such as using sterile gloves and a large sterile drape. The procedure service has addressed the supply problem by stocking hard‐to‐find items on a procedure cart.

Only 2 clinically evident complications associated with catheter insertion occurred (one patient with clinical sepsis and one with a hematoma). Although it is possible that we missed minor complications, our rates were similar to those reported by other investigators: clinically diagnosed venous thromboembolism, 0%2.2%3, 19, 20; pneumothorax, 1.4%21; catheter‐associated primary bloodstream infection, 1‐6/1000 catheter‐days.22, 23 Comparing complication rates was hindered by variability in definitions, methods of ascertainment, and populations evaluated. For example, the rate of venous thromboembolism was dramatically higher when routine diagnostic imaging was used, and detection of catheter‐associated infections likely increased when catheter‐tip cultures were routinely performed. We required clinical evidence of complications, and our study differs from others in that we evaluated general medicine ward patients.

This study had several limitations. Placement of central venous catheters on general medicine wards was less frequent than we anticipated based on a brief period of pilot data collection; therefore, our study was not powered to detect relatively small changes in venous insertion sites or differences in complications. Also, because direct observation was not possible, we relied on self‐reported adherence to infection‐control practices. However, intervention residents' self‐reported poor adherence to gown, glove, and cap use suggests that their responses were unbiased.

An educational session focused on central venous catheter insertion practices was well received by residents, increased their knowledge about complications, and improved infection‐control practices, but had no effect on increasing use of subclavian or internal jugular veins for catheter insertion. Despite continued frequent use of femoral venous catheters, clinically apparent complications were infrequent. However, we believe it is important to teach residents optimal catheter insertion techniques, including preferential placement of catheters in subclavian or internal jugular veins. Therefore, the section of hospital medicine at our hospital initiated a procedure service that provides expert bedside supervision, including use of a portable ultrasound machine, for catheter insertions.

At times central venous catheters are essential to the delivery of appropriate medical care. Because catheter‐related complications are associated with limited operator experience,1 insertion technique,2 and venous site of insertion (eg, femoral, internal jugular, or subclavian vein),3 house staff training programs strive to provide their residents with appropriate training and oversight for this skill. Most quality improvement initiatives directed at reducing complications associated with central venous catheters have focused on patients in the intensive care unit (ICU).4, 5 However, in some hospitals more central venous catheters are inserted in patients not in the ICU,6 and practices that increase the risk of complications may be more common on wards.7

In our hospital, most catheters are placed in the femoral vein. Because femoral venous placement likely increases a patient's risk of thrombosis, hematoma, and bloodstream infection,8 we developed a program to change residents' choice of venous insertion site and improve their infection‐control practices during their general medicine ward rotation. The program provided simulated hands‐on experience in a simulation laboratory. We evaluated our intervention through a firm‐based clinical trial that compared the usual practice to our intervention. We compared infection‐control practices and resident choice of venous insertion site between the intervention and control groups; we also assessed residents' knowledge about catheter‐related complications, and we monitored patients for catheter‐related complications.

METHODS

RESULTS

DISCUSSION

An educational session designed to teach residents appropriate central venous catheter insertion practices that included simulated hands‐on training increased knowledge about insertion‐related complications and improved certain infection‐control practices. Although residents' confidence in inserting subclavian or internal jugular catheters initially improved, our training session did not change the choice of venous insertion site from femoral to subclavian or internal jugular veins, possibly because there were few opportunities for residents to insert a catheter during the 4‐week general medical ward rotations. Thus, although an active educational intervention improved the knowledge and confidence of residents, it had a minimal effect on behavior (only improved certain infection‐control practices). Catheter‐associated complications were infrequent and similar in the intervention and control groups.

Central venous catheter insertion is a skill that many general internists do not perform10; however, until recently the American Board of Internal Medicine considered it a requisite skill for internal medicine residents, and most residents at our hospital reported a desire to learn this skill. Although in our study complications were infrequent, suggesting that a change in venous insertion site is unlikely to dramatically improve patient safety, we believe that residents should become skilled at inserting catheters in internal jugular or subclavian veins, the currently recommended optimal venous insertion.8

There is evidence that single educational interventions are unlikely to result in substantial, sustained behavioral change, especially passive educational programs.11 However, a previous study documented a change in provider behavior and possibly a reduction in bloodstream infections after a single hands‐on training session.12 Our hands‐on educational format was very popular and likely improved some infection‐control practices but did not change provider behavior about choice of venous insertion site. In other institutions, mentoring residents on appropriate catheter insertion technique has been accomplished by establishing a procedure service13 or by resident rotation in a high‐volume location (eg, cardiac catheterization laboratory).14 Another option to facilitate behavioral change would be to provide a portable ultrasound machine, as requested by our residents, which may reduce complication rates.15, 16 At our hospital, we decided to supplement hands‐on training with expert bedside supervision during catheter insertion; the expert is provided through a procedure service that is led by hospitalists. The procedure service has a dedicated portable ultrasound machine to assist with internal jugular vein cannulation.

By the end of our study period, residents' confidence in subclavian or internal jugular catheter insertions had returned to presession levels; however, they reported increased confidence in femoral venous catheter insertions. These findings suggest that the session increased residents' confidence with catheter insertions in general, but not specifically for venous sites for which they had no previous experience. For subclavian or internal jugular catheter insertions, their confidence decayed to the presession baseline, likely because of few opportunities to insert catheters in patients; on average, each resident inserts 1 central venous catheter on the general medicine wards approximately every 4 months.

Our survey found that our intervention changed residents' attitudes about infection‐control practices. In particular, intervention‐group residents reported that they were more likely to remove unnecessary catheters and that they had used a mask and large drape during catheter insertion. Use of full‐barrier precautions (ie, sterile gloves and gown, large sterile drape, cap, and mask) has been shown to reduce the risk of bloodstream infection2 and is included in national guidelines.17 Adherence to these guidelines has been included in successful quality improvement initiatives.4, 5, 18 Compared to internists' adherence to recommendations for infection control reported in another survey,10 residents who attended our educational session reported more use of large sterile drapes (94% vs. 35%) or masks (75% vs. 66%); however, they were less likely to use a sterile gown (50% vs. 72%). Use of a large sterile drape is common in our hospital, likely because the drape is included in the central venous catheter package. We suspect that at our hospital, poor adherence to certain recommendations (eg, using a sterile gown) was due in part to difficulty accessing supplies. Another possibility is that use of a cap, compared to use of large drapes, is perceived as not giving the patient much additional protection. In fact, there is no evidence that using a cap provides benefit beyond that of other, more intuitively beneficial recommended infection‐control practices, such as using sterile gloves and a large sterile drape. The procedure service has addressed the supply problem by stocking hard‐to‐find items on a procedure cart.

Only 2 clinically evident complications associated with catheter insertion occurred (one patient with clinical sepsis and one with a hematoma). Although it is possible that we missed minor complications, our rates were similar to those reported by other investigators: clinically diagnosed venous thromboembolism, 0%2.2%3, 19, 20; pneumothorax, 1.4%21; catheter‐associated primary bloodstream infection, 1‐6/1000 catheter‐days.22, 23 Comparing complication rates was hindered by variability in definitions, methods of ascertainment, and populations evaluated. For example, the rate of venous thromboembolism was dramatically higher when routine diagnostic imaging was used, and detection of catheter‐associated infections likely increased when catheter‐tip cultures were routinely performed. We required clinical evidence of complications, and our study differs from others in that we evaluated general medicine ward patients.

This study had several limitations. Placement of central venous catheters on general medicine wards was less frequent than we anticipated based on a brief period of pilot data collection; therefore, our study was not powered to detect relatively small changes in venous insertion sites or differences in complications. Also, because direct observation was not possible, we relied on self‐reported adherence to infection‐control practices. However, intervention residents' self‐reported poor adherence to gown, glove, and cap use suggests that their responses were unbiased.

An educational session focused on central venous catheter insertion practices was well received by residents, increased their knowledge about complications, and improved infection‐control practices, but had no effect on increasing use of subclavian or internal jugular veins for catheter insertion. Despite continued frequent use of femoral venous catheters, clinically apparent complications were infrequent. However, we believe it is important to teach residents optimal catheter insertion techniques, including preferential placement of catheters in subclavian or internal jugular veins. Therefore, the section of hospital medicine at our hospital initiated a procedure service that provides expert bedside supervision, including use of a portable ultrasound machine, for catheter insertions.