Greg Maynard, MD

Enactment of federal legislation imposing hospital reimbursement penalties for excess rates of rehospitalizations among Medicare fee for service beneficiaries markedly increased interest in hospital quality improvement (QI) efforts to reduce the observed 30‐day rehospitalization of 19.6% in this elderly population.[1, 2] The Congressional Budget Office estimated that reimbursement penalties to hospitals for high readmission rates are expected to save the Medicare program approximately $7 billion between 2010 and 2019.[3] These penalties are complemented by resources from the Center for Medicare and Medicaid Innovation aiming to reduce hospital readmissions by 20% by the end of 2013 through the Partnership for Patients campaign.[4] Although potential financial penalties and provision of resources for QI intensified efforts to enhance the quality of the hospital discharge transition, patient safety risks associated with hospital discharge are well documented.[5, 6] Approximately 20% of patients discharged from the hospital may suffer adverse events,[7, 8] of which up to three‐quarters (72%) are medication related,[9] and over one‐third of required follow‐up testing after discharge is not completed.[10] Such findings indicate opportunities for improvement in the discharge process.[11]

Numerous publications describe studies aiming to improve the hospital discharge process and mitigate these hazards, though a systematic review of interventions to reduce 30‐day rehospitalization indicated that the existing evidence base for the effectiveness of transition interventions demonstrates irregular effectiveness and limitations to generalizability.[12] Most studies showing effectiveness are confined to single academic medical centers. Existing evidence supports multifaceted interventions implemented in both the pre‐ and postdischarge periods and focused on risk assessment and tailored, patient‐centered application of interventions to mitigate risk. For example Project RED (Re‐Engineered Discharge) applied a bundled intervention consisting of intensified patient education and discharge planning, improved medication reconciliation and discharge instructions, and longitudinal patient contact with follow‐up phone calls and a dedicated discharge advocate.[13] However, the mean age of patients participating in the study was 50 years, and it excluded patients admitted from or discharged to skilled nursing facilities, making generalizability to the geriatric population uncertain.

An integral aspect of QI projects is the contribution of local context to translation of best practices to disparate settings.[14, 15, 16] Most available reports of successful interventions to reduce rehospitalization have not fully described the specifics of either the intervention context or design. Moreover, the available evidence base for common interventions to reduce rehospitalization was developed in the academic setting. Validation of single academic center studies in a broader healthcare context is necessary.

Project BOOST (Better Outcomes for Older adults through Safe Transitions) recruited a diverse national cohort of both academic and nonacademic hospitals to participate in a QI effort to implement best practices for hospital discharge care transitions using a national collaborative approach facilitated by external expert mentorship. This study aimed to determine the effectiveness of BOOST in lowering hospital readmission rates and impact on length of stay.

METHODS

The study of Project BOOST was undertaken in accordance with the SQUIRE (Standards for Quality Improvement Reporting Excellence) Guidelines.[17]

RESULTS

Eleven hospitals provided rehospitalization and length‐of‐stay outcome data for both a BOOST and control unit for the pre‐ and postimplementation periods. Compared to the 19 sites that did not participate in the analysis, these 11 sites were significantly larger (559188 beds vs 350205 beds, P=0.003), more likely to be located in an urban area (100.0% [n=11] vs 78.9% [n=15], P=0.035), and more likely to be academic medical centers (45.5% [n=5] vs 26.3% [n=5], P=0.036) (Table 1).

The mean number of tools implemented by sites participating in the analysis was 3.50.9. All sites implemented at least 2 tools. The duration between attendance at the BOOST kickoff event and first tool implementation ranged from 3 months (first tool implemented prior to attending the kickoff) and 9 months (mean duration, 3.34.3 months) (Table 2).

The average rate of 30‐day rehospitalization among BOOST units was 14.7% in the preimplementation period and 12.7% during the postimplementation period (P=0.010) (Figure 1). Rehospitalization rates for matched control units were 14.0% in the preintervention period and 14.1% in the postintervention period (P=0.831). The mean absolute reduction in readmission rates over the 1‐year study period in BOOST units compared to control units was 2.0%, or a relative reduction of 13.6% (P=0.054 for signed rank test comparing differences in readmission rate reduction in BOOST units compared to site‐matched control units). Length of stay in BOOST and control units decreased an average of 0.5 days and 0.3 days, respectively. There was no difference in length of stay change between BOOST units and control units (P=0.966).

Figure 1

DISCUSSION

As hospitals strive to reduce their readmission rates to avoid Centers for Medicare and Medicaid Services penalties, Project BOOST may be a viable QI approach to achieve their goals. This initial evaluation of participation in Project BOOST by 11 hospitals of varying sizes across the United States showed an associated reduction in rehospitalization rates (absolute=2.0% and relative=13.6%, P=0.054). We did not find any significant change in length of stay among these hospitals implementing BOOST tools.

The tools provided to participating hospitals were developed from evidence found in peer‐reviewed literature established through experimental methods in well‐controlled academic settings. Further tool development was informed by recommendations of an advisory board consisting of expert representatives and advocates involved in the hospital discharge process: patients, caregivers, physicians, nurses, case managers, social workers, insurers, and regulatory and research agencies.[19] The toolkit components address multiple aspects of hospital discharge and follow‐up with the goal of improving health by optimizing the safety of care transitions. Our observation that readmission rates appeared to improve in a diverse hospital sample including nonacademic and community hospitals engaged in Project BOOST is reassuring that the benefits seen in existing research literature, developed in distinctly academic settings, can be replicated in diverse acute‐care settings.

The effect size observed in our study was modest but consistent with several studies identified in a recent review of trials measuring interventions to reduce rehospitalization, where 7 of 16 studies showing a significant improvement registered change in the 0% to 5% absolute range.[12] Impact of this project may have been tempered by the need to translate external QI content to the local setting. Additionally, in contrast to experimental studies that are limited in scope and timing and often scaled to a research budget, BOOST sites were encouraged to implement Project BOOST in the clinical setting even if no new funds were available to support the effort.[12]

The recruitment of a national sample of both academic and nonacademic hospital participants imposed several limitations on our study and analysis. We recognize that intervention units selected by hospitals may have had unmeasured unit and patient characteristics that facilitated successful change and contributed to the observed improvements. However, because external pressure to reduce readmission is present across all hospitals independent of the BOOST intervention, we felt site‐matched controls were essential to understanding effects attributable to the BOOST tools. Differences between units would be expected to be stable over the course of the study period, and comparison of outcome differences between 2 different time periods would be reasonable. Additionally, we could not collect data on readmissions to other hospitals. Theoretically, patients discharged from BOOST units might be more likely to have been rehospitalized elsewhere, but the fraction of rehospitalizations occurring at alternate facilities would also be expected to be similar on the matched control unit.

We report findings from a voluntary cohort willing and capable of designating a comparison clinical unit and contributing the requested data outcomes. Pilot sites that did not report outcomes were not analyzed, but comparison of hospital characteristics shows that participating hospitals were more likely to be large, urban, academic medical centers. Although barriers to data submission were not formally analyzed, reports from nonparticipating sites describe data submission limited by local implementation design (no geographic rollout or simultaneous rollout on all appropriate clinical units), site specific inability to generate unit level outcome statistics, and competing organizational priorities for data analyst time (electronic medical record deployment, alternative QI initiatives). The external validity of our results may be limited to organizations capable of analytics at the level of the individual clinical unit as well as those with sufficient QI resources to support reporting to a national database in the absence of a payer mandate. It is possible that additional financial support for on‐site data collection would have bolstered participation, making the example of participation rates we present potentially informative to organizations hoping to widely disseminate a QI agenda.

Nonetheless, the effectiveness demonstrated in the 11 sites that did participate is encouraging, and ongoing collaboration with subsequent BOOST cohorts has been designed to further facilitate data collection. Among the insights gained from this pilot experience, and incorporated into ongoing BOOST cohorts, is the importance of intensive mentor engagement to foster accountability among participant sites, assist with implementation troubleshooting, and offer expertise that is often particularly effective in gaining local support. We now encourage sites to have 2 mentor site visits to further these roles and more frequent conference calls. Further research to understand the marginal benefit of the mentored implementation approach is ongoing.

The limitations in data submission we experienced with the pilot cohort likely reflect resource constraints not uncommon at many hospitals. Increasing pressure placed on hospitals as a result of the Readmission Reduction Program within the Affordable Care Act as well as increasing interest from private and Medicaid payors to incorporate similar readmission‐based penalties provide encouragement for hospitals to enhance their data and analytic skills. National incentives for implementation of electronic health records (EHR) should also foster such capabilities, though we often saw EHRs as a barrier to QI, especially rapid cycle trials. Fortunately, hospitals are increasingly being afforded access to comprehensive claims databases to assist in tracking readmission rates to other facilities, and these data are becoming available in a more timely fashion. This more robust data collection, facilitated by private payors, state QI organizations, and state hospital associations, will support additional analytic methods such as multivariate regression models and interrupted time series designs to appreciate the experience of current BOOST participants.

Additional research is needed to understand the role of organizational context in the effectiveness of Project BOOST. Differences in rates of tool implementation and changes in clinical outcomes are likely dependent on local implementation context at the level of the healthcare organization and individual clinical unit.[20] Progress reports from site mentors and previously described experiences of QI implementation indicate that successful implementation of a multidimensional bundle of interventions may have reflected a higher level of institutional support, more robust team engagement in the work of reducing readmissions, increased clinical staff support for change, the presence of an effective project champion, or a key facilitating role of external mentorship.[21, 22] Ongoing data collection will continue to measure the sustainability of tool use and observed outcome changes to inform strategies to maintain gains associated with implementation. The role of mentored implementation in facilitating gains also requires further study.

Increasing attention to the problem of avoidable rehospitalization is driving hospitals, insurers, and policy makers to pursue QI efforts that favorably impact readmission rates. Our analysis of the BOOST intervention suggests that modest gains can be achieved following evidence‐based hospital process change facilitated by a mentored implementation model. However, realization of the goal of a 20% reduction in rehospitalization proposed by the Center for Medicare and Medicaid Services' Partnership for Patients initiative may be difficult to achieve on a national scale,[23] especially if efforts focus on just the hospital.

Disclosures

Project BOOST was funded by a grant from The John A. Hartford Foundation. Project BOOST is administered by the Society of Hospital Medicine (SHM). The development of the Project BOOST toolkit, recruitment of sites for this study, mentorship of the pilot cohort, project evaluation planning, and collection of pilot data were funded by a grant from The John A. Harford Foundation. Additional funding for continued data collection and analysis was funded by the SHM through funds from hospitals to participate in Project BOOST, specifically with funding support for Dr. Hansen. Dr. Williams has received funding to serve as Principal Investigator for Project BOOST. Since the time of initial cohort participation, approximately 125 additional hospitals have participated in the mentored implementation of Project BOOST. This participation was funded through a combination of site‐based tuition, third‐party payor support from private insurers, foundations, and federal funding through the Center for Medicare and Medicaid Innovation Partnership for Patients program. Drs. Greenwald, Hansen, and Williams are Project BOOST mentors for current Project BOOST sites and receive financial support through the SHM for this work. Dr. Howell has previously received funding as a Project BOOST mentor. Ms. Budnitz is the BOOST Project Director and is Chief Strategy and Development Officer for the HM. Dr. Maynard is the Senior Vice President of the SHM's Center for Hospital Innovation and Improvement.

Enactment of federal legislation imposing hospital reimbursement penalties for excess rates of rehospitalizations among Medicare fee for service beneficiaries markedly increased interest in hospital quality improvement (QI) efforts to reduce the observed 30‐day rehospitalization of 19.6% in this elderly population.[1, 2] The Congressional Budget Office estimated that reimbursement penalties to hospitals for high readmission rates are expected to save the Medicare program approximately $7 billion between 2010 and 2019.[3] These penalties are complemented by resources from the Center for Medicare and Medicaid Innovation aiming to reduce hospital readmissions by 20% by the end of 2013 through the Partnership for Patients campaign.[4] Although potential financial penalties and provision of resources for QI intensified efforts to enhance the quality of the hospital discharge transition, patient safety risks associated with hospital discharge are well documented.[5, 6] Approximately 20% of patients discharged from the hospital may suffer adverse events,[7, 8] of which up to three‐quarters (72%) are medication related,[9] and over one‐third of required follow‐up testing after discharge is not completed.[10] Such findings indicate opportunities for improvement in the discharge process.[11]

Numerous publications describe studies aiming to improve the hospital discharge process and mitigate these hazards, though a systematic review of interventions to reduce 30‐day rehospitalization indicated that the existing evidence base for the effectiveness of transition interventions demonstrates irregular effectiveness and limitations to generalizability.[12] Most studies showing effectiveness are confined to single academic medical centers. Existing evidence supports multifaceted interventions implemented in both the pre‐ and postdischarge periods and focused on risk assessment and tailored, patient‐centered application of interventions to mitigate risk. For example Project RED (Re‐Engineered Discharge) applied a bundled intervention consisting of intensified patient education and discharge planning, improved medication reconciliation and discharge instructions, and longitudinal patient contact with follow‐up phone calls and a dedicated discharge advocate.[13] However, the mean age of patients participating in the study was 50 years, and it excluded patients admitted from or discharged to skilled nursing facilities, making generalizability to the geriatric population uncertain.

An integral aspect of QI projects is the contribution of local context to translation of best practices to disparate settings.[14, 15, 16] Most available reports of successful interventions to reduce rehospitalization have not fully described the specifics of either the intervention context or design. Moreover, the available evidence base for common interventions to reduce rehospitalization was developed in the academic setting. Validation of single academic center studies in a broader healthcare context is necessary.

Project BOOST (Better Outcomes for Older adults through Safe Transitions) recruited a diverse national cohort of both academic and nonacademic hospitals to participate in a QI effort to implement best practices for hospital discharge care transitions using a national collaborative approach facilitated by external expert mentorship. This study aimed to determine the effectiveness of BOOST in lowering hospital readmission rates and impact on length of stay.

METHODS

The study of Project BOOST was undertaken in accordance with the SQUIRE (Standards for Quality Improvement Reporting Excellence) Guidelines.[17]

RESULTS

Eleven hospitals provided rehospitalization and length‐of‐stay outcome data for both a BOOST and control unit for the pre‐ and postimplementation periods. Compared to the 19 sites that did not participate in the analysis, these 11 sites were significantly larger (559188 beds vs 350205 beds, P=0.003), more likely to be located in an urban area (100.0% [n=11] vs 78.9% [n=15], P=0.035), and more likely to be academic medical centers (45.5% [n=5] vs 26.3% [n=5], P=0.036) (Table 1).

The mean number of tools implemented by sites participating in the analysis was 3.50.9. All sites implemented at least 2 tools. The duration between attendance at the BOOST kickoff event and first tool implementation ranged from 3 months (first tool implemented prior to attending the kickoff) and 9 months (mean duration, 3.34.3 months) (Table 2).

The average rate of 30‐day rehospitalization among BOOST units was 14.7% in the preimplementation period and 12.7% during the postimplementation period (P=0.010) (Figure 1). Rehospitalization rates for matched control units were 14.0% in the preintervention period and 14.1% in the postintervention period (P=0.831). The mean absolute reduction in readmission rates over the 1‐year study period in BOOST units compared to control units was 2.0%, or a relative reduction of 13.6% (P=0.054 for signed rank test comparing differences in readmission rate reduction in BOOST units compared to site‐matched control units). Length of stay in BOOST and control units decreased an average of 0.5 days and 0.3 days, respectively. There was no difference in length of stay change between BOOST units and control units (P=0.966).

Figure 1

DISCUSSION

As hospitals strive to reduce their readmission rates to avoid Centers for Medicare and Medicaid Services penalties, Project BOOST may be a viable QI approach to achieve their goals. This initial evaluation of participation in Project BOOST by 11 hospitals of varying sizes across the United States showed an associated reduction in rehospitalization rates (absolute=2.0% and relative=13.6%, P=0.054). We did not find any significant change in length of stay among these hospitals implementing BOOST tools.

The tools provided to participating hospitals were developed from evidence found in peer‐reviewed literature established through experimental methods in well‐controlled academic settings. Further tool development was informed by recommendations of an advisory board consisting of expert representatives and advocates involved in the hospital discharge process: patients, caregivers, physicians, nurses, case managers, social workers, insurers, and regulatory and research agencies.[19] The toolkit components address multiple aspects of hospital discharge and follow‐up with the goal of improving health by optimizing the safety of care transitions. Our observation that readmission rates appeared to improve in a diverse hospital sample including nonacademic and community hospitals engaged in Project BOOST is reassuring that the benefits seen in existing research literature, developed in distinctly academic settings, can be replicated in diverse acute‐care settings.

The effect size observed in our study was modest but consistent with several studies identified in a recent review of trials measuring interventions to reduce rehospitalization, where 7 of 16 studies showing a significant improvement registered change in the 0% to 5% absolute range.[12] Impact of this project may have been tempered by the need to translate external QI content to the local setting. Additionally, in contrast to experimental studies that are limited in scope and timing and often scaled to a research budget, BOOST sites were encouraged to implement Project BOOST in the clinical setting even if no new funds were available to support the effort.[12]

The recruitment of a national sample of both academic and nonacademic hospital participants imposed several limitations on our study and analysis. We recognize that intervention units selected by hospitals may have had unmeasured unit and patient characteristics that facilitated successful change and contributed to the observed improvements. However, because external pressure to reduce readmission is present across all hospitals independent of the BOOST intervention, we felt site‐matched controls were essential to understanding effects attributable to the BOOST tools. Differences between units would be expected to be stable over the course of the study period, and comparison of outcome differences between 2 different time periods would be reasonable. Additionally, we could not collect data on readmissions to other hospitals. Theoretically, patients discharged from BOOST units might be more likely to have been rehospitalized elsewhere, but the fraction of rehospitalizations occurring at alternate facilities would also be expected to be similar on the matched control unit.

We report findings from a voluntary cohort willing and capable of designating a comparison clinical unit and contributing the requested data outcomes. Pilot sites that did not report outcomes were not analyzed, but comparison of hospital characteristics shows that participating hospitals were more likely to be large, urban, academic medical centers. Although barriers to data submission were not formally analyzed, reports from nonparticipating sites describe data submission limited by local implementation design (no geographic rollout or simultaneous rollout on all appropriate clinical units), site specific inability to generate unit level outcome statistics, and competing organizational priorities for data analyst time (electronic medical record deployment, alternative QI initiatives). The external validity of our results may be limited to organizations capable of analytics at the level of the individual clinical unit as well as those with sufficient QI resources to support reporting to a national database in the absence of a payer mandate. It is possible that additional financial support for on‐site data collection would have bolstered participation, making the example of participation rates we present potentially informative to organizations hoping to widely disseminate a QI agenda.

Nonetheless, the effectiveness demonstrated in the 11 sites that did participate is encouraging, and ongoing collaboration with subsequent BOOST cohorts has been designed to further facilitate data collection. Among the insights gained from this pilot experience, and incorporated into ongoing BOOST cohorts, is the importance of intensive mentor engagement to foster accountability among participant sites, assist with implementation troubleshooting, and offer expertise that is often particularly effective in gaining local support. We now encourage sites to have 2 mentor site visits to further these roles and more frequent conference calls. Further research to understand the marginal benefit of the mentored implementation approach is ongoing.

The limitations in data submission we experienced with the pilot cohort likely reflect resource constraints not uncommon at many hospitals. Increasing pressure placed on hospitals as a result of the Readmission Reduction Program within the Affordable Care Act as well as increasing interest from private and Medicaid payors to incorporate similar readmission‐based penalties provide encouragement for hospitals to enhance their data and analytic skills. National incentives for implementation of electronic health records (EHR) should also foster such capabilities, though we often saw EHRs as a barrier to QI, especially rapid cycle trials. Fortunately, hospitals are increasingly being afforded access to comprehensive claims databases to assist in tracking readmission rates to other facilities, and these data are becoming available in a more timely fashion. This more robust data collection, facilitated by private payors, state QI organizations, and state hospital associations, will support additional analytic methods such as multivariate regression models and interrupted time series designs to appreciate the experience of current BOOST participants.

Additional research is needed to understand the role of organizational context in the effectiveness of Project BOOST. Differences in rates of tool implementation and changes in clinical outcomes are likely dependent on local implementation context at the level of the healthcare organization and individual clinical unit.[20] Progress reports from site mentors and previously described experiences of QI implementation indicate that successful implementation of a multidimensional bundle of interventions may have reflected a higher level of institutional support, more robust team engagement in the work of reducing readmissions, increased clinical staff support for change, the presence of an effective project champion, or a key facilitating role of external mentorship.[21, 22] Ongoing data collection will continue to measure the sustainability of tool use and observed outcome changes to inform strategies to maintain gains associated with implementation. The role of mentored implementation in facilitating gains also requires further study.

Increasing attention to the problem of avoidable rehospitalization is driving hospitals, insurers, and policy makers to pursue QI efforts that favorably impact readmission rates. Our analysis of the BOOST intervention suggests that modest gains can be achieved following evidence‐based hospital process change facilitated by a mentored implementation model. However, realization of the goal of a 20% reduction in rehospitalization proposed by the Center for Medicare and Medicaid Services' Partnership for Patients initiative may be difficult to achieve on a national scale,[23] especially if efforts focus on just the hospital.

Disclosures

Project BOOST was funded by a grant from The John A. Hartford Foundation. Project BOOST is administered by the Society of Hospital Medicine (SHM). The development of the Project BOOST toolkit, recruitment of sites for this study, mentorship of the pilot cohort, project evaluation planning, and collection of pilot data were funded by a grant from The John A. Harford Foundation. Additional funding for continued data collection and analysis was funded by the SHM through funds from hospitals to participate in Project BOOST, specifically with funding support for Dr. Hansen. Dr. Williams has received funding to serve as Principal Investigator for Project BOOST. Since the time of initial cohort participation, approximately 125 additional hospitals have participated in the mentored implementation of Project BOOST. This participation was funded through a combination of site‐based tuition, third‐party payor support from private insurers, foundations, and federal funding through the Center for Medicare and Medicaid Innovation Partnership for Patients program. Drs. Greenwald, Hansen, and Williams are Project BOOST mentors for current Project BOOST sites and receive financial support through the SHM for this work. Dr. Howell has previously received funding as a Project BOOST mentor. Ms. Budnitz is the BOOST Project Director and is Chief Strategy and Development Officer for the HM. Dr. Maynard is the Senior Vice President of the SHM's Center for Hospital Innovation and Improvement.

Enactment of federal legislation imposing hospital reimbursement penalties for excess rates of rehospitalizations among Medicare fee for service beneficiaries markedly increased interest in hospital quality improvement (QI) efforts to reduce the observed 30‐day rehospitalization of 19.6% in this elderly population.[1, 2] The Congressional Budget Office estimated that reimbursement penalties to hospitals for high readmission rates are expected to save the Medicare program approximately $7 billion between 2010 and 2019.[3] These penalties are complemented by resources from the Center for Medicare and Medicaid Innovation aiming to reduce hospital readmissions by 20% by the end of 2013 through the Partnership for Patients campaign.[4] Although potential financial penalties and provision of resources for QI intensified efforts to enhance the quality of the hospital discharge transition, patient safety risks associated with hospital discharge are well documented.[5, 6] Approximately 20% of patients discharged from the hospital may suffer adverse events,[7, 8] of which up to three‐quarters (72%) are medication related,[9] and over one‐third of required follow‐up testing after discharge is not completed.[10] Such findings indicate opportunities for improvement in the discharge process.[11]

Numerous publications describe studies aiming to improve the hospital discharge process and mitigate these hazards, though a systematic review of interventions to reduce 30‐day rehospitalization indicated that the existing evidence base for the effectiveness of transition interventions demonstrates irregular effectiveness and limitations to generalizability.[12] Most studies showing effectiveness are confined to single academic medical centers. Existing evidence supports multifaceted interventions implemented in both the pre‐ and postdischarge periods and focused on risk assessment and tailored, patient‐centered application of interventions to mitigate risk. For example Project RED (Re‐Engineered Discharge) applied a bundled intervention consisting of intensified patient education and discharge planning, improved medication reconciliation and discharge instructions, and longitudinal patient contact with follow‐up phone calls and a dedicated discharge advocate.[13] However, the mean age of patients participating in the study was 50 years, and it excluded patients admitted from or discharged to skilled nursing facilities, making generalizability to the geriatric population uncertain.

An integral aspect of QI projects is the contribution of local context to translation of best practices to disparate settings.[14, 15, 16] Most available reports of successful interventions to reduce rehospitalization have not fully described the specifics of either the intervention context or design. Moreover, the available evidence base for common interventions to reduce rehospitalization was developed in the academic setting. Validation of single academic center studies in a broader healthcare context is necessary.

Project BOOST (Better Outcomes for Older adults through Safe Transitions) recruited a diverse national cohort of both academic and nonacademic hospitals to participate in a QI effort to implement best practices for hospital discharge care transitions using a national collaborative approach facilitated by external expert mentorship. This study aimed to determine the effectiveness of BOOST in lowering hospital readmission rates and impact on length of stay.

METHODS

The study of Project BOOST was undertaken in accordance with the SQUIRE (Standards for Quality Improvement Reporting Excellence) Guidelines.[17]

RESULTS

Eleven hospitals provided rehospitalization and length‐of‐stay outcome data for both a BOOST and control unit for the pre‐ and postimplementation periods. Compared to the 19 sites that did not participate in the analysis, these 11 sites were significantly larger (559188 beds vs 350205 beds, P=0.003), more likely to be located in an urban area (100.0% [n=11] vs 78.9% [n=15], P=0.035), and more likely to be academic medical centers (45.5% [n=5] vs 26.3% [n=5], P=0.036) (Table 1).

The mean number of tools implemented by sites participating in the analysis was 3.50.9. All sites implemented at least 2 tools. The duration between attendance at the BOOST kickoff event and first tool implementation ranged from 3 months (first tool implemented prior to attending the kickoff) and 9 months (mean duration, 3.34.3 months) (Table 2).

The average rate of 30‐day rehospitalization among BOOST units was 14.7% in the preimplementation period and 12.7% during the postimplementation period (P=0.010) (Figure 1). Rehospitalization rates for matched control units were 14.0% in the preintervention period and 14.1% in the postintervention period (P=0.831). The mean absolute reduction in readmission rates over the 1‐year study period in BOOST units compared to control units was 2.0%, or a relative reduction of 13.6% (P=0.054 for signed rank test comparing differences in readmission rate reduction in BOOST units compared to site‐matched control units). Length of stay in BOOST and control units decreased an average of 0.5 days and 0.3 days, respectively. There was no difference in length of stay change between BOOST units and control units (P=0.966).

Figure 1

DISCUSSION

As hospitals strive to reduce their readmission rates to avoid Centers for Medicare and Medicaid Services penalties, Project BOOST may be a viable QI approach to achieve their goals. This initial evaluation of participation in Project BOOST by 11 hospitals of varying sizes across the United States showed an associated reduction in rehospitalization rates (absolute=2.0% and relative=13.6%, P=0.054). We did not find any significant change in length of stay among these hospitals implementing BOOST tools.

The tools provided to participating hospitals were developed from evidence found in peer‐reviewed literature established through experimental methods in well‐controlled academic settings. Further tool development was informed by recommendations of an advisory board consisting of expert representatives and advocates involved in the hospital discharge process: patients, caregivers, physicians, nurses, case managers, social workers, insurers, and regulatory and research agencies.[19] The toolkit components address multiple aspects of hospital discharge and follow‐up with the goal of improving health by optimizing the safety of care transitions. Our observation that readmission rates appeared to improve in a diverse hospital sample including nonacademic and community hospitals engaged in Project BOOST is reassuring that the benefits seen in existing research literature, developed in distinctly academic settings, can be replicated in diverse acute‐care settings.

The effect size observed in our study was modest but consistent with several studies identified in a recent review of trials measuring interventions to reduce rehospitalization, where 7 of 16 studies showing a significant improvement registered change in the 0% to 5% absolute range.[12] Impact of this project may have been tempered by the need to translate external QI content to the local setting. Additionally, in contrast to experimental studies that are limited in scope and timing and often scaled to a research budget, BOOST sites were encouraged to implement Project BOOST in the clinical setting even if no new funds were available to support the effort.[12]

The recruitment of a national sample of both academic and nonacademic hospital participants imposed several limitations on our study and analysis. We recognize that intervention units selected by hospitals may have had unmeasured unit and patient characteristics that facilitated successful change and contributed to the observed improvements. However, because external pressure to reduce readmission is present across all hospitals independent of the BOOST intervention, we felt site‐matched controls were essential to understanding effects attributable to the BOOST tools. Differences between units would be expected to be stable over the course of the study period, and comparison of outcome differences between 2 different time periods would be reasonable. Additionally, we could not collect data on readmissions to other hospitals. Theoretically, patients discharged from BOOST units might be more likely to have been rehospitalized elsewhere, but the fraction of rehospitalizations occurring at alternate facilities would also be expected to be similar on the matched control unit.

We report findings from a voluntary cohort willing and capable of designating a comparison clinical unit and contributing the requested data outcomes. Pilot sites that did not report outcomes were not analyzed, but comparison of hospital characteristics shows that participating hospitals were more likely to be large, urban, academic medical centers. Although barriers to data submission were not formally analyzed, reports from nonparticipating sites describe data submission limited by local implementation design (no geographic rollout or simultaneous rollout on all appropriate clinical units), site specific inability to generate unit level outcome statistics, and competing organizational priorities for data analyst time (electronic medical record deployment, alternative QI initiatives). The external validity of our results may be limited to organizations capable of analytics at the level of the individual clinical unit as well as those with sufficient QI resources to support reporting to a national database in the absence of a payer mandate. It is possible that additional financial support for on‐site data collection would have bolstered participation, making the example of participation rates we present potentially informative to organizations hoping to widely disseminate a QI agenda.

Nonetheless, the effectiveness demonstrated in the 11 sites that did participate is encouraging, and ongoing collaboration with subsequent BOOST cohorts has been designed to further facilitate data collection. Among the insights gained from this pilot experience, and incorporated into ongoing BOOST cohorts, is the importance of intensive mentor engagement to foster accountability among participant sites, assist with implementation troubleshooting, and offer expertise that is often particularly effective in gaining local support. We now encourage sites to have 2 mentor site visits to further these roles and more frequent conference calls. Further research to understand the marginal benefit of the mentored implementation approach is ongoing.

The limitations in data submission we experienced with the pilot cohort likely reflect resource constraints not uncommon at many hospitals. Increasing pressure placed on hospitals as a result of the Readmission Reduction Program within the Affordable Care Act as well as increasing interest from private and Medicaid payors to incorporate similar readmission‐based penalties provide encouragement for hospitals to enhance their data and analytic skills. National incentives for implementation of electronic health records (EHR) should also foster such capabilities, though we often saw EHRs as a barrier to QI, especially rapid cycle trials. Fortunately, hospitals are increasingly being afforded access to comprehensive claims databases to assist in tracking readmission rates to other facilities, and these data are becoming available in a more timely fashion. This more robust data collection, facilitated by private payors, state QI organizations, and state hospital associations, will support additional analytic methods such as multivariate regression models and interrupted time series designs to appreciate the experience of current BOOST participants.

Additional research is needed to understand the role of organizational context in the effectiveness of Project BOOST. Differences in rates of tool implementation and changes in clinical outcomes are likely dependent on local implementation context at the level of the healthcare organization and individual clinical unit.[20] Progress reports from site mentors and previously described experiences of QI implementation indicate that successful implementation of a multidimensional bundle of interventions may have reflected a higher level of institutional support, more robust team engagement in the work of reducing readmissions, increased clinical staff support for change, the presence of an effective project champion, or a key facilitating role of external mentorship.[21, 22] Ongoing data collection will continue to measure the sustainability of tool use and observed outcome changes to inform strategies to maintain gains associated with implementation. The role of mentored implementation in facilitating gains also requires further study.

Increasing attention to the problem of avoidable rehospitalization is driving hospitals, insurers, and policy makers to pursue QI efforts that favorably impact readmission rates. Our analysis of the BOOST intervention suggests that modest gains can be achieved following evidence‐based hospital process change facilitated by a mentored implementation model. However, realization of the goal of a 20% reduction in rehospitalization proposed by the Center for Medicare and Medicaid Services' Partnership for Patients initiative may be difficult to achieve on a national scale,[23] especially if efforts focus on just the hospital.

Disclosures

Project BOOST was funded by a grant from The John A. Hartford Foundation. Project BOOST is administered by the Society of Hospital Medicine (SHM). The development of the Project BOOST toolkit, recruitment of sites for this study, mentorship of the pilot cohort, project evaluation planning, and collection of pilot data were funded by a grant from The John A. Harford Foundation. Additional funding for continued data collection and analysis was funded by the SHM through funds from hospitals to participate in Project BOOST, specifically with funding support for Dr. Hansen. Dr. Williams has received funding to serve as Principal Investigator for Project BOOST. Since the time of initial cohort participation, approximately 125 additional hospitals have participated in the mentored implementation of Project BOOST. This participation was funded through a combination of site‐based tuition, third‐party payor support from private insurers, foundations, and federal funding through the Center for Medicare and Medicaid Innovation Partnership for Patients program. Drs. Greenwald, Hansen, and Williams are Project BOOST mentors for current Project BOOST sites and receive financial support through the SHM for this work. Dr. Howell has previously received funding as a Project BOOST mentor. Ms. Budnitz is the BOOST Project Director and is Chief Strategy and Development Officer for the HM. Dr. Maynard is the Senior Vice President of the SHM's Center for Hospital Innovation and Improvement.

Data collection, analysis, and presentation are key to the success of any hospital glycemic control initiative. Such efforts enable the management team to track improvements in processes and outcomes, make necessary changes to their quality improvement efforts, justify the provision of necessary time and resources, and share their results with others. Reliable metrics for assessing glycemic control and frequency of hypoglycemia are essential to accomplish these tasks and to assess whether interventions result in more benefit than harm. Hypoglycemia metrics must be especially convincing because fear of hypoglycemia remains a major source of clinical inertia, impeding efforts to improve glucose control.

Currently, there are no official standards or guidelines for formulating metrics on the quality of inpatient glycemic control. This creates several problems. First, different metrics vary in their biases and in their responsiveness to change. Thus, use of a poor metric could lead to either a falsely positive or falsely negative impression that a quality improvement intervention is in fact improving glycemic control. Second, the proliferation of different measures and analytical plans in the research and quality improvement literature make it very difficult for hospitals to compare baseline performance, determine need for improvement, and understand which interventions may be most effective.

A related article in this supplement provides the rationale for improved inpatient glycemic control. That article argues that the current state of inpatient glycemic control, with the frequent occurrence of severe hyperglycemia and irrational insulin ordering, cannot be considered acceptable, especially given the large body of data (albeit largely observational) linking hyperglycemia to negative patient outcomes. However, regardless of whether one is an advocate or skeptic of tighter glucose control in the intensive care unit (ICU) and especially the non‐ICU setting, there is no question that standardized, valid, and reliable metrics are needed to compare efforts to improve glycemic control, better understand whether such control actually improves patient care, and closely monitor patient safety.

This article provides a summary of practical suggestions to assess glycemic control, insulin use patterns, and safety (hypoglycemia and severe hyperglycemia). In particular, we discuss the pros and cons of various measurement choices. We conclude with a tiered summary of recommendations for practical metrics that we hope will be useful to individual improvement teams. This article is not a consensus statement but rather a starting place that we hope will begin to standardize measurement across institutions and advance the dialogue on this subject. To more definitely address this problem, we call on the American Association of Clinical Endocrinologists (AACE), American Diabetes Association (ADA), Society of Hospital Medicine (SHM), and others to agree on consensus standards regarding metrics for the quality of inpatient glycemic control.

MEASURING GLYCEMIC CONTROL: GLUCOMETRICS

Glucometrics may be defined as the systematic analysis of blood glucose (BG) dataa phrase initially coined specifically for the inpatient setting. There are numerous ways to do these analyses, depending on which patients and glucose values are considered, the definitions used for hypoglycemia and hyperglycemia, the unit of measurement (eg, patient, patient‐day, individual glucose value), and the measure of control (eg, mean, median, percent of glucose readings within a certain range). We consider each of these dimensions in turn.

APPROACHES TO ANALYSIS: RUN CHARTS

Most conventional clinical trials hold interventions fixed for a period of time and compare results with and without the intervention. For quality improvement studies, this is still a valid way to proceed, especially if studied as a randomized controlled trial. Such methods may be preferred when the clinical question is Does this type of intervention work in general? and the desired output is publication in peer‐reviewed journals so that others can learn about and adopt the intervention to their own institution. A before and after study with a similar analytic approach may also be valid, although concerns about temporal trends and cointerventions potentially compromise the validity of such studies. This approach again assumes that an intervention is held fixed over time such that it is clear what patients received during each time period.

If the desired result is improvement at a given institution (the question is Did we improve care?) then it may be preferable to present results over time using run‐charts. In a run chart, the x‐axis is time and the y‐axis the desired metric, such as patient‐day weighted mean glucose. Points in time when interventions were introduced or modified can be highlighted. Run charts have several advantages over before‐and‐after summaries: they do not require interventions remaining fixed and are more compatible with continuous quality improvement methods, it is easier to see the effect of different aspects of the interventions as they occur, one can get a quicker picture of whether something is working, and it is easier to separate out the impact of the intervention from secular trends. Finally, the use of run charts does not imply the absence of statistical rigor. Run charts with statistical process control (SPC) limits5 can easily convey when the observed time trend is unlikely to be due to chance using prespecified P values. (A full discussion of SPC and other methods to study quality improvement interventions is beyond the scope of this article.)

ASSESSING PATTERNS OF INSULIN USE AND ORDER SET UTILIZATION

Besides measuring the impact of quality improvement interventions on glucose control, it is important to measure processes such as proper insulin use. As mentioned in other articles in this supplement, processes are much more sensitive to change than outcomes. Failure to change processes should lead one to make changes to the intervention.

OTHER CONSIDERATIONS AND METRICS

Examples of other metrics that can be used to track the success of quality improvement efforts include:

• 1

  Glucose measurement within 8 hours of hospital admission.

• 2

  Glycated hemoglobin (A1C) measurement obtained or available within 30 days of admission to help guide inpatient and especially discharge management.

• 3

  Appropriate glucose testing in patients with diabetes or hyperglycemia (eg, 4 times per day in patients not on insulin infusion protocols, at least until 24 hours of euglycemia is documented).

• 4

  The percentage of patients on insulin with on‐time tray delivery.

• 5

  The timing of subcutaneous insulin administration in relation to glucose testing and nutrition delivery.

• 6

  Documentation of carbohydrate intake among patients who are eating.

• 7

  Satisfaction of physicians and nurses with order sets or protocols, using standard surveys.

• 8

  Physician and nurse knowledge, attitudes, and beliefs about insulin administration, fear of hypoglycemia, treatment of hypoglycemia, and glycemic control in the hospital.

• 9

  Patient satisfaction with their diabetes care in the hospital, including the education they received.

• 10

  Nursing and physician education/certification in insulin prescribing, insulin administration, and other diabetes care issues.

• 11

  Patient outcomes strongly associated with glycemic control, (eg, surgical wound infections, ICU LOS, catheter‐related bloodstream infections).

• 12

  Appropriate treatment and documentation of hypoglycemia (eg, in accordance with hospital policy).

• 13

  Documentation of severe hypoglycemic events through the hospital's adverse events reporting system (these may actually increase as change comes to the organization and as clinical personnel are more attuned to glycemic control).

• 14

  Root causes of hypoglycemic events, which can be used to understand and prevent future events.

• 15

  Appropriate transitions from IV to SC insulin regimens, (eg, starting basal insulin prior to discontinuing infusion in patients who have been on an insulin infusion of at least 2 units/hour or who have a known diagnosis of diabetes or A1C >7).

(Survey instruments and other measurement tools are available from the authors upon request.)

SHM GLYCEMIC CONTROL TASK FORCE SUMMARY RECOMMENDATIONS

The SHM Glycemic Control Task Force is working to develop standardized measures of inpatient glucose control and related indicators to track progress of hospital glycemic control initiatives (see the introduction to this supplement for a description of the charge and membership of this task force). The goals of the Task Force's metrics recommendations (Table 1) are several‐fold: (1) create a set of measurements that are complete but not overly burdensome; (2) create realistic measures that can be applied to institutions with different data management capabilities; and (3) allow for comparison across institutions for benchmarking purposes, evaluation of quality improvement projects, and reporting of results for formal research studies in this field.

For each domain of glycemic management (glycemic control, safety, and insulin use), the task force chose a set of best measures. They are presented as two tiers of measurement standards, depending on the capabilities of the institution and the planned uses of the data. Tier 1 includes measures that, although they do take time and resources to collect, are feasible for most institutions. Tier 2 measures are recommended for hospitals with easy manipulation of electronic sources of data and for reporting quality‐of‐care measures for widespread publication, that is, in the context of a research study. It should be emphasized that these recommendations are only meant as a guide: the actual measures chosen should meet the needs and capabilities of each institution.

We recognize that few data support the recommendations made by this task force, that such data are needed, and that the field of data collection and analysis for hospital glycemic management is rapidly evolving. The hope is to begin the standardization process, promote dialogue in this field, and eventually reach a consensus in collaboration with the ADA, AACE, and other pertinent stakeholders.

CONCLUSIONS

Like the field of inpatient glycemic management itself, the field of devising metrics to measure the quality of inpatient glycemic control is also in its infancy and quickly evolving. One should not be paralyzed by the lack of consensus regarding measurementthe important point is to pick a few complementary metrics and begin the process. The table of recommendations can hopefully serve as a starting point for many institutions, with a focus on efficacy (glycemic control), safety (hypoglycemia), and process (insulin use patterns). As your institution gains experience with measurement and the field evolves, your metrics will likely change. We recommend keeping all process and outcome data in its raw form so that it can be summarized in different ways over time. It is also important not to wait for the perfect data collection tool before beginning to analyze data: sampling and paper processes are acceptable if automated data collection is not yet possible. Eventually, blood glucose meter readings should be downloaded into a central database that interfaces with hospital data repositories so data can be analyzed in conjunction with patient, service, and unit‐level information. Only with a rigorous measurement process can institutions hope to know whether their changes are resulting in improved care for patients.

Data collection, analysis, and presentation are key to the success of any hospital glycemic control initiative. Such efforts enable the management team to track improvements in processes and outcomes, make necessary changes to their quality improvement efforts, justify the provision of necessary time and resources, and share their results with others. Reliable metrics for assessing glycemic control and frequency of hypoglycemia are essential to accomplish these tasks and to assess whether interventions result in more benefit than harm. Hypoglycemia metrics must be especially convincing because fear of hypoglycemia remains a major source of clinical inertia, impeding efforts to improve glucose control.

Currently, there are no official standards or guidelines for formulating metrics on the quality of inpatient glycemic control. This creates several problems. First, different metrics vary in their biases and in their responsiveness to change. Thus, use of a poor metric could lead to either a falsely positive or falsely negative impression that a quality improvement intervention is in fact improving glycemic control. Second, the proliferation of different measures and analytical plans in the research and quality improvement literature make it very difficult for hospitals to compare baseline performance, determine need for improvement, and understand which interventions may be most effective.

A related article in this supplement provides the rationale for improved inpatient glycemic control. That article argues that the current state of inpatient glycemic control, with the frequent occurrence of severe hyperglycemia and irrational insulin ordering, cannot be considered acceptable, especially given the large body of data (albeit largely observational) linking hyperglycemia to negative patient outcomes. However, regardless of whether one is an advocate or skeptic of tighter glucose control in the intensive care unit (ICU) and especially the non‐ICU setting, there is no question that standardized, valid, and reliable metrics are needed to compare efforts to improve glycemic control, better understand whether such control actually improves patient care, and closely monitor patient safety.

This article provides a summary of practical suggestions to assess glycemic control, insulin use patterns, and safety (hypoglycemia and severe hyperglycemia). In particular, we discuss the pros and cons of various measurement choices. We conclude with a tiered summary of recommendations for practical metrics that we hope will be useful to individual improvement teams. This article is not a consensus statement but rather a starting place that we hope will begin to standardize measurement across institutions and advance the dialogue on this subject. To more definitely address this problem, we call on the American Association of Clinical Endocrinologists (AACE), American Diabetes Association (ADA), Society of Hospital Medicine (SHM), and others to agree on consensus standards regarding metrics for the quality of inpatient glycemic control.

MEASURING GLYCEMIC CONTROL: GLUCOMETRICS

Glucometrics may be defined as the systematic analysis of blood glucose (BG) dataa phrase initially coined specifically for the inpatient setting. There are numerous ways to do these analyses, depending on which patients and glucose values are considered, the definitions used for hypoglycemia and hyperglycemia, the unit of measurement (eg, patient, patient‐day, individual glucose value), and the measure of control (eg, mean, median, percent of glucose readings within a certain range). We consider each of these dimensions in turn.

APPROACHES TO ANALYSIS: RUN CHARTS

Most conventional clinical trials hold interventions fixed for a period of time and compare results with and without the intervention. For quality improvement studies, this is still a valid way to proceed, especially if studied as a randomized controlled trial. Such methods may be preferred when the clinical question is Does this type of intervention work in general? and the desired output is publication in peer‐reviewed journals so that others can learn about and adopt the intervention to their own institution. A before and after study with a similar analytic approach may also be valid, although concerns about temporal trends and cointerventions potentially compromise the validity of such studies. This approach again assumes that an intervention is held fixed over time such that it is clear what patients received during each time period.

If the desired result is improvement at a given institution (the question is Did we improve care?) then it may be preferable to present results over time using run‐charts. In a run chart, the x‐axis is time and the y‐axis the desired metric, such as patient‐day weighted mean glucose. Points in time when interventions were introduced or modified can be highlighted. Run charts have several advantages over before‐and‐after summaries: they do not require interventions remaining fixed and are more compatible with continuous quality improvement methods, it is easier to see the effect of different aspects of the interventions as they occur, one can get a quicker picture of whether something is working, and it is easier to separate out the impact of the intervention from secular trends. Finally, the use of run charts does not imply the absence of statistical rigor. Run charts with statistical process control (SPC) limits5 can easily convey when the observed time trend is unlikely to be due to chance using prespecified P values. (A full discussion of SPC and other methods to study quality improvement interventions is beyond the scope of this article.)

ASSESSING PATTERNS OF INSULIN USE AND ORDER SET UTILIZATION

Besides measuring the impact of quality improvement interventions on glucose control, it is important to measure processes such as proper insulin use. As mentioned in other articles in this supplement, processes are much more sensitive to change than outcomes. Failure to change processes should lead one to make changes to the intervention.

OTHER CONSIDERATIONS AND METRICS

Examples of other metrics that can be used to track the success of quality improvement efforts include:

• 1

  Glucose measurement within 8 hours of hospital admission.

• 2

  Glycated hemoglobin (A1C) measurement obtained or available within 30 days of admission to help guide inpatient and especially discharge management.

• 3

  Appropriate glucose testing in patients with diabetes or hyperglycemia (eg, 4 times per day in patients not on insulin infusion protocols, at least until 24 hours of euglycemia is documented).

• 4

  The percentage of patients on insulin with on‐time tray delivery.

• 5

  The timing of subcutaneous insulin administration in relation to glucose testing and nutrition delivery.

• 6

  Documentation of carbohydrate intake among patients who are eating.

• 7

  Satisfaction of physicians and nurses with order sets or protocols, using standard surveys.

• 8

  Physician and nurse knowledge, attitudes, and beliefs about insulin administration, fear of hypoglycemia, treatment of hypoglycemia, and glycemic control in the hospital.

• 9

  Patient satisfaction with their diabetes care in the hospital, including the education they received.

• 10

  Nursing and physician education/certification in insulin prescribing, insulin administration, and other diabetes care issues.

• 11

  Patient outcomes strongly associated with glycemic control, (eg, surgical wound infections, ICU LOS, catheter‐related bloodstream infections).

• 12

  Appropriate treatment and documentation of hypoglycemia (eg, in accordance with hospital policy).

• 13

  Documentation of severe hypoglycemic events through the hospital's adverse events reporting system (these may actually increase as change comes to the organization and as clinical personnel are more attuned to glycemic control).

• 14

  Root causes of hypoglycemic events, which can be used to understand and prevent future events.

• 15

  Appropriate transitions from IV to SC insulin regimens, (eg, starting basal insulin prior to discontinuing infusion in patients who have been on an insulin infusion of at least 2 units/hour or who have a known diagnosis of diabetes or A1C >7).

(Survey instruments and other measurement tools are available from the authors upon request.)

SHM GLYCEMIC CONTROL TASK FORCE SUMMARY RECOMMENDATIONS

The SHM Glycemic Control Task Force is working to develop standardized measures of inpatient glucose control and related indicators to track progress of hospital glycemic control initiatives (see the introduction to this supplement for a description of the charge and membership of this task force). The goals of the Task Force's metrics recommendations (Table 1) are several‐fold: (1) create a set of measurements that are complete but not overly burdensome; (2) create realistic measures that can be applied to institutions with different data management capabilities; and (3) allow for comparison across institutions for benchmarking purposes, evaluation of quality improvement projects, and reporting of results for formal research studies in this field.

For each domain of glycemic management (glycemic control, safety, and insulin use), the task force chose a set of best measures. They are presented as two tiers of measurement standards, depending on the capabilities of the institution and the planned uses of the data. Tier 1 includes measures that, although they do take time and resources to collect, are feasible for most institutions. Tier 2 measures are recommended for hospitals with easy manipulation of electronic sources of data and for reporting quality‐of‐care measures for widespread publication, that is, in the context of a research study. It should be emphasized that these recommendations are only meant as a guide: the actual measures chosen should meet the needs and capabilities of each institution.

We recognize that few data support the recommendations made by this task force, that such data are needed, and that the field of data collection and analysis for hospital glycemic management is rapidly evolving. The hope is to begin the standardization process, promote dialogue in this field, and eventually reach a consensus in collaboration with the ADA, AACE, and other pertinent stakeholders.

CONCLUSIONS

Like the field of inpatient glycemic management itself, the field of devising metrics to measure the quality of inpatient glycemic control is also in its infancy and quickly evolving. One should not be paralyzed by the lack of consensus regarding measurementthe important point is to pick a few complementary metrics and begin the process. The table of recommendations can hopefully serve as a starting point for many institutions, with a focus on efficacy (glycemic control), safety (hypoglycemia), and process (insulin use patterns). As your institution gains experience with measurement and the field evolves, your metrics will likely change. We recommend keeping all process and outcome data in its raw form so that it can be summarized in different ways over time. It is also important not to wait for the perfect data collection tool before beginning to analyze data: sampling and paper processes are acceptable if automated data collection is not yet possible. Eventually, blood glucose meter readings should be downloaded into a central database that interfaces with hospital data repositories so data can be analyzed in conjunction with patient, service, and unit‐level information. Only with a rigorous measurement process can institutions hope to know whether their changes are resulting in improved care for patients.

Data collection, analysis, and presentation are key to the success of any hospital glycemic control initiative. Such efforts enable the management team to track improvements in processes and outcomes, make necessary changes to their quality improvement efforts, justify the provision of necessary time and resources, and share their results with others. Reliable metrics for assessing glycemic control and frequency of hypoglycemia are essential to accomplish these tasks and to assess whether interventions result in more benefit than harm. Hypoglycemia metrics must be especially convincing because fear of hypoglycemia remains a major source of clinical inertia, impeding efforts to improve glucose control.

Currently, there are no official standards or guidelines for formulating metrics on the quality of inpatient glycemic control. This creates several problems. First, different metrics vary in their biases and in their responsiveness to change. Thus, use of a poor metric could lead to either a falsely positive or falsely negative impression that a quality improvement intervention is in fact improving glycemic control. Second, the proliferation of different measures and analytical plans in the research and quality improvement literature make it very difficult for hospitals to compare baseline performance, determine need for improvement, and understand which interventions may be most effective.

A related article in this supplement provides the rationale for improved inpatient glycemic control. That article argues that the current state of inpatient glycemic control, with the frequent occurrence of severe hyperglycemia and irrational insulin ordering, cannot be considered acceptable, especially given the large body of data (albeit largely observational) linking hyperglycemia to negative patient outcomes. However, regardless of whether one is an advocate or skeptic of tighter glucose control in the intensive care unit (ICU) and especially the non‐ICU setting, there is no question that standardized, valid, and reliable metrics are needed to compare efforts to improve glycemic control, better understand whether such control actually improves patient care, and closely monitor patient safety.

This article provides a summary of practical suggestions to assess glycemic control, insulin use patterns, and safety (hypoglycemia and severe hyperglycemia). In particular, we discuss the pros and cons of various measurement choices. We conclude with a tiered summary of recommendations for practical metrics that we hope will be useful to individual improvement teams. This article is not a consensus statement but rather a starting place that we hope will begin to standardize measurement across institutions and advance the dialogue on this subject. To more definitely address this problem, we call on the American Association of Clinical Endocrinologists (AACE), American Diabetes Association (ADA), Society of Hospital Medicine (SHM), and others to agree on consensus standards regarding metrics for the quality of inpatient glycemic control.

MEASURING GLYCEMIC CONTROL: GLUCOMETRICS

Glucometrics may be defined as the systematic analysis of blood glucose (BG) dataa phrase initially coined specifically for the inpatient setting. There are numerous ways to do these analyses, depending on which patients and glucose values are considered, the definitions used for hypoglycemia and hyperglycemia, the unit of measurement (eg, patient, patient‐day, individual glucose value), and the measure of control (eg, mean, median, percent of glucose readings within a certain range). We consider each of these dimensions in turn.

APPROACHES TO ANALYSIS: RUN CHARTS

Most conventional clinical trials hold interventions fixed for a period of time and compare results with and without the intervention. For quality improvement studies, this is still a valid way to proceed, especially if studied as a randomized controlled trial. Such methods may be preferred when the clinical question is Does this type of intervention work in general? and the desired output is publication in peer‐reviewed journals so that others can learn about and adopt the intervention to their own institution. A before and after study with a similar analytic approach may also be valid, although concerns about temporal trends and cointerventions potentially compromise the validity of such studies. This approach again assumes that an intervention is held fixed over time such that it is clear what patients received during each time period.

If the desired result is improvement at a given institution (the question is Did we improve care?) then it may be preferable to present results over time using run‐charts. In a run chart, the x‐axis is time and the y‐axis the desired metric, such as patient‐day weighted mean glucose. Points in time when interventions were introduced or modified can be highlighted. Run charts have several advantages over before‐and‐after summaries: they do not require interventions remaining fixed and are more compatible with continuous quality improvement methods, it is easier to see the effect of different aspects of the interventions as they occur, one can get a quicker picture of whether something is working, and it is easier to separate out the impact of the intervention from secular trends. Finally, the use of run charts does not imply the absence of statistical rigor. Run charts with statistical process control (SPC) limits5 can easily convey when the observed time trend is unlikely to be due to chance using prespecified P values. (A full discussion of SPC and other methods to study quality improvement interventions is beyond the scope of this article.)

ASSESSING PATTERNS OF INSULIN USE AND ORDER SET UTILIZATION

Besides measuring the impact of quality improvement interventions on glucose control, it is important to measure processes such as proper insulin use. As mentioned in other articles in this supplement, processes are much more sensitive to change than outcomes. Failure to change processes should lead one to make changes to the intervention.

OTHER CONSIDERATIONS AND METRICS

Examples of other metrics that can be used to track the success of quality improvement efforts include:

• 1

  Glucose measurement within 8 hours of hospital admission.

• 2

  Glycated hemoglobin (A1C) measurement obtained or available within 30 days of admission to help guide inpatient and especially discharge management.

• 3

  Appropriate glucose testing in patients with diabetes or hyperglycemia (eg, 4 times per day in patients not on insulin infusion protocols, at least until 24 hours of euglycemia is documented).

• 4

  The percentage of patients on insulin with on‐time tray delivery.

• 5

  The timing of subcutaneous insulin administration in relation to glucose testing and nutrition delivery.

• 6

  Documentation of carbohydrate intake among patients who are eating.

• 7

  Satisfaction of physicians and nurses with order sets or protocols, using standard surveys.

• 8

  Physician and nurse knowledge, attitudes, and beliefs about insulin administration, fear of hypoglycemia, treatment of hypoglycemia, and glycemic control in the hospital.

• 9

  Patient satisfaction with their diabetes care in the hospital, including the education they received.

• 10

  Nursing and physician education/certification in insulin prescribing, insulin administration, and other diabetes care issues.

• 11

  Patient outcomes strongly associated with glycemic control, (eg, surgical wound infections, ICU LOS, catheter‐related bloodstream infections).

• 12

  Appropriate treatment and documentation of hypoglycemia (eg, in accordance with hospital policy).

• 13

  Documentation of severe hypoglycemic events through the hospital's adverse events reporting system (these may actually increase as change comes to the organization and as clinical personnel are more attuned to glycemic control).

• 14

  Root causes of hypoglycemic events, which can be used to understand and prevent future events.

• 15

  Appropriate transitions from IV to SC insulin regimens, (eg, starting basal insulin prior to discontinuing infusion in patients who have been on an insulin infusion of at least 2 units/hour or who have a known diagnosis of diabetes or A1C >7).

(Survey instruments and other measurement tools are available from the authors upon request.)

SHM GLYCEMIC CONTROL TASK FORCE SUMMARY RECOMMENDATIONS

The SHM Glycemic Control Task Force is working to develop standardized measures of inpatient glucose control and related indicators to track progress of hospital glycemic control initiatives (see the introduction to this supplement for a description of the charge and membership of this task force). The goals of the Task Force's metrics recommendations (Table 1) are several‐fold: (1) create a set of measurements that are complete but not overly burdensome; (2) create realistic measures that can be applied to institutions with different data management capabilities; and (3) allow for comparison across institutions for benchmarking purposes, evaluation of quality improvement projects, and reporting of results for formal research studies in this field.

For each domain of glycemic management (glycemic control, safety, and insulin use), the task force chose a set of best measures. They are presented as two tiers of measurement standards, depending on the capabilities of the institution and the planned uses of the data. Tier 1 includes measures that, although they do take time and resources to collect, are feasible for most institutions. Tier 2 measures are recommended for hospitals with easy manipulation of electronic sources of data and for reporting quality‐of‐care measures for widespread publication, that is, in the context of a research study. It should be emphasized that these recommendations are only meant as a guide: the actual measures chosen should meet the needs and capabilities of each institution.

We recognize that few data support the recommendations made by this task force, that such data are needed, and that the field of data collection and analysis for hospital glycemic management is rapidly evolving. The hope is to begin the standardization process, promote dialogue in this field, and eventually reach a consensus in collaboration with the ADA, AACE, and other pertinent stakeholders.

CONCLUSIONS

Like the field of inpatient glycemic management itself, the field of devising metrics to measure the quality of inpatient glycemic control is also in its infancy and quickly evolving. One should not be paralyzed by the lack of consensus regarding measurementthe important point is to pick a few complementary metrics and begin the process. The table of recommendations can hopefully serve as a starting point for many institutions, with a focus on efficacy (glycemic control), safety (hypoglycemia), and process (insulin use patterns). As your institution gains experience with measurement and the field evolves, your metrics will likely change. We recommend keeping all process and outcome data in its raw form so that it can be summarized in different ways over time. It is also important not to wait for the perfect data collection tool before beginning to analyze data: sampling and paper processes are acceptable if automated data collection is not yet possible. Eventually, blood glucose meter readings should be downloaded into a central database that interfaces with hospital data repositories so data can be analyzed in conjunction with patient, service, and unit‐level information. Only with a rigorous measurement process can institutions hope to know whether their changes are resulting in improved care for patients.

The goal of this article is to explain how the first in a series of online resource rooms provides trainees and hospitalists with quality improvement tools that can be applied locally to improve inpatient care.1 During the emergence and explosive growth of hospital medicine, the SHM recognized the need to revise training relating to inpatient care and hospital process design to meet the evolving expectation of hospitalists that their performance will be measured, to actively set quality parameters, and to lead multidisciplinary teams to improve hospital performance.2 Armed with the appropriate skill set, hospitalists would be uniquely situated to lead and manage improvements in processes in the hospitals in which they work.

The content of the first Society of Hospital Medicine (SHM) Quality Improvement Resource Room (QI RR) supports hospitalists leading a multidisciplinary team dedicated to improving inpatient outcomes by preventing hospital‐acquired venous thromboembolism (VTE), a common cause of morbidity and mortality in hospitalized patients.3 The SHM developed this educational resource in the context of numerous reports on the incidence of medical errors in US hospitals and calls for action to improve the quality of health care.'47 Hospital report cards on quality measures are now public record, and hospitals will require uniformity in practice among physicians. Hospitalists are increasingly expected to lead initiatives that will implement national standards in key practices such as VTE prophylaxis2.

The QI RRs of the SHM are a collection of electronic tools accessible through the SHM Web site. They are designed to enhance the readiness of hospitalists and members of the multidisciplinary inpatient team to redesign care at the institutional level. Although all performance improvement is ultimately occurs locally, many QI methods and tools transcend hospital geography and disease topic. Leveraging a Web‐based platform, the SHM QI RRs present hospitalists with a general approach to QI, enriched by customizable workbooks that can be downloaded to best meet user needs. This resource is an innovation in practice‐based learning, quality improvement, and systems‐based practice.

METHODS

Development of the first QI RR followed a series of steps described in Curriculum Development for Medical Education8 (for process and timeline, see Table 1). Inadequate VTE prophylaxis was identified as an ongoing widespread problem of health care underutilization despite randomized clinical trials supporting the efficacy of prophylaxis.9, 10 Mirroring the AHRQ's assessment of underutilization of VTE prophylaxis as the single most important safety priority,6 the first QI RR focused on VTE, with plans to cover additional clinical conditions over time. As experts in the care of inpatients, hospitalists should be able to take custody of predictable complications of serious illness, identify and lower barriers to prevention, critically review prophylaxis options, utilize hospital‐specific data, and devise strategies to bridge the gap between knowledge and practice. Already leaders of multidisciplinary care teams, hospitalists are primed to lead multidisciplinary improvement teams as well.

Available data on the demographics of hospitalists and feedback from the SHM membership, leadership, and committees indicated that most learners would have minimal previous exposure to QI concepts and only a few years of management experience. Any previous quality improvement initiatives would tend to have been isolated, experimental, or smaller in scale. The resource rooms are designed to facilitate quality improvement learning among hospitalists that is practice‐based and immediately relevant to patient care. Measurable improvement in particular care processes or outcomes should correlate with actual learning.

The educational strategy of the SHM was predicated on ensuring that a quality and patient safety curriculum would retain clinical applicability in the hospital setting. This approach, grounded in adult learning principles and common to medical education, teaches general principles by framing the learning experience as problem centered.11 Several domains were identified as universally important to any quality improvement effort: raising awareness of a local performance gap, applying the best current evidence to practice, tapping the experience of others leading QI efforts, and using measurements derived from rapid‐cycle tests of change. Such a template delineates the components of successful QI planning, implementation, and evaluation and provides users with a familiar RR format applicable to improving any care process, not just VTE.

The Internet was chosen as the mechanism for delivering training on the basis of previous surveys of the SHM membership in which members expressed a preference for electronic and Web‐based forms of educational content delivery. Drawing from the example of other organizations teaching quality improvement, including the Institute for Healthcare Improvement and Intermountain Health Care, the SHM valued the ubiquity of a Web‐based educational resource. To facilitate on‐the‐job training, the first SHM QI RR provides a comprehensive tool kit to guide hospitalists through the process of advocating, developing, implementing, and evaluating a QI initiative for VTE.

Prior to launching the resource room, formative input was collected from SHM leaders, a panel of education and QI experts, and attendees of the society's annual meetings. Such input followed each significant step in the development of the RR curricula. For example, visitors at a kiosk at the 2005 SHM annual meeting completed surveys as they navigated through the VTE QI RR. This focused feedback shaped prelaunch development. The ultimate performance evaluation and feedback for the QI RR curricula will be gauged by user reports of measurable improvement in specific hospital process or outcomes measures. The VTE QI RR was launched in August 2005 and promoted at the SHM Web site.

RESULTS

The content and layout of the VTE QI RR are depicted in Figure 1. The self‐directed learner may navigate through the entire resource room or just select areas for study. Those likely to visit only a single area are individuals looking for guidance to support discrete roles on the improvement team: champion, clinical leader, facilitator of the QI process, or educator of staff or patient audiences (see Figure 2).

Figure 1

Figure 2

DISCUSSION

A case example, shown in Figure 3, demonstrates how content accessible through the SHM VTE QI RR may be used to catalyze a local quality improvement effort.

Figure 3

Hospitals will be measured on rates of VTE prophylaxis on medical and surgical services. Failure to standardize prophylaxis among different physician groups may adversely affect overall performance, with implications for both patient care and accreditation. The lack of a agreed‐on gold standard of what constitutes appropriate prophylaxis for a given patient does not absolve an institution of the duty to implement its own standards. The challenge of achieving local consensus on appropriate prophylaxis should not outweigh the urgency to address preventable in‐hospital deaths. In caring for increasing numbers of general medical and surgical patients, hospitalists are likely to be asked to develop and implement a protocol for VTE prophylaxis that can be used hospitalwide. In many instances hospitalists will accept this charge in the aftermath of previous hospital failures in which admission order sets or VTE assessment protocols were launched but never widely implemented. As the National Quality Forum or JCAHO regulations for uniformity among hospitals shift VTE prophylaxis from being voluntary to compulsory, hospitalists will need to develop improvement strategies that have greater reliability.

Hospitalists with no formal training in either vascular medicine or quality improvement may not be able to immediately cite the most current data about VTE prophylaxis rates and regimens and may not have the time to enroll in a training course on quality improvement. How would hospitalists determine baseline rates of appropriate VTE prophylaxis? How can medical education be used to build consensus and recruit support from other physicians? What should be the scope of the QI initiative, and what patient population should be targeted for intervention?

The goal of the SHM QI RR is to provide the tools and the framework to help hospitalists develop, implement, and manage a VTE prophylaxis quality improvement initiative. Suggested Steps to Action in the Awareness section depict exactly how a hospital medicine service can use the campaign's materials to raise institutional support for tackling this preventable problem. Hospital quality officers can direct the hospital's public relations department to the Awareness section for DVT Awareness Month materials, including public service announcements in audio, visual, and print formats. The hold music at the hospital can be temporarily replaced, television kiosks can be set up to run video loops, and banners can be printed and hung in central locations, all to get out the message simultaneously to patients and medical staff.

The Evidence section of the VTE QI RR references a key benchmark study, the DVT‐Free Prospective Registry.9 This study reported that at 183 sites in North America and Europe, more than twice as many medical patients as surgical patients failed to receive prophylaxis. The Evidence section includes the 7th American College of Chest Physicians Consensus Conference on Antithrombotic and Thrombolytic Therapy and also highlights 3 randomized placebo‐controlled clinical trials (MEDENOX 1999, ARTEMIS 2003, and PREVENT 2004) that have reported significant reduction of risk of VTE (50%‐60%) from pharmacologic prophylaxis in moderate‐risk medical inpatients.1315 Review of the data helps to determine which patient population to study first, which prophylaxis options a hospital could deploy appropriately, and the expected magnitude of the effect. Because the literature has already been narrowed and is kept current, hospitalists can save time in answering a range of questions, from the most commonly agreed‐on factors to stratify risk to which populations require alternative interventions.

The Experience section references the first clinical trial demonstrating improved patient outcomes from a quality improvement initiative aimed at improving utilization of VTE prophylaxis.10 At the large teaching hospital where the electronic alerts were studied, a preexisting wealth of educational information on the hospital Web site, in the form of multiple seminars and lectures on VTE prophylaxis by opinion leaders and international experts, had little impact on practice. For this reason, the investigators implemented a trial of how to change physician behavior by introducing a point‐of‐care intervention, the computer alerts. Clinicians prompted by an electronic alert to consider DVT prophylaxis for at‐risk patients employed nearly double the rate of pharmacologic prophylaxis and reduced the incidence of DVT or pulmonary embolism (PE) by 41%. This study suggests that a change introduced to the clinical workflow can improve evidence‐based VTE prophylaxis and also can reduce the incidence of VTE in acutely ill hospitalized patients.

We believe that if hospitalists use the current evidence and experience assembled in the VTE QI RR, they could develop and lead a systematic approach to improving utilization of VTE prophylaxis. Although there is no gold standard method for integrating VTE risk assessment into clinical workflow, the VTE QI RR presents key lessons both from the literature and real world experiences. The crucial take‐home message is that hospitalists can facilitate implementation of VTE risk assessments if they stress simplicity (ie, the sick, old, surgery benefit), link the risk assessment to a menu of evidence‐based prophylaxis options, and require assessment of VTE risk as part of a regular routine (on admission and at regular intervals). Although many hospitals do not yet have computerized entry of physician orders, the simple 4‐point VTE risk assessment described by Kucher et al might be applied to other hospitals.10 The 4‐point system would identify the patients at highest risk, a reasonable starting point for a QI initiative. Whatever the modelCPOE alerts of very high‐risk patients, CPOE‐forced VTE risk assessments, nursing assessments, or paper‐based order setsregular VTE risk assessment can be incorporated into the daily routine of hospital care.

The QI workbook sequences the steps of a multidisciplinary improvement team and prompts users to set specific goals, collect practical metrics, and conduct plan‐do‐study‐act (PDSA) cycles of learning and action (Figure 4). Hospitalists and other team members can use the information in the workbook to estimate the prevalence of use of the appropriate VTE prophylaxis and the incidence of hospital‐acquired VTE at their medical centers, develop a suitable VTE risk assessment model, and plan interventions. Starting with all patients admitted to one nurse on one unit, then expanding to an entire nursing unit, an improvement team could implement rapid PDSA cycles to iron out the wrinkles of a risk assessment protocol. After demonstrating a measurable benefit for the patients at highest risk, the team would then be expected to capture more patients at risk for VTE by modifying the risk assessment protocol to identify moderate‐risk patients (hospitalized patients with one risk factor), as in the MEDENOX, ARTEMIS, and PREVENT clinical trials. Within the first several months, the QI intervention could be expanded to more nursing units. An improvement report profiling a clinically important increase in the rate of appropriate VTE prophylaxis would advocate for additional local resources and projects.

Figure 4

As questions arise in assembling an improvement team, setting useful aims and metrics, choosing interventions, implementing and studying change, or collecting performance data, hospitalists can review answers to questions already posted and post their own questions in the Ask the Expert area. For example, one user asked whether there was a standard risk assessment tool for identifying patients at high risk of VTE. Another asked about the use of unfractionated heparin as a low‐cost alternative to low‐molecular‐weight heparin. Both these questions were answered within 24 hours by the content editor of the VTE QI RR and, for one question, also by 2 pharmacists and an international expert in VTE.

As other hospitalists begin de novo efforts of their own, success stories and strategies posted in the online forums of the VTE QI RR will be an evolving resource for basic know‐how and innovation.

Suggestions from a community of resource room users will be solicited, evaluated, and incorporated into the QI RR in order to improve its educational value and utility. The curricula could also be adapted or refined by others with an interest in systems‐based care or practice‐based learning, such as directors of residency training programs.

CONCLUSIONS

The QI RRs bring QI theory and practice to the hospitalist, when and wherever it is wanted, minimizing time away from patient care. The workbook links theory to practice and can be used to launch, sustain, and document a local VTE‐specific QI initiative. A range of experience is accommodated. Content is provided in a way that enables the user to immediately apply and adapt it to a local contextusers can access and download the subset of tools that best meet their needs. For practicing hospitalists, this QI resource offers an opportunity to bridge the training gap in systems‐based hospital care and should increase the quality and quantity of and support for opportunities to lead successful QI projects.

The Accreditation Council of Graduate Medical Education (ACGME) now requires education in health care systems, a requirement not previously mandated for traditional medical residency programs.17 Because the resource rooms should increase the number of hospitalists competently leading local efforts that achieve measurable gains in hospital outcomes, a wider potential constituency also includes residency program directors, internal medicine residents, physician assistants and nurse‐practitioners, nurses, hospital quality officers, and hospital medicine practice leaders.

Further research is needed to determine the clinical impact of the VTE QI workbook on outcomes for hospitalized patients. The effectiveness of such an educational method should be evaluated, at least in part, by documenting changes in clinically important process and outcome measures, in this case those specific to hospital‐acquired VTE. Investigation also will need to generate an impact assessment to see if the curricula are effective in meeting the strategic educational goals of the Society of Hospital Medicine. Further investigation will examine whether this resource can help residency training programs achieve ACGME goals for practice‐based learning and systems‐based care.

The goal of this article is to explain how the first in a series of online resource rooms provides trainees and hospitalists with quality improvement tools that can be applied locally to improve inpatient care.1 During the emergence and explosive growth of hospital medicine, the SHM recognized the need to revise training relating to inpatient care and hospital process design to meet the evolving expectation of hospitalists that their performance will be measured, to actively set quality parameters, and to lead multidisciplinary teams to improve hospital performance.2 Armed with the appropriate skill set, hospitalists would be uniquely situated to lead and manage improvements in processes in the hospitals in which they work.

The content of the first Society of Hospital Medicine (SHM) Quality Improvement Resource Room (QI RR) supports hospitalists leading a multidisciplinary team dedicated to improving inpatient outcomes by preventing hospital‐acquired venous thromboembolism (VTE), a common cause of morbidity and mortality in hospitalized patients.3 The SHM developed this educational resource in the context of numerous reports on the incidence of medical errors in US hospitals and calls for action to improve the quality of health care.'47 Hospital report cards on quality measures are now public record, and hospitals will require uniformity in practice among physicians. Hospitalists are increasingly expected to lead initiatives that will implement national standards in key practices such as VTE prophylaxis2.

The QI RRs of the SHM are a collection of electronic tools accessible through the SHM Web site. They are designed to enhance the readiness of hospitalists and members of the multidisciplinary inpatient team to redesign care at the institutional level. Although all performance improvement is ultimately occurs locally, many QI methods and tools transcend hospital geography and disease topic. Leveraging a Web‐based platform, the SHM QI RRs present hospitalists with a general approach to QI, enriched by customizable workbooks that can be downloaded to best meet user needs. This resource is an innovation in practice‐based learning, quality improvement, and systems‐based practice.

METHODS

Development of the first QI RR followed a series of steps described in Curriculum Development for Medical Education8 (for process and timeline, see Table 1). Inadequate VTE prophylaxis was identified as an ongoing widespread problem of health care underutilization despite randomized clinical trials supporting the efficacy of prophylaxis.9, 10 Mirroring the AHRQ's assessment of underutilization of VTE prophylaxis as the single most important safety priority,6 the first QI RR focused on VTE, with plans to cover additional clinical conditions over time. As experts in the care of inpatients, hospitalists should be able to take custody of predictable complications of serious illness, identify and lower barriers to prevention, critically review prophylaxis options, utilize hospital‐specific data, and devise strategies to bridge the gap between knowledge and practice. Already leaders of multidisciplinary care teams, hospitalists are primed to lead multidisciplinary improvement teams as well.

Available data on the demographics of hospitalists and feedback from the SHM membership, leadership, and committees indicated that most learners would have minimal previous exposure to QI concepts and only a few years of management experience. Any previous quality improvement initiatives would tend to have been isolated, experimental, or smaller in scale. The resource rooms are designed to facilitate quality improvement learning among hospitalists that is practice‐based and immediately relevant to patient care. Measurable improvement in particular care processes or outcomes should correlate with actual learning.

The educational strategy of the SHM was predicated on ensuring that a quality and patient safety curriculum would retain clinical applicability in the hospital setting. This approach, grounded in adult learning principles and common to medical education, teaches general principles by framing the learning experience as problem centered.11 Several domains were identified as universally important to any quality improvement effort: raising awareness of a local performance gap, applying the best current evidence to practice, tapping the experience of others leading QI efforts, and using measurements derived from rapid‐cycle tests of change. Such a template delineates the components of successful QI planning, implementation, and evaluation and provides users with a familiar RR format applicable to improving any care process, not just VTE.

The Internet was chosen as the mechanism for delivering training on the basis of previous surveys of the SHM membership in which members expressed a preference for electronic and Web‐based forms of educational content delivery. Drawing from the example of other organizations teaching quality improvement, including the Institute for Healthcare Improvement and Intermountain Health Care, the SHM valued the ubiquity of a Web‐based educational resource. To facilitate on‐the‐job training, the first SHM QI RR provides a comprehensive tool kit to guide hospitalists through the process of advocating, developing, implementing, and evaluating a QI initiative for VTE.

Prior to launching the resource room, formative input was collected from SHM leaders, a panel of education and QI experts, and attendees of the society's annual meetings. Such input followed each significant step in the development of the RR curricula. For example, visitors at a kiosk at the 2005 SHM annual meeting completed surveys as they navigated through the VTE QI RR. This focused feedback shaped prelaunch development. The ultimate performance evaluation and feedback for the QI RR curricula will be gauged by user reports of measurable improvement in specific hospital process or outcomes measures. The VTE QI RR was launched in August 2005 and promoted at the SHM Web site.

RESULTS

The content and layout of the VTE QI RR are depicted in Figure 1. The self‐directed learner may navigate through the entire resource room or just select areas for study. Those likely to visit only a single area are individuals looking for guidance to support discrete roles on the improvement team: champion, clinical leader, facilitator of the QI process, or educator of staff or patient audiences (see Figure 2).

Figure 1

Figure 2

DISCUSSION

A case example, shown in Figure 3, demonstrates how content accessible through the SHM VTE QI RR may be used to catalyze a local quality improvement effort.

Figure 3

Hospitals will be measured on rates of VTE prophylaxis on medical and surgical services. Failure to standardize prophylaxis among different physician groups may adversely affect overall performance, with implications for both patient care and accreditation. The lack of a agreed‐on gold standard of what constitutes appropriate prophylaxis for a given patient does not absolve an institution of the duty to implement its own standards. The challenge of achieving local consensus on appropriate prophylaxis should not outweigh the urgency to address preventable in‐hospital deaths. In caring for increasing numbers of general medical and surgical patients, hospitalists are likely to be asked to develop and implement a protocol for VTE prophylaxis that can be used hospitalwide. In many instances hospitalists will accept this charge in the aftermath of previous hospital failures in which admission order sets or VTE assessment protocols were launched but never widely implemented. As the National Quality Forum or JCAHO regulations for uniformity among hospitals shift VTE prophylaxis from being voluntary to compulsory, hospitalists will need to develop improvement strategies that have greater reliability.

Hospitalists with no formal training in either vascular medicine or quality improvement may not be able to immediately cite the most current data about VTE prophylaxis rates and regimens and may not have the time to enroll in a training course on quality improvement. How would hospitalists determine baseline rates of appropriate VTE prophylaxis? How can medical education be used to build consensus and recruit support from other physicians? What should be the scope of the QI initiative, and what patient population should be targeted for intervention?

The goal of the SHM QI RR is to provide the tools and the framework to help hospitalists develop, implement, and manage a VTE prophylaxis quality improvement initiative. Suggested Steps to Action in the Awareness section depict exactly how a hospital medicine service can use the campaign's materials to raise institutional support for tackling this preventable problem. Hospital quality officers can direct the hospital's public relations department to the Awareness section for DVT Awareness Month materials, including public service announcements in audio, visual, and print formats. The hold music at the hospital can be temporarily replaced, television kiosks can be set up to run video loops, and banners can be printed and hung in central locations, all to get out the message simultaneously to patients and medical staff.

The Evidence section of the VTE QI RR references a key benchmark study, the DVT‐Free Prospective Registry.9 This study reported that at 183 sites in North America and Europe, more than twice as many medical patients as surgical patients failed to receive prophylaxis. The Evidence section includes the 7th American College of Chest Physicians Consensus Conference on Antithrombotic and Thrombolytic Therapy and also highlights 3 randomized placebo‐controlled clinical trials (MEDENOX 1999, ARTEMIS 2003, and PREVENT 2004) that have reported significant reduction of risk of VTE (50%‐60%) from pharmacologic prophylaxis in moderate‐risk medical inpatients.1315 Review of the data helps to determine which patient population to study first, which prophylaxis options a hospital could deploy appropriately, and the expected magnitude of the effect. Because the literature has already been narrowed and is kept current, hospitalists can save time in answering a range of questions, from the most commonly agreed‐on factors to stratify risk to which populations require alternative interventions.

The Experience section references the first clinical trial demonstrating improved patient outcomes from a quality improvement initiative aimed at improving utilization of VTE prophylaxis.10 At the large teaching hospital where the electronic alerts were studied, a preexisting wealth of educational information on the hospital Web site, in the form of multiple seminars and lectures on VTE prophylaxis by opinion leaders and international experts, had little impact on practice. For this reason, the investigators implemented a trial of how to change physician behavior by introducing a point‐of‐care intervention, the computer alerts. Clinicians prompted by an electronic alert to consider DVT prophylaxis for at‐risk patients employed nearly double the rate of pharmacologic prophylaxis and reduced the incidence of DVT or pulmonary embolism (PE) by 41%. This study suggests that a change introduced to the clinical workflow can improve evidence‐based VTE prophylaxis and also can reduce the incidence of VTE in acutely ill hospitalized patients.

We believe that if hospitalists use the current evidence and experience assembled in the VTE QI RR, they could develop and lead a systematic approach to improving utilization of VTE prophylaxis. Although there is no gold standard method for integrating VTE risk assessment into clinical workflow, the VTE QI RR presents key lessons both from the literature and real world experiences. The crucial take‐home message is that hospitalists can facilitate implementation of VTE risk assessments if they stress simplicity (ie, the sick, old, surgery benefit), link the risk assessment to a menu of evidence‐based prophylaxis options, and require assessment of VTE risk as part of a regular routine (on admission and at regular intervals). Although many hospitals do not yet have computerized entry of physician orders, the simple 4‐point VTE risk assessment described by Kucher et al might be applied to other hospitals.10 The 4‐point system would identify the patients at highest risk, a reasonable starting point for a QI initiative. Whatever the modelCPOE alerts of very high‐risk patients, CPOE‐forced VTE risk assessments, nursing assessments, or paper‐based order setsregular VTE risk assessment can be incorporated into the daily routine of hospital care.

The QI workbook sequences the steps of a multidisciplinary improvement team and prompts users to set specific goals, collect practical metrics, and conduct plan‐do‐study‐act (PDSA) cycles of learning and action (Figure 4). Hospitalists and other team members can use the information in the workbook to estimate the prevalence of use of the appropriate VTE prophylaxis and the incidence of hospital‐acquired VTE at their medical centers, develop a suitable VTE risk assessment model, and plan interventions. Starting with all patients admitted to one nurse on one unit, then expanding to an entire nursing unit, an improvement team could implement rapid PDSA cycles to iron out the wrinkles of a risk assessment protocol. After demonstrating a measurable benefit for the patients at highest risk, the team would then be expected to capture more patients at risk for VTE by modifying the risk assessment protocol to identify moderate‐risk patients (hospitalized patients with one risk factor), as in the MEDENOX, ARTEMIS, and PREVENT clinical trials. Within the first several months, the QI intervention could be expanded to more nursing units. An improvement report profiling a clinically important increase in the rate of appropriate VTE prophylaxis would advocate for additional local resources and projects.

Figure 4

As questions arise in assembling an improvement team, setting useful aims and metrics, choosing interventions, implementing and studying change, or collecting performance data, hospitalists can review answers to questions already posted and post their own questions in the Ask the Expert area. For example, one user asked whether there was a standard risk assessment tool for identifying patients at high risk of VTE. Another asked about the use of unfractionated heparin as a low‐cost alternative to low‐molecular‐weight heparin. Both these questions were answered within 24 hours by the content editor of the VTE QI RR and, for one question, also by 2 pharmacists and an international expert in VTE.

As other hospitalists begin de novo efforts of their own, success stories and strategies posted in the online forums of the VTE QI RR will be an evolving resource for basic know‐how and innovation.

Suggestions from a community of resource room users will be solicited, evaluated, and incorporated into the QI RR in order to improve its educational value and utility. The curricula could also be adapted or refined by others with an interest in systems‐based care or practice‐based learning, such as directors of residency training programs.

CONCLUSIONS

The QI RRs bring QI theory and practice to the hospitalist, when and wherever it is wanted, minimizing time away from patient care. The workbook links theory to practice and can be used to launch, sustain, and document a local VTE‐specific QI initiative. A range of experience is accommodated. Content is provided in a way that enables the user to immediately apply and adapt it to a local contextusers can access and download the subset of tools that best meet their needs. For practicing hospitalists, this QI resource offers an opportunity to bridge the training gap in systems‐based hospital care and should increase the quality and quantity of and support for opportunities to lead successful QI projects.

The Accreditation Council of Graduate Medical Education (ACGME) now requires education in health care systems, a requirement not previously mandated for traditional medical residency programs.17 Because the resource rooms should increase the number of hospitalists competently leading local efforts that achieve measurable gains in hospital outcomes, a wider potential constituency also includes residency program directors, internal medicine residents, physician assistants and nurse‐practitioners, nurses, hospital quality officers, and hospital medicine practice leaders.

Further research is needed to determine the clinical impact of the VTE QI workbook on outcomes for hospitalized patients. The effectiveness of such an educational method should be evaluated, at least in part, by documenting changes in clinically important process and outcome measures, in this case those specific to hospital‐acquired VTE. Investigation also will need to generate an impact assessment to see if the curricula are effective in meeting the strategic educational goals of the Society of Hospital Medicine. Further investigation will examine whether this resource can help residency training programs achieve ACGME goals for practice‐based learning and systems‐based care.

The goal of this article is to explain how the first in a series of online resource rooms provides trainees and hospitalists with quality improvement tools that can be applied locally to improve inpatient care.1 During the emergence and explosive growth of hospital medicine, the SHM recognized the need to revise training relating to inpatient care and hospital process design to meet the evolving expectation of hospitalists that their performance will be measured, to actively set quality parameters, and to lead multidisciplinary teams to improve hospital performance.2 Armed with the appropriate skill set, hospitalists would be uniquely situated to lead and manage improvements in processes in the hospitals in which they work.

The content of the first Society of Hospital Medicine (SHM) Quality Improvement Resource Room (QI RR) supports hospitalists leading a multidisciplinary team dedicated to improving inpatient outcomes by preventing hospital‐acquired venous thromboembolism (VTE), a common cause of morbidity and mortality in hospitalized patients.3 The SHM developed this educational resource in the context of numerous reports on the incidence of medical errors in US hospitals and calls for action to improve the quality of health care.'47 Hospital report cards on quality measures are now public record, and hospitals will require uniformity in practice among physicians. Hospitalists are increasingly expected to lead initiatives that will implement national standards in key practices such as VTE prophylaxis2.

The QI RRs of the SHM are a collection of electronic tools accessible through the SHM Web site. They are designed to enhance the readiness of hospitalists and members of the multidisciplinary inpatient team to redesign care at the institutional level. Although all performance improvement is ultimately occurs locally, many QI methods and tools transcend hospital geography and disease topic. Leveraging a Web‐based platform, the SHM QI RRs present hospitalists with a general approach to QI, enriched by customizable workbooks that can be downloaded to best meet user needs. This resource is an innovation in practice‐based learning, quality improvement, and systems‐based practice.

METHODS

Development of the first QI RR followed a series of steps described in Curriculum Development for Medical Education8 (for process and timeline, see Table 1). Inadequate VTE prophylaxis was identified as an ongoing widespread problem of health care underutilization despite randomized clinical trials supporting the efficacy of prophylaxis.9, 10 Mirroring the AHRQ's assessment of underutilization of VTE prophylaxis as the single most important safety priority,6 the first QI RR focused on VTE, with plans to cover additional clinical conditions over time. As experts in the care of inpatients, hospitalists should be able to take custody of predictable complications of serious illness, identify and lower barriers to prevention, critically review prophylaxis options, utilize hospital‐specific data, and devise strategies to bridge the gap between knowledge and practice. Already leaders of multidisciplinary care teams, hospitalists are primed to lead multidisciplinary improvement teams as well.

Available data on the demographics of hospitalists and feedback from the SHM membership, leadership, and committees indicated that most learners would have minimal previous exposure to QI concepts and only a few years of management experience. Any previous quality improvement initiatives would tend to have been isolated, experimental, or smaller in scale. The resource rooms are designed to facilitate quality improvement learning among hospitalists that is practice‐based and immediately relevant to patient care. Measurable improvement in particular care processes or outcomes should correlate with actual learning.

The educational strategy of the SHM was predicated on ensuring that a quality and patient safety curriculum would retain clinical applicability in the hospital setting. This approach, grounded in adult learning principles and common to medical education, teaches general principles by framing the learning experience as problem centered.11 Several domains were identified as universally important to any quality improvement effort: raising awareness of a local performance gap, applying the best current evidence to practice, tapping the experience of others leading QI efforts, and using measurements derived from rapid‐cycle tests of change. Such a template delineates the components of successful QI planning, implementation, and evaluation and provides users with a familiar RR format applicable to improving any care process, not just VTE.

The Internet was chosen as the mechanism for delivering training on the basis of previous surveys of the SHM membership in which members expressed a preference for electronic and Web‐based forms of educational content delivery. Drawing from the example of other organizations teaching quality improvement, including the Institute for Healthcare Improvement and Intermountain Health Care, the SHM valued the ubiquity of a Web‐based educational resource. To facilitate on‐the‐job training, the first SHM QI RR provides a comprehensive tool kit to guide hospitalists through the process of advocating, developing, implementing, and evaluating a QI initiative for VTE.

Prior to launching the resource room, formative input was collected from SHM leaders, a panel of education and QI experts, and attendees of the society's annual meetings. Such input followed each significant step in the development of the RR curricula. For example, visitors at a kiosk at the 2005 SHM annual meeting completed surveys as they navigated through the VTE QI RR. This focused feedback shaped prelaunch development. The ultimate performance evaluation and feedback for the QI RR curricula will be gauged by user reports of measurable improvement in specific hospital process or outcomes measures. The VTE QI RR was launched in August 2005 and promoted at the SHM Web site.

RESULTS

The content and layout of the VTE QI RR are depicted in Figure 1. The self‐directed learner may navigate through the entire resource room or just select areas for study. Those likely to visit only a single area are individuals looking for guidance to support discrete roles on the improvement team: champion, clinical leader, facilitator of the QI process, or educator of staff or patient audiences (see Figure 2).

Figure 1

Figure 2

DISCUSSION

A case example, shown in Figure 3, demonstrates how content accessible through the SHM VTE QI RR may be used to catalyze a local quality improvement effort.

Figure 3

Hospitals will be measured on rates of VTE prophylaxis on medical and surgical services. Failure to standardize prophylaxis among different physician groups may adversely affect overall performance, with implications for both patient care and accreditation. The lack of a agreed‐on gold standard of what constitutes appropriate prophylaxis for a given patient does not absolve an institution of the duty to implement its own standards. The challenge of achieving local consensus on appropriate prophylaxis should not outweigh the urgency to address preventable in‐hospital deaths. In caring for increasing numbers of general medical and surgical patients, hospitalists are likely to be asked to develop and implement a protocol for VTE prophylaxis that can be used hospitalwide. In many instances hospitalists will accept this charge in the aftermath of previous hospital failures in which admission order sets or VTE assessment protocols were launched but never widely implemented. As the National Quality Forum or JCAHO regulations for uniformity among hospitals shift VTE prophylaxis from being voluntary to compulsory, hospitalists will need to develop improvement strategies that have greater reliability.

Hospitalists with no formal training in either vascular medicine or quality improvement may not be able to immediately cite the most current data about VTE prophylaxis rates and regimens and may not have the time to enroll in a training course on quality improvement. How would hospitalists determine baseline rates of appropriate VTE prophylaxis? How can medical education be used to build consensus and recruit support from other physicians? What should be the scope of the QI initiative, and what patient population should be targeted for intervention?

The goal of the SHM QI RR is to provide the tools and the framework to help hospitalists develop, implement, and manage a VTE prophylaxis quality improvement initiative. Suggested Steps to Action in the Awareness section depict exactly how a hospital medicine service can use the campaign's materials to raise institutional support for tackling this preventable problem. Hospital quality officers can direct the hospital's public relations department to the Awareness section for DVT Awareness Month materials, including public service announcements in audio, visual, and print formats. The hold music at the hospital can be temporarily replaced, television kiosks can be set up to run video loops, and banners can be printed and hung in central locations, all to get out the message simultaneously to patients and medical staff.

The Evidence section of the VTE QI RR references a key benchmark study, the DVT‐Free Prospective Registry.9 This study reported that at 183 sites in North America and Europe, more than twice as many medical patients as surgical patients failed to receive prophylaxis. The Evidence section includes the 7th American College of Chest Physicians Consensus Conference on Antithrombotic and Thrombolytic Therapy and also highlights 3 randomized placebo‐controlled clinical trials (MEDENOX 1999, ARTEMIS 2003, and PREVENT 2004) that have reported significant reduction of risk of VTE (50%‐60%) from pharmacologic prophylaxis in moderate‐risk medical inpatients.1315 Review of the data helps to determine which patient population to study first, which prophylaxis options a hospital could deploy appropriately, and the expected magnitude of the effect. Because the literature has already been narrowed and is kept current, hospitalists can save time in answering a range of questions, from the most commonly agreed‐on factors to stratify risk to which populations require alternative interventions.

The Experience section references the first clinical trial demonstrating improved patient outcomes from a quality improvement initiative aimed at improving utilization of VTE prophylaxis.10 At the large teaching hospital where the electronic alerts were studied, a preexisting wealth of educational information on the hospital Web site, in the form of multiple seminars and lectures on VTE prophylaxis by opinion leaders and international experts, had little impact on practice. For this reason, the investigators implemented a trial of how to change physician behavior by introducing a point‐of‐care intervention, the computer alerts. Clinicians prompted by an electronic alert to consider DVT prophylaxis for at‐risk patients employed nearly double the rate of pharmacologic prophylaxis and reduced the incidence of DVT or pulmonary embolism (PE) by 41%. This study suggests that a change introduced to the clinical workflow can improve evidence‐based VTE prophylaxis and also can reduce the incidence of VTE in acutely ill hospitalized patients.

We believe that if hospitalists use the current evidence and experience assembled in the VTE QI RR, they could develop and lead a systematic approach to improving utilization of VTE prophylaxis. Although there is no gold standard method for integrating VTE risk assessment into clinical workflow, the VTE QI RR presents key lessons both from the literature and real world experiences. The crucial take‐home message is that hospitalists can facilitate implementation of VTE risk assessments if they stress simplicity (ie, the sick, old, surgery benefit), link the risk assessment to a menu of evidence‐based prophylaxis options, and require assessment of VTE risk as part of a regular routine (on admission and at regular intervals). Although many hospitals do not yet have computerized entry of physician orders, the simple 4‐point VTE risk assessment described by Kucher et al might be applied to other hospitals.10 The 4‐point system would identify the patients at highest risk, a reasonable starting point for a QI initiative. Whatever the modelCPOE alerts of very high‐risk patients, CPOE‐forced VTE risk assessments, nursing assessments, or paper‐based order setsregular VTE risk assessment can be incorporated into the daily routine of hospital care.

The QI workbook sequences the steps of a multidisciplinary improvement team and prompts users to set specific goals, collect practical metrics, and conduct plan‐do‐study‐act (PDSA) cycles of learning and action (Figure 4). Hospitalists and other team members can use the information in the workbook to estimate the prevalence of use of the appropriate VTE prophylaxis and the incidence of hospital‐acquired VTE at their medical centers, develop a suitable VTE risk assessment model, and plan interventions. Starting with all patients admitted to one nurse on one unit, then expanding to an entire nursing unit, an improvement team could implement rapid PDSA cycles to iron out the wrinkles of a risk assessment protocol. After demonstrating a measurable benefit for the patients at highest risk, the team would then be expected to capture more patients at risk for VTE by modifying the risk assessment protocol to identify moderate‐risk patients (hospitalized patients with one risk factor), as in the MEDENOX, ARTEMIS, and PREVENT clinical trials. Within the first several months, the QI intervention could be expanded to more nursing units. An improvement report profiling a clinically important increase in the rate of appropriate VTE prophylaxis would advocate for additional local resources and projects.

Figure 4

As questions arise in assembling an improvement team, setting useful aims and metrics, choosing interventions, implementing and studying change, or collecting performance data, hospitalists can review answers to questions already posted and post their own questions in the Ask the Expert area. For example, one user asked whether there was a standard risk assessment tool for identifying patients at high risk of VTE. Another asked about the use of unfractionated heparin as a low‐cost alternative to low‐molecular‐weight heparin. Both these questions were answered within 24 hours by the content editor of the VTE QI RR and, for one question, also by 2 pharmacists and an international expert in VTE.

As other hospitalists begin de novo efforts of their own, success stories and strategies posted in the online forums of the VTE QI RR will be an evolving resource for basic know‐how and innovation.

Suggestions from a community of resource room users will be solicited, evaluated, and incorporated into the QI RR in order to improve its educational value and utility. The curricula could also be adapted or refined by others with an interest in systems‐based care or practice‐based learning, such as directors of residency training programs.

CONCLUSIONS

The QI RRs bring QI theory and practice to the hospitalist, when and wherever it is wanted, minimizing time away from patient care. The workbook links theory to practice and can be used to launch, sustain, and document a local VTE‐specific QI initiative. A range of experience is accommodated. Content is provided in a way that enables the user to immediately apply and adapt it to a local contextusers can access and download the subset of tools that best meet their needs. For practicing hospitalists, this QI resource offers an opportunity to bridge the training gap in systems‐based hospital care and should increase the quality and quantity of and support for opportunities to lead successful QI projects.

The Accreditation Council of Graduate Medical Education (ACGME) now requires education in health care systems, a requirement not previously mandated for traditional medical residency programs.17 Because the resource rooms should increase the number of hospitalists competently leading local efforts that achieve measurable gains in hospital outcomes, a wider potential constituency also includes residency program directors, internal medicine residents, physician assistants and nurse‐practitioners, nurses, hospital quality officers, and hospital medicine practice leaders.

Further research is needed to determine the clinical impact of the VTE QI workbook on outcomes for hospitalized patients. The effectiveness of such an educational method should be evaluated, at least in part, by documenting changes in clinically important process and outcome measures, in this case those specific to hospital‐acquired VTE. Investigation also will need to generate an impact assessment to see if the curricula are effective in meeting the strategic educational goals of the Society of Hospital Medicine. Further investigation will examine whether this resource can help residency training programs achieve ACGME goals for practice‐based learning and systems‐based care.