Jeffrey L Schnipper, MD, MPH

In their report celebrating the increase in the number of hospitalists from a few hundred in the 1990s to more than 50,000 in 2016, Drs Robert Wachter and Lee Goldman also noted the stunted growth of productive hospital medicine research programs, which presents a challenge to academic credibility in hospital medicine.¹ Given the substantial increase in the number of hospitalists over the past two decades, we surveyed adult academic hospital medicine groups to quantify the number of hospitalist clinician investigators and identify gaps in resources for researchers. The number of clinician investigators supported at academic medical centers (AMCs) remains disturbingly low despite the rapid growth of our specialty. Some programs also reported a lack of access to fundamental research services. We report selected results from our survey and provide recommendations to support and facilitate the development of clinician investigators in hospital medicine.

We performed a survey of hospital medicine programs at AMCs in the United States through the Hospital Medicine Reengineering Network (HOMERuN), a hospital medicine research collaborative that facilitates and conducts multisite research studies.² The purpose of this survey was to obtain a profile of adult academic hospital medicine groups. Surveys were distributed via email to directors and/or senior leaders of each hospital medicine group between January and August 2019. In the survey, a clinician investigator was defined as “faculty whose primary nonclinical focus is scientific papers and grant writing.”

We received responses from 43 of the 86 invitees (50%), each of whom represented a unique hospital medicine group; 41 of the representatives responded to the questions concerning available research services. Collectively, these 43 programs represented 2,503 hospitalists. There were 79 clinician investigators reported among all surveyed hospital medicine groups (3.1% of all hospitalists). The median number of clinician investigators per hospital medicine group was 0 (range 0-12) (Appendix Figure 1), and 22 of 43 (51.2%) hospital medicine groups reported having no clinician investigators. Two of the hospital medicine groups, however, reported having 12 clinician investigators at their respective institutions, comprising nearly one third of the total number of clinician investigators reported in the survey.

Many of the programs reported lack of access to resources such as research assistants (56.1%) and dedicated research fellowships (53.7%) (Appendix Figure 2). A number of groups reported a need for more support for various junior faculty development activities, including research mentoring (53.5%), networking with other researchers (60.5%), and access to clinical data from multiple sites (62.8%).

One of the limitations of this survey was the manner in which the participating hospital medicine groups were chosen. Selection was based on groups affiliated with HOMERuN; among those chosen were highly visible US AMCs, including 70% of the top 20 AMCs based on National Institutes of Health (NIH) funding.³ Therefore, our results likely overestimate the research presence of hospital medicine across all AMCs in the United States.

Despite the substantial growth of hospital medicine over the past 2 decades, there has been no proportional increase in the number of hospitalist clinician investigators, with earlier surveys also demonstrating low numbers.^4,5 Along with the survey by Chopra and colleagues published in 2019,⁶ our survey provides an additional contemporary appraisal of research activities for adult academic hospital medicine groups. In the survey by Chopra et al, only 54% (15 of 28) of responding programs reported having any faculty with research as their major activity (ie, >50% effort), and 3% of total faculty reported having funding for >50% effort toward research.⁶ Our study expands upon these findings by providing more detailed data on the number of clinician investigators per hospital medicine group. Results of our survey showed a concentration of hospitalists within a small number of programs, which may have contributed to the observed lack of growth. We also expand on prior work by identifying a lack of resources and services to support hospitalist researchers.

The findings of our survey have important implications for the field of hospital medicine. Without a critical mass of hospitalist clinician investigators, the quality of research that addresses important questions in our field will suffer. It will also limit academic credibility of the field, as well as individual academic achievement; previous studies have consistently demonstrated that few hospitalists at AMCs achieve the rank of associate or full professor.^5-9

The results of our study additionally offer possible explanations for the dearth of clinician investigators in hospital medicine. The limited access to research resources and fellowship training identified in our survey are critical domains that must be addressed in order to develop successful academic hospital medicine programs.^4,6,8,10

Regarding dedicated hospital medicine research fellowships, there are only a handful across the country. The small number of existing research fellowships only have one or two fellows per year, and these positions often go unfilled because of a lack of applicants and lower salaries compared to full-time clinical positions.¹¹ The lack of applicants for adult hospital medicine fellowship positions is also integrally linked to board certification requirements. Unlike pediatric hospital medicine where additional fellowship training is required to become board-certified, no such fellowship is required in adult hospital medicine. In pediatrics, this requirement has led to a rapid increase in the number of fellowships with scholarly work requirements (more than 60 fellowships, plus additional programs in development) and greater standardization among training experiences.^12,13

The lack of fellowship applicants may also stem from the fact that many trainees are not aware of a potential career as a hospitalist clinician investigator due to limited exposure to this career at most AMCs. Our results revealed that nearly half of sites in our survey had zero clinician investigators, depriving trainees at these programs of role models and thus perpetuating a negative feedback loop. Lastly, although unfilled fellowship positions may indicate that demand is a larger problem than supply, it is also true that fellowship programs generate their own demand through recruitment efforts and the gradual establishment of a positive reputation.

Another potential explanation could relate to the development of hospital medicine in response to rising clinical demands at hospitals: compared with other medical specialties, AMCs may regard hospitalists as being clinicians first and academicians second.^1,7,10 Also, hospitalists may be perceived as being beholden to hospitals and less engaged with their surrounding communities than other general medicine fields. With a small footprint in health equity research, academic hospital medicine may be less of a draw to generalists interested in pursuing this area of research. Further, there are very few underrepresented in medicine (URiM) hospital medicine research faculty.⁵

Another challenge to the career development of hospitalist researchers is the lack of available funding for the type of research typically conducted by hospitalists (eg, rigorous quality improvement implementation and evaluation, optimizing best evidence-based care delivery models, evaluation of patient safety in the hospital setting). As hospitalists tend to be system-level thinkers, this lack of funding may steer potential researchers away from externally funded research careers and into hospital operations and quality improvement positions. Also, unlike other medical specialties, there is no dedicated NIH funding source for hospital medicine research (eg, cardiology and the National Heart, Lung, and Blood Institute), placing hospitalists at a disadvantage in seeking funding compared to subspecialists.

We recommend several approaches—ones that should be pursued simultaneously—to increase the number of clinician investigators in hospital medicine. First, hospital medicine groups and their respective divisions, departments, and hospitals should allocate funding to support research resources; this includes investing in research assistants, data analysts, statisticians, and administrative support. Through the funding of such research infrastructure programs, AMCs could incentivize hospitalists to research best approaches to improve the value of healthcare delivery, ultimately leading to cost savings.

With 60% of respondents identifying the need for improved access to data across multiple sites, our survey also emphasizes the requirement for further collaboration among hospital medicine groups. Such collaboration could lead to high-powered observational studies and the evaluation of interventions across multiple sites, thus improving the generalizability of study findings.

The Society of Hospital Medicine (SHM) and its research committee can continue to expand the research footprint of hospital medicine. To date, the committee has achieved this by highlighting hospitalist research activity at the SHM Annual Conference Scientific Abstract and Poster Competition and developing a visiting professorship exchange program. In addition to these efforts, SHM could foster collaboration and networking between institutions, as well as take advantage of the current political push for expanded Medicare access by lobbying for robust funding for the Agency for Healthcare Research and Quality, which could provide more opportunities for hospitalists to study the effects of healthcare policy reform on the delivery of inpatient care.

Another strategy to increase the number of hospitalist clinician investigators is to expand hospital medicine research fellowships and recruit trainees for these programs. Fellowships could be internally funded wherein a fellow’s clinical productivity is used to offset the costs associated with obtaining advanced degrees. As an incentive to encourage applicants to temporarily forego a full-time clinical salary during fellowship, hospital medicine groups could offer expanded moonlighting opportunities and contribute to repayment of medical school loans. Hospital medicine groups should also advocate for NIH-funded T32 or K12 training grants for hospital medicine. (There are, however, challenges with this approach because the number of T32 spots per NIH institute is usually fixed). The success of academic emergency medicine offers a precedent for such efforts: After the development of a K12 research training program in emergency medicine, the number of NIH-sponsored principal investigators in this specialty increased by 40% in 6 years.¹⁴ Additionally, now that fellowships are required for the pediatric hospital medicine clinician investigators, it would be revealing to track the growth of this workforce.^12,13

Structured and formalized mentorship is an essential part of the development of clinician investigators in hospital medicine.^4,7,8,10 One successful strategy for mentorship has been the partnering of hospital medicine groups with faculty of general internal medicine and other subspecialty divisions with robust research programs.^7,8,15 In addition to developing sustainable mentorship programs, hospital medicine researchers must increase their visibility to trainees. Therefore, it is essential that the majority of academic hospital medicine groups not only hire clinician investigators but also invest in their development, rather than rely on the few programs that have several such faculty members. With this strategy, we could dramatically increase the number of hospitalist clinician investigators from a diverse background of training institutions.

SHM could also play a greater role in organizing events for networking and mentoring for trainees and medical students interested in pursuing a career in hospital medicine research. It is also critically important that hospital medicine groups actively recruit, retain, and develop URiM hospital medicine research faculty in order to attract talented researchers and actively participate in the necessary effort to mitigate the inequities prevalent throughout our healthcare system.

Despite the growth of hospital medicine over the past decade, there remains a dearth of hospitalist clinician investigators at major AMCs in the United States. This may be due in part to lack of research resources and mentorship within hospital medicine groups. We believe that investment in these resources, expanded funding opportunities, mentorship development, research fellowship programs, and greater exposure of trainees to hospitalist researchers are solutions that should be strongly considered to develop hospitalist clinician investigators.

The authors thank HOMERuN executive committee members, including Grant Fletcher, MD, James Harrison, PhD, BSC, MPH, Peter K. Lindenauer, MD, Melissa Mattison, MD, David Meltzer, MD, PhD, Joshua Metlay, MD, PhD, Jennifer Myers, MD, Sumant Ranji, MD, Gregory Ruhnke, MD, MPH, Edmondo Robinson, MD, MBA, and Neil Sehgal, MPH PhD, for their assistance in developing the survey. They also thank Tiffany Lee, MA, for her project management assistance for HOMERuN.

In their report celebrating the increase in the number of hospitalists from a few hundred in the 1990s to more than 50,000 in 2016, Drs Robert Wachter and Lee Goldman also noted the stunted growth of productive hospital medicine research programs, which presents a challenge to academic credibility in hospital medicine.¹ Given the substantial increase in the number of hospitalists over the past two decades, we surveyed adult academic hospital medicine groups to quantify the number of hospitalist clinician investigators and identify gaps in resources for researchers. The number of clinician investigators supported at academic medical centers (AMCs) remains disturbingly low despite the rapid growth of our specialty. Some programs also reported a lack of access to fundamental research services. We report selected results from our survey and provide recommendations to support and facilitate the development of clinician investigators in hospital medicine.

We performed a survey of hospital medicine programs at AMCs in the United States through the Hospital Medicine Reengineering Network (HOMERuN), a hospital medicine research collaborative that facilitates and conducts multisite research studies.² The purpose of this survey was to obtain a profile of adult academic hospital medicine groups. Surveys were distributed via email to directors and/or senior leaders of each hospital medicine group between January and August 2019. In the survey, a clinician investigator was defined as “faculty whose primary nonclinical focus is scientific papers and grant writing.”

We received responses from 43 of the 86 invitees (50%), each of whom represented a unique hospital medicine group; 41 of the representatives responded to the questions concerning available research services. Collectively, these 43 programs represented 2,503 hospitalists. There were 79 clinician investigators reported among all surveyed hospital medicine groups (3.1% of all hospitalists). The median number of clinician investigators per hospital medicine group was 0 (range 0-12) (Appendix Figure 1), and 22 of 43 (51.2%) hospital medicine groups reported having no clinician investigators. Two of the hospital medicine groups, however, reported having 12 clinician investigators at their respective institutions, comprising nearly one third of the total number of clinician investigators reported in the survey.

Many of the programs reported lack of access to resources such as research assistants (56.1%) and dedicated research fellowships (53.7%) (Appendix Figure 2). A number of groups reported a need for more support for various junior faculty development activities, including research mentoring (53.5%), networking with other researchers (60.5%), and access to clinical data from multiple sites (62.8%).

One of the limitations of this survey was the manner in which the participating hospital medicine groups were chosen. Selection was based on groups affiliated with HOMERuN; among those chosen were highly visible US AMCs, including 70% of the top 20 AMCs based on National Institutes of Health (NIH) funding.³ Therefore, our results likely overestimate the research presence of hospital medicine across all AMCs in the United States.

Despite the substantial growth of hospital medicine over the past 2 decades, there has been no proportional increase in the number of hospitalist clinician investigators, with earlier surveys also demonstrating low numbers.^4,5 Along with the survey by Chopra and colleagues published in 2019,⁶ our survey provides an additional contemporary appraisal of research activities for adult academic hospital medicine groups. In the survey by Chopra et al, only 54% (15 of 28) of responding programs reported having any faculty with research as their major activity (ie, >50% effort), and 3% of total faculty reported having funding for >50% effort toward research.⁶ Our study expands upon these findings by providing more detailed data on the number of clinician investigators per hospital medicine group. Results of our survey showed a concentration of hospitalists within a small number of programs, which may have contributed to the observed lack of growth. We also expand on prior work by identifying a lack of resources and services to support hospitalist researchers.

The findings of our survey have important implications for the field of hospital medicine. Without a critical mass of hospitalist clinician investigators, the quality of research that addresses important questions in our field will suffer. It will also limit academic credibility of the field, as well as individual academic achievement; previous studies have consistently demonstrated that few hospitalists at AMCs achieve the rank of associate or full professor.^5-9

The results of our study additionally offer possible explanations for the dearth of clinician investigators in hospital medicine. The limited access to research resources and fellowship training identified in our survey are critical domains that must be addressed in order to develop successful academic hospital medicine programs.^4,6,8,10

Regarding dedicated hospital medicine research fellowships, there are only a handful across the country. The small number of existing research fellowships only have one or two fellows per year, and these positions often go unfilled because of a lack of applicants and lower salaries compared to full-time clinical positions.¹¹ The lack of applicants for adult hospital medicine fellowship positions is also integrally linked to board certification requirements. Unlike pediatric hospital medicine where additional fellowship training is required to become board-certified, no such fellowship is required in adult hospital medicine. In pediatrics, this requirement has led to a rapid increase in the number of fellowships with scholarly work requirements (more than 60 fellowships, plus additional programs in development) and greater standardization among training experiences.^12,13

The lack of fellowship applicants may also stem from the fact that many trainees are not aware of a potential career as a hospitalist clinician investigator due to limited exposure to this career at most AMCs. Our results revealed that nearly half of sites in our survey had zero clinician investigators, depriving trainees at these programs of role models and thus perpetuating a negative feedback loop. Lastly, although unfilled fellowship positions may indicate that demand is a larger problem than supply, it is also true that fellowship programs generate their own demand through recruitment efforts and the gradual establishment of a positive reputation.

Another potential explanation could relate to the development of hospital medicine in response to rising clinical demands at hospitals: compared with other medical specialties, AMCs may regard hospitalists as being clinicians first and academicians second.^1,7,10 Also, hospitalists may be perceived as being beholden to hospitals and less engaged with their surrounding communities than other general medicine fields. With a small footprint in health equity research, academic hospital medicine may be less of a draw to generalists interested in pursuing this area of research. Further, there are very few underrepresented in medicine (URiM) hospital medicine research faculty.⁵

Another challenge to the career development of hospitalist researchers is the lack of available funding for the type of research typically conducted by hospitalists (eg, rigorous quality improvement implementation and evaluation, optimizing best evidence-based care delivery models, evaluation of patient safety in the hospital setting). As hospitalists tend to be system-level thinkers, this lack of funding may steer potential researchers away from externally funded research careers and into hospital operations and quality improvement positions. Also, unlike other medical specialties, there is no dedicated NIH funding source for hospital medicine research (eg, cardiology and the National Heart, Lung, and Blood Institute), placing hospitalists at a disadvantage in seeking funding compared to subspecialists.

We recommend several approaches—ones that should be pursued simultaneously—to increase the number of clinician investigators in hospital medicine. First, hospital medicine groups and their respective divisions, departments, and hospitals should allocate funding to support research resources; this includes investing in research assistants, data analysts, statisticians, and administrative support. Through the funding of such research infrastructure programs, AMCs could incentivize hospitalists to research best approaches to improve the value of healthcare delivery, ultimately leading to cost savings.

With 60% of respondents identifying the need for improved access to data across multiple sites, our survey also emphasizes the requirement for further collaboration among hospital medicine groups. Such collaboration could lead to high-powered observational studies and the evaluation of interventions across multiple sites, thus improving the generalizability of study findings.

The Society of Hospital Medicine (SHM) and its research committee can continue to expand the research footprint of hospital medicine. To date, the committee has achieved this by highlighting hospitalist research activity at the SHM Annual Conference Scientific Abstract and Poster Competition and developing a visiting professorship exchange program. In addition to these efforts, SHM could foster collaboration and networking between institutions, as well as take advantage of the current political push for expanded Medicare access by lobbying for robust funding for the Agency for Healthcare Research and Quality, which could provide more opportunities for hospitalists to study the effects of healthcare policy reform on the delivery of inpatient care.

Another strategy to increase the number of hospitalist clinician investigators is to expand hospital medicine research fellowships and recruit trainees for these programs. Fellowships could be internally funded wherein a fellow’s clinical productivity is used to offset the costs associated with obtaining advanced degrees. As an incentive to encourage applicants to temporarily forego a full-time clinical salary during fellowship, hospital medicine groups could offer expanded moonlighting opportunities and contribute to repayment of medical school loans. Hospital medicine groups should also advocate for NIH-funded T32 or K12 training grants for hospital medicine. (There are, however, challenges with this approach because the number of T32 spots per NIH institute is usually fixed). The success of academic emergency medicine offers a precedent for such efforts: After the development of a K12 research training program in emergency medicine, the number of NIH-sponsored principal investigators in this specialty increased by 40% in 6 years.¹⁴ Additionally, now that fellowships are required for the pediatric hospital medicine clinician investigators, it would be revealing to track the growth of this workforce.^12,13

Structured and formalized mentorship is an essential part of the development of clinician investigators in hospital medicine.^4,7,8,10 One successful strategy for mentorship has been the partnering of hospital medicine groups with faculty of general internal medicine and other subspecialty divisions with robust research programs.^7,8,15 In addition to developing sustainable mentorship programs, hospital medicine researchers must increase their visibility to trainees. Therefore, it is essential that the majority of academic hospital medicine groups not only hire clinician investigators but also invest in their development, rather than rely on the few programs that have several such faculty members. With this strategy, we could dramatically increase the number of hospitalist clinician investigators from a diverse background of training institutions.

SHM could also play a greater role in organizing events for networking and mentoring for trainees and medical students interested in pursuing a career in hospital medicine research. It is also critically important that hospital medicine groups actively recruit, retain, and develop URiM hospital medicine research faculty in order to attract talented researchers and actively participate in the necessary effort to mitigate the inequities prevalent throughout our healthcare system.

Despite the growth of hospital medicine over the past decade, there remains a dearth of hospitalist clinician investigators at major AMCs in the United States. This may be due in part to lack of research resources and mentorship within hospital medicine groups. We believe that investment in these resources, expanded funding opportunities, mentorship development, research fellowship programs, and greater exposure of trainees to hospitalist researchers are solutions that should be strongly considered to develop hospitalist clinician investigators.

The authors thank HOMERuN executive committee members, including Grant Fletcher, MD, James Harrison, PhD, BSC, MPH, Peter K. Lindenauer, MD, Melissa Mattison, MD, David Meltzer, MD, PhD, Joshua Metlay, MD, PhD, Jennifer Myers, MD, Sumant Ranji, MD, Gregory Ruhnke, MD, MPH, Edmondo Robinson, MD, MBA, and Neil Sehgal, MPH PhD, for their assistance in developing the survey. They also thank Tiffany Lee, MA, for her project management assistance for HOMERuN.

In their report celebrating the increase in the number of hospitalists from a few hundred in the 1990s to more than 50,000 in 2016, Drs Robert Wachter and Lee Goldman also noted the stunted growth of productive hospital medicine research programs, which presents a challenge to academic credibility in hospital medicine.¹ Given the substantial increase in the number of hospitalists over the past two decades, we surveyed adult academic hospital medicine groups to quantify the number of hospitalist clinician investigators and identify gaps in resources for researchers. The number of clinician investigators supported at academic medical centers (AMCs) remains disturbingly low despite the rapid growth of our specialty. Some programs also reported a lack of access to fundamental research services. We report selected results from our survey and provide recommendations to support and facilitate the development of clinician investigators in hospital medicine.

We performed a survey of hospital medicine programs at AMCs in the United States through the Hospital Medicine Reengineering Network (HOMERuN), a hospital medicine research collaborative that facilitates and conducts multisite research studies.² The purpose of this survey was to obtain a profile of adult academic hospital medicine groups. Surveys were distributed via email to directors and/or senior leaders of each hospital medicine group between January and August 2019. In the survey, a clinician investigator was defined as “faculty whose primary nonclinical focus is scientific papers and grant writing.”

We received responses from 43 of the 86 invitees (50%), each of whom represented a unique hospital medicine group; 41 of the representatives responded to the questions concerning available research services. Collectively, these 43 programs represented 2,503 hospitalists. There were 79 clinician investigators reported among all surveyed hospital medicine groups (3.1% of all hospitalists). The median number of clinician investigators per hospital medicine group was 0 (range 0-12) (Appendix Figure 1), and 22 of 43 (51.2%) hospital medicine groups reported having no clinician investigators. Two of the hospital medicine groups, however, reported having 12 clinician investigators at their respective institutions, comprising nearly one third of the total number of clinician investigators reported in the survey.

Many of the programs reported lack of access to resources such as research assistants (56.1%) and dedicated research fellowships (53.7%) (Appendix Figure 2). A number of groups reported a need for more support for various junior faculty development activities, including research mentoring (53.5%), networking with other researchers (60.5%), and access to clinical data from multiple sites (62.8%).

One of the limitations of this survey was the manner in which the participating hospital medicine groups were chosen. Selection was based on groups affiliated with HOMERuN; among those chosen were highly visible US AMCs, including 70% of the top 20 AMCs based on National Institutes of Health (NIH) funding.³ Therefore, our results likely overestimate the research presence of hospital medicine across all AMCs in the United States.

Despite the substantial growth of hospital medicine over the past 2 decades, there has been no proportional increase in the number of hospitalist clinician investigators, with earlier surveys also demonstrating low numbers.^4,5 Along with the survey by Chopra and colleagues published in 2019,⁶ our survey provides an additional contemporary appraisal of research activities for adult academic hospital medicine groups. In the survey by Chopra et al, only 54% (15 of 28) of responding programs reported having any faculty with research as their major activity (ie, >50% effort), and 3% of total faculty reported having funding for >50% effort toward research.⁶ Our study expands upon these findings by providing more detailed data on the number of clinician investigators per hospital medicine group. Results of our survey showed a concentration of hospitalists within a small number of programs, which may have contributed to the observed lack of growth. We also expand on prior work by identifying a lack of resources and services to support hospitalist researchers.

The findings of our survey have important implications for the field of hospital medicine. Without a critical mass of hospitalist clinician investigators, the quality of research that addresses important questions in our field will suffer. It will also limit academic credibility of the field, as well as individual academic achievement; previous studies have consistently demonstrated that few hospitalists at AMCs achieve the rank of associate or full professor.^5-9

The results of our study additionally offer possible explanations for the dearth of clinician investigators in hospital medicine. The limited access to research resources and fellowship training identified in our survey are critical domains that must be addressed in order to develop successful academic hospital medicine programs.^4,6,8,10

Regarding dedicated hospital medicine research fellowships, there are only a handful across the country. The small number of existing research fellowships only have one or two fellows per year, and these positions often go unfilled because of a lack of applicants and lower salaries compared to full-time clinical positions.¹¹ The lack of applicants for adult hospital medicine fellowship positions is also integrally linked to board certification requirements. Unlike pediatric hospital medicine where additional fellowship training is required to become board-certified, no such fellowship is required in adult hospital medicine. In pediatrics, this requirement has led to a rapid increase in the number of fellowships with scholarly work requirements (more than 60 fellowships, plus additional programs in development) and greater standardization among training experiences.^12,13

The lack of fellowship applicants may also stem from the fact that many trainees are not aware of a potential career as a hospitalist clinician investigator due to limited exposure to this career at most AMCs. Our results revealed that nearly half of sites in our survey had zero clinician investigators, depriving trainees at these programs of role models and thus perpetuating a negative feedback loop. Lastly, although unfilled fellowship positions may indicate that demand is a larger problem than supply, it is also true that fellowship programs generate their own demand through recruitment efforts and the gradual establishment of a positive reputation.

Another potential explanation could relate to the development of hospital medicine in response to rising clinical demands at hospitals: compared with other medical specialties, AMCs may regard hospitalists as being clinicians first and academicians second.^1,7,10 Also, hospitalists may be perceived as being beholden to hospitals and less engaged with their surrounding communities than other general medicine fields. With a small footprint in health equity research, academic hospital medicine may be less of a draw to generalists interested in pursuing this area of research. Further, there are very few underrepresented in medicine (URiM) hospital medicine research faculty.⁵

Another challenge to the career development of hospitalist researchers is the lack of available funding for the type of research typically conducted by hospitalists (eg, rigorous quality improvement implementation and evaluation, optimizing best evidence-based care delivery models, evaluation of patient safety in the hospital setting). As hospitalists tend to be system-level thinkers, this lack of funding may steer potential researchers away from externally funded research careers and into hospital operations and quality improvement positions. Also, unlike other medical specialties, there is no dedicated NIH funding source for hospital medicine research (eg, cardiology and the National Heart, Lung, and Blood Institute), placing hospitalists at a disadvantage in seeking funding compared to subspecialists.

We recommend several approaches—ones that should be pursued simultaneously—to increase the number of clinician investigators in hospital medicine. First, hospital medicine groups and their respective divisions, departments, and hospitals should allocate funding to support research resources; this includes investing in research assistants, data analysts, statisticians, and administrative support. Through the funding of such research infrastructure programs, AMCs could incentivize hospitalists to research best approaches to improve the value of healthcare delivery, ultimately leading to cost savings.

With 60% of respondents identifying the need for improved access to data across multiple sites, our survey also emphasizes the requirement for further collaboration among hospital medicine groups. Such collaboration could lead to high-powered observational studies and the evaluation of interventions across multiple sites, thus improving the generalizability of study findings.

The Society of Hospital Medicine (SHM) and its research committee can continue to expand the research footprint of hospital medicine. To date, the committee has achieved this by highlighting hospitalist research activity at the SHM Annual Conference Scientific Abstract and Poster Competition and developing a visiting professorship exchange program. In addition to these efforts, SHM could foster collaboration and networking between institutions, as well as take advantage of the current political push for expanded Medicare access by lobbying for robust funding for the Agency for Healthcare Research and Quality, which could provide more opportunities for hospitalists to study the effects of healthcare policy reform on the delivery of inpatient care.

Another strategy to increase the number of hospitalist clinician investigators is to expand hospital medicine research fellowships and recruit trainees for these programs. Fellowships could be internally funded wherein a fellow’s clinical productivity is used to offset the costs associated with obtaining advanced degrees. As an incentive to encourage applicants to temporarily forego a full-time clinical salary during fellowship, hospital medicine groups could offer expanded moonlighting opportunities and contribute to repayment of medical school loans. Hospital medicine groups should also advocate for NIH-funded T32 or K12 training grants for hospital medicine. (There are, however, challenges with this approach because the number of T32 spots per NIH institute is usually fixed). The success of academic emergency medicine offers a precedent for such efforts: After the development of a K12 research training program in emergency medicine, the number of NIH-sponsored principal investigators in this specialty increased by 40% in 6 years.¹⁴ Additionally, now that fellowships are required for the pediatric hospital medicine clinician investigators, it would be revealing to track the growth of this workforce.^12,13

Structured and formalized mentorship is an essential part of the development of clinician investigators in hospital medicine.^4,7,8,10 One successful strategy for mentorship has been the partnering of hospital medicine groups with faculty of general internal medicine and other subspecialty divisions with robust research programs.^7,8,15 In addition to developing sustainable mentorship programs, hospital medicine researchers must increase their visibility to trainees. Therefore, it is essential that the majority of academic hospital medicine groups not only hire clinician investigators but also invest in their development, rather than rely on the few programs that have several such faculty members. With this strategy, we could dramatically increase the number of hospitalist clinician investigators from a diverse background of training institutions.

SHM could also play a greater role in organizing events for networking and mentoring for trainees and medical students interested in pursuing a career in hospital medicine research. It is also critically important that hospital medicine groups actively recruit, retain, and develop URiM hospital medicine research faculty in order to attract talented researchers and actively participate in the necessary effort to mitigate the inequities prevalent throughout our healthcare system.

Despite the growth of hospital medicine over the past decade, there remains a dearth of hospitalist clinician investigators at major AMCs in the United States. This may be due in part to lack of research resources and mentorship within hospital medicine groups. We believe that investment in these resources, expanded funding opportunities, mentorship development, research fellowship programs, and greater exposure of trainees to hospitalist researchers are solutions that should be strongly considered to develop hospitalist clinician investigators.

The authors thank HOMERuN executive committee members, including Grant Fletcher, MD, James Harrison, PhD, BSC, MPH, Peter K. Lindenauer, MD, Melissa Mattison, MD, David Meltzer, MD, PhD, Joshua Metlay, MD, PhD, Jennifer Myers, MD, Sumant Ranji, MD, Gregory Ruhnke, MD, MPH, Edmondo Robinson, MD, MBA, and Neil Sehgal, MPH PhD, for their assistance in developing the survey. They also thank Tiffany Lee, MA, for her project management assistance for HOMERuN.

Transitions from the hospital to the ambulatory setting are high-risk periods for patients in terms of adverse events, poor clinical outcomes, and readmission. Processes of care during care transitions are suboptimal, including poor communication among inpatient providers, patients, and ambulatory providers^1,2; suboptimal communication of postdischarge plans of care to patients and their ability to carry out these plans³; medication discrepancies and nonadherence after discharge⁴; and lack of timely follow-up with ambulatory providers.⁵ Healthcare organizations continue to struggle with the question of which interventions to implement and how best to implement them.

Interventions to improve care transitions typically focus on readmission rates, but some studies have focused on postdischarge adverse events, defined as injuries in the 30 days after discharge caused by medical management rather than underlying disease processes.² These adverse events cause psychological distress, out-of-pocket expenses, decreases in functional status, and caregiver burden. An estimated 20% of hospitalized patients suffer a postdischarge adverse event.^1,2 Approximately two-thirds of these may be preventable or ameliorable.

The advent of Accountable Care Organizations (ACOs), defined as “groups of doctors, hospitals, and other health care providers who come together voluntarily to give coordinated high quality care to their patients,” creates an opportunity for improvements in patient safety during care transitions.⁶ Another opportunity has been the advent of Patient-Centered Medical Homes (PCMH), consisting of patient-oriented, comprehensive, team-based primary care enhanced by health information technology and population-based disease management tools.^7,8 In theory, a hospital-PCMH collaboration within an ACO can improve transitional interventions since optimal communication and collaboration are more likely when both inpatient and primary care providers (PCPs) share infrastructure and are similarly incentivized. The objectives of this study were to design and implement a collaborative hospital-PCMH care transitions intervention within an ACO and evaluate its effects.

This study was a two-arm, single-blind (blinded outcomes assessor), stepped-wedge, multisite cluster-randomized clinical trial (NCT02130570) approved by the institutional review board of Partners HealthCare.

Study Design and Randomization

The study employed a “stepped-wedge” design, which is a cluster-randomized study design in which an intervention is sequentially rolled out to different groups at different, prespecified, randomly determined times.⁹ Each cluster (in this case, each primary care practice) served as its own control, while still allowing for adjustment for temporal trends. Originally, 18 practices participated, but one withdrew due to the low number of patients enrolled in the study, leaving 17 clusters and 16 sequences; see Figure 1 of Appendix 1 for a full description of the sample size and timeline for each cluster. Practices were not aware of this timeline until after recruitment.

Study Setting and Participants

Conducted within a large Pioneer ACO in Boston and funded by the Patient-Centered Outcomes Research Institute (PCORI), the Partners-PCORI Transitions Study was designed as a “real-world” quality improvement project. Potential participants were adult patients who were admitted to medical and surgical services of two large academic hospitals (Hospital A and Hospital B) affiliated with an ACO, who were likely to be discharged back to the community, and whose PCP belonged to a primary care practice that was affiliated with the ACO, agreed to participate, and were designated PCMHs or on their way to being designated by meeting certain criteria: electronic health record, patient portal, team-based care, practice redesign, care management, and identification of high-risk patients. See Study Protocol (Appendix 2) for detailed patient and primary care practice inclusion criteria.

Patient Enrollment

Study staff screened participants from a daily automated list of patients admitted the day before, using medical records to determine eligibility, which was then confirmed by the patient’s nurse. Exclusion criteria included likely discharge to a location other than home, being in police custody, lack of a telephone, being homeless, previous enrollment in the study, and being unable to communicate in English or Spanish. Allocation to study arm was concealed until the patient or proxy provided informed written consent. The research assistant administered questionnaires to all study subjects to assess potential confounders and functional status 1 month prior to admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF-12]).¹⁰ Patients were recruited between March 2013 and October 2015.

Intervention

The intervention was based on a conceptual model of an ideal discharge¹¹ that we developed based on work by Naylor et al,¹² work by Coleman and Berenson,³ best practices in medication reconciliation and information transfer according to our own research,^13-15 the best examples of interventions to improve the discharge process,^12,16,17 and a systematic review of discharge interventions.¹⁸ Some of the factors necessary for an ideal care transition include complete, organized, and timely documentation of the patient’s hospital course and postdischarge plan; effective discharge planning; coordination of care among the patient’s providers; methods to ensure medication safety; advanced care planning in appropriate patients; and education and “coaching” of patients and their caregivers so they learn how to manage their conditions. The final multifaceted intervention addressed each component of the ideal discharge and included inpatient and outpatient components (Table 1 and Table 1 of Appendix 1).

Patient and Public Involvement in Research

As with all PCORI-funded studies, this study involved a patient-family advisory council (PFAC). Our PFAC included six recently hospitalized patients or caregivers of recently hospitalized patients. The PFAC participated in monthly meetings throughout the study period. They helped inform the research questions, including confirmation that the endpoints were patient centered, and provided valuable input for the design of the intervention and the patient-facing components of the data collection instruments. They also interviewed several patient participants in the study regarding their experiences with the intervention. Lastly, they helped develop plans for dissemination of study results to the public.¹⁹

We also formed a steering committee consisting of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at both hospitals and Partners Healthcare. PFAC members took turns participating in quarterly steering committee meetings.

Evolution of the Intervention and Implementation

The intervention was iteratively refined during the course of the study in response to input from the PFAC, steering committee, and members of the intervention team; cases of adverse events and readmissions from patients despite being in the intervention arm; exit interviews of patients who had recently completed the intervention; and informal feedback from inpatient and outpatient clinicians. For example, we learned that the more complicated a patient’s conditions are, the sooner the clinical team wanted them to be seen after discharge. However, these patients were also less likely to feel well enough to keep that appointment. Therefore, the timing of follow-up of appointments needed to be a negotiation among the inpatient team, the patient, any caregivers, and the outpatient provider. PFAC members also emphasized that patients wanted one person to trust and to be the “point person” during a complicated transition such as hospital discharge.

At the same time, the intervention components evolved because of factors outside our control (eg, resource limitations). In keeping with the real-world nature of the research, the aim was for the intervention to be internally supported because incentives were theoretically more aligned with improvement of care transitions under the ACO model. By design, the PCORI contract only paid for limited parts of the intervention, such as a nurse practitioner to act as the discharge advocate at one hospital, overtime costs of inpatient pharmacists, and project manager time to facilitate inpatient-outpatient provider communication. (See Table 1 of Appendix 1 for details about the modifications to the intervention.)

Lastly, in keeping with PCORI’s methodology standards for studies of complex interventions,²⁰ we strove to standardize the intervention by function across hospitals, units, and practices, while still allowing for local adaptation in the form. In other words, rather than specifying exactly how a task (eg, medication counseling) needed to be performed, the study design offered sites flexibility in how they implemented the task given their available personnel and institutional culture.

Intervention Fidelity

To determine the extent to which each patient in the intervention arm received each intervention component, a project manager unblinded to treatment arm reviewed the electronic medical record for documentation of each component implemented by providers (eg, inpatient pharmacists, outpatient nurses). Because each intervention component produced documentation, this provided an accurate assessment of intervention fidelity, ie, the extent to which the intervention was implemented as intended.

Outcome Assessment

Postdischarge Follow-up

Based on previous studies,^2,21 a trained research assistant attempted to contact all study subjects 30 days (±5 days) after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any healthcare use since discharge, and functional status in the previous week. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms to medications or other aspects of medical management. Research assistants followed up any positive responses with directed medical record review for objective findings, diagnoses, treatments, and responses. If patients could not be reached after five attempts, the research assistant instead conducted a thorough review of the outpatient medical record alone for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results in all patients for evidence of postdischarge renal failure, elevated liver function tests, or new/worsening anemia.

Hospital Readmissions

We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for hospitalizations within the ACO network plus patient report during the 30-day phone call for all other readmissions.²²

Adjudication of Outcomes

Adverse events and preventable adverse events: All cases of new or worsening symptoms or signs, along with all supporting documentation, were then presented to teams of two trained blinded physician adjudicators through application of methods established in previous studies.^4,21 Each of the two adjudicators independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury due to medical management) and to classify the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, and whether the event was preventable or ameliorable. The two adjudicators then met to resolve any differences in their findings and come to consensus.

Preventable readmissions: If patients were readmitted to either study hospital, we conducted an evaluation, based on previous studies,²³ to determine if and how the readmission could have been prevented including (a) a standardized patient and caregiver interview to identify possible problems with the transitions process and (b) an email questionnaire to the patient’s PCP and the inpatient teams who cared for the patient during the index admission and readmission regarding possible deficiencies with the transitions process. As with adverse event adjudications, two physician adjudicators worked independently to classify the preventability of the readmission and then met to come to consensus. Conflicts were resolved by a third adjudicator.

Analysis Plan

To evaluate the effects of the intervention on the primary outcome, the number of postdischarge adverse events per patient, we used multivariable Poisson regression, with study arm as the main predictor. A similar approach was used to evaluate the number of new or worsening postdischarge signs or symptoms and the number of preventable adverse events per patient. We used an intention-to-treat analysis: If a practice did not start the intervention when they were scheduled to, based on our randomization, we counted all patients in that practice admitted after that point as intervention patients. We adjusted for patient demographics, clinical characteristics, month, inpatient unit, and primary care practice as fixed effects. We clustered by PCP using general linear models. Intervention effects were expressed as both unadjusted and adjusted incidence rate ratios (IRRs). We also conducted a limited number of subgroup analyses, determined a priori, to determine whether the intervention was more effective in certain patient populations; we used interaction terms (intervention × subgroup) to determine the statistical significance of any effect modification.

To evaluate the effects of the intervention on nonelective readmissions and preventable readmissions, we used a similar approach, using multivariable logistic regression. Postdischarge functional status, adjusted for status prior to admission, was analyzed using multivariable linear regression and random effects by primary care practice. The general linear mixed model (GLIMMIX) procedure in the SAS 9.3 statistical package (SAS Institute) was used to carry out all analyses.

Power and Sample Size

We assumed a baseline rate of postdischarge adverse events of 0.30 per patient.²¹ We conservatively assumed an effect size of a change from 0.30 in the control group to 0.23 in the intervention group (a relative reduction of 22%, which was based on studies of preventability rates²³ and close to the minimum clinically important difference). Based on prior studies,^4,22 we assumed an intraclass correlation coefficient of 0.01 with an average cluster size of seven patients per PCP. Assuming a 10% loss to follow-up rate and an alpha of 0.05, we targeted a sample size of 1,800 patients to achieve 80% power, with one-third of the patients in the usual care arm and two-thirds in the intervention arm.

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from Hospital A (out of 13 approached), 8 from Hospital B (out of 11), and 2 from other ACO practices (out of 9) (plus two pilot practices). Reasons for not participating included not having dedicated personnel to play the role of the responsible outpatient clinician, undergoing recent turn-over in practice leadership, and not having enough patients admitted to the two hospitals. One practice only enrolled 5 patients in the study and withdrew from participation, which left 17 practices.

Study Patients

We enrolled 1,679 patients (Figure 1). Reasons for nonenrollment included being unable to complete the screen prior to discharge, not meeting inclusion criteria or meeting exclusion criteria, being assigned to a pilot practice, and declining informed written consent. The baseline characteristics of enrolled patients are presented in Table 2. Differences between the two study arms were small. About 47% of the cohort was not reachable by phone after five attempts for the 30-day phone call, but only 69 (4.1%) were truly lost to follow-up because they were unreachable by phone and had no documentation in the electronic medical record in the 30-days after discharge.

Intervention Fidelity

The majority of patients did not receive most intervention components, even those components that were supposed to be delivered to all intervention patients (Table 3). A minority of patients were referred to visiting nurse services and to the home pharmacy program. However, 855 patients (87%) in the intervention arm received at least one intervention component.

Outcome Measures

The intervention was associated with a statistically significant reduction in several of the outcomes of interest, including the primary outcome, number of postdischarge adverse events (45% reduction), and new or worsening postdischarge signs or symptoms (22% reduction), as well as preventable postdischarge adverse events (58% reduction) (Table 4). There was a nonsignificant difference in functional status. There was no significant effect on total nonelective or on preventable readmission rates. When analyzed by type of adverse event, the intervention was associated with a reduction in adverse drug events and in procedural complications (Table 2 of Appendix 1). Of note, there was no significant difference in the proportion of patients with at least one adverse event whether the outcome was determined by phone call and medical record review (49%) or medical record review alone (51%) (P = .48).

In subgroup analyses, there was no evidence of effect modification by service, hospital, patient age, readmission risk, health literacy, or comorbidity score (Table 3 of Appendix 1). Table 4 of Appendix 1 provides examples of postdischarge adverse events seen in the usual care arm that might have been prevented in the intervention.

This intervention was associated with a reduction in postdischarge adverse events. The relative improvement in each outcome aligned with the hypothesized sensitivity to change: the smallest improvement was seen in new or worsening signs or symptoms, followed by postdischarge adverse events and then by preventable postdischarge adverse events. The intervention was not associated with a difference in readmissions. The lack of effect on hospital readmissions may have been caused by the low proportion of readmissions that are preventable, as well as low intervention fidelity and lack of resources to implement facets such as postdischarge coaching, an evidence-based intervention that was never adopted.^16,23 One lesson of this study is that it may be easier to reduce postdischarge injury (still an important outcome) than readmissions.

Putting this study in context, we should note that the literature on interventions to improve care transitions is mixed.¹⁸ While there are several reports of successful interventions, there are many reports of unsuccessful ones, often using similar components. Success is often the result of adequate resources and attention to myriad details regarding implementation.²⁴ The intervention in our study likely contributed to improvements in patient and caregiver engagement in the hospital, enhancements of communication between inpatient and outpatient clinicians, and implementation of pharmacist-led interventions to improve medication safety. Regarding the latter, several prior studies have shown the benefits of pharmacist interventions in decreasing postdischarge adverse drug events.^4,25,26 Therefore, even an intervention with incomplete intervention fidelity can reduce postdischarge adverse events, especially because adverse drug events make up the majority of adverse events.^1,2,21

Perhaps the biggest lesson we learned was regarding the limitations of the hospital-led ACO model to incentivize sufficient up-front investments in transitional care interventions. By design, we chose a real-world approach in which interventions were integrated with existing ACO efforts, which were paid for internally by the institution. As a result, many of the interventions had to be scaled back because of resource constraints. The ACO model theoretically incentivizes more integrated care, but this may not always be true in practice. Emerging evidence suggests that physician group–led ACOs are associated with lower costs and use compared with hospital-led ACOs, likely because of more aligned incentives in physician group–led ACOs to reduce use of inpatient care.^27,28

An unresolved question is whether the ideal implementation approach is to protect the time of existing clinical personnel to carry out transitional care tasks or to hire external personnel to do these tasks. We purposely spread the intervention over several clinician types to minimize the additional burden on any one of them, minimize additional costs, and play to each clinician’s expertise, but in retrospect, this may not have been the right approach. By providing additional personnel with dedicated time, interest, and training in care transitions, the intervention may be delivered with higher quantitative and qualitative fidelity, and it could create a single point of contact for patients, which was considered highly desirable by our PFAC.

This study has several limitations. A large proportion of patients (44%) were unavailable for postdischarge phone calls. However, we were able to perform medical record review for worsening signs (eg, lab abnormalities) and symptoms (as reported by patients’ providers) in the postdischarge period and adjudicate them for adverse events for all but 69 of these patients. Because all these patients had ACO-affiliated PCPs, we would expect most of their utilization to have been within the system and, therefore, to be present in the medical record. The proportion of patients with at least one adverse event did not vary by the method of follow-up, which suggests that this issue is an unlikely source of bias. Assessment of readmission was imperfect because we do not have statewide or national data. However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmission has been shown to be fairly complete in previous studies.²⁹ Adjudicators could not be fully blinded to intervention status due to the lack of blinding of admission date. We did not calculate a kappa value for interrater reliability of individual assessments of adverse events; rather, coming to consensus among the two adjudicators was part of the process. In only a handful of cases was a third adjudicator required. Lastly, this study was conducted at two academic medical centers and their affiliated primary care clinics, which potentially limits generalizability; however, the results are likely generalizable to other ACOs that include major academic medical centers.

In conclusion, in this real-world clinical trial, we designed, implemented, and iteratively refined a multifaceted intervention to improve care transitions within a hospital-based academic ACO. Evolution of the intervention components was the result of stakeholder input, experience with the intervention, and ACO resource constraints. The intervention reduced postdischarge adverse events. However, across the ACO network, intervention fidelity was low, and this may have contributed to the lack of effect on readmission rates. ACOs that implement interventions without hiring new personnel or protecting the time of existing personnel to conduct transitional tasks are likely to face the same challenges of low fidelity.

The authors would like to acknowledge the many people who worked on designing, implementing, and evaluating this intervention, including but not limited to: Natasha Isaac, Hilary Heyison, Jacqueline Minahan, Molly O’Reilly, Michelle Potter, Nailah Khoory, Maureen Fagan, David Bates, Laura Carr, Joseph Frolkis, Eric Weil, Jacqueline Somerville, Stephanie Ahmed, Marcy Bergeron, Jessica Smith, and Jane Millett. We would also like to thank the members of our Patient-Family Advisory Council: Maureen Fagan, Karen Spikes, Margie Hodges, Win Hodges, Aureldon Henderson, Dena Salzberg, and Kay Bander.

Transitions from the hospital to the ambulatory setting are high-risk periods for patients in terms of adverse events, poor clinical outcomes, and readmission. Processes of care during care transitions are suboptimal, including poor communication among inpatient providers, patients, and ambulatory providers^1,2; suboptimal communication of postdischarge plans of care to patients and their ability to carry out these plans³; medication discrepancies and nonadherence after discharge⁴; and lack of timely follow-up with ambulatory providers.⁵ Healthcare organizations continue to struggle with the question of which interventions to implement and how best to implement them.

Interventions to improve care transitions typically focus on readmission rates, but some studies have focused on postdischarge adverse events, defined as injuries in the 30 days after discharge caused by medical management rather than underlying disease processes.² These adverse events cause psychological distress, out-of-pocket expenses, decreases in functional status, and caregiver burden. An estimated 20% of hospitalized patients suffer a postdischarge adverse event.^1,2 Approximately two-thirds of these may be preventable or ameliorable.

The advent of Accountable Care Organizations (ACOs), defined as “groups of doctors, hospitals, and other health care providers who come together voluntarily to give coordinated high quality care to their patients,” creates an opportunity for improvements in patient safety during care transitions.⁶ Another opportunity has been the advent of Patient-Centered Medical Homes (PCMH), consisting of patient-oriented, comprehensive, team-based primary care enhanced by health information technology and population-based disease management tools.^7,8 In theory, a hospital-PCMH collaboration within an ACO can improve transitional interventions since optimal communication and collaboration are more likely when both inpatient and primary care providers (PCPs) share infrastructure and are similarly incentivized. The objectives of this study were to design and implement a collaborative hospital-PCMH care transitions intervention within an ACO and evaluate its effects.

This study was a two-arm, single-blind (blinded outcomes assessor), stepped-wedge, multisite cluster-randomized clinical trial (NCT02130570) approved by the institutional review board of Partners HealthCare.

Study Design and Randomization

The study employed a “stepped-wedge” design, which is a cluster-randomized study design in which an intervention is sequentially rolled out to different groups at different, prespecified, randomly determined times.⁹ Each cluster (in this case, each primary care practice) served as its own control, while still allowing for adjustment for temporal trends. Originally, 18 practices participated, but one withdrew due to the low number of patients enrolled in the study, leaving 17 clusters and 16 sequences; see Figure 1 of Appendix 1 for a full description of the sample size and timeline for each cluster. Practices were not aware of this timeline until after recruitment.

Study Setting and Participants

Conducted within a large Pioneer ACO in Boston and funded by the Patient-Centered Outcomes Research Institute (PCORI), the Partners-PCORI Transitions Study was designed as a “real-world” quality improvement project. Potential participants were adult patients who were admitted to medical and surgical services of two large academic hospitals (Hospital A and Hospital B) affiliated with an ACO, who were likely to be discharged back to the community, and whose PCP belonged to a primary care practice that was affiliated with the ACO, agreed to participate, and were designated PCMHs or on their way to being designated by meeting certain criteria: electronic health record, patient portal, team-based care, practice redesign, care management, and identification of high-risk patients. See Study Protocol (Appendix 2) for detailed patient and primary care practice inclusion criteria.

Patient Enrollment

Study staff screened participants from a daily automated list of patients admitted the day before, using medical records to determine eligibility, which was then confirmed by the patient’s nurse. Exclusion criteria included likely discharge to a location other than home, being in police custody, lack of a telephone, being homeless, previous enrollment in the study, and being unable to communicate in English or Spanish. Allocation to study arm was concealed until the patient or proxy provided informed written consent. The research assistant administered questionnaires to all study subjects to assess potential confounders and functional status 1 month prior to admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF-12]).¹⁰ Patients were recruited between March 2013 and October 2015.

Intervention

The intervention was based on a conceptual model of an ideal discharge¹¹ that we developed based on work by Naylor et al,¹² work by Coleman and Berenson,³ best practices in medication reconciliation and information transfer according to our own research,^13-15 the best examples of interventions to improve the discharge process,^12,16,17 and a systematic review of discharge interventions.¹⁸ Some of the factors necessary for an ideal care transition include complete, organized, and timely documentation of the patient’s hospital course and postdischarge plan; effective discharge planning; coordination of care among the patient’s providers; methods to ensure medication safety; advanced care planning in appropriate patients; and education and “coaching” of patients and their caregivers so they learn how to manage their conditions. The final multifaceted intervention addressed each component of the ideal discharge and included inpatient and outpatient components (Table 1 and Table 1 of Appendix 1).

Patient and Public Involvement in Research

As with all PCORI-funded studies, this study involved a patient-family advisory council (PFAC). Our PFAC included six recently hospitalized patients or caregivers of recently hospitalized patients. The PFAC participated in monthly meetings throughout the study period. They helped inform the research questions, including confirmation that the endpoints were patient centered, and provided valuable input for the design of the intervention and the patient-facing components of the data collection instruments. They also interviewed several patient participants in the study regarding their experiences with the intervention. Lastly, they helped develop plans for dissemination of study results to the public.¹⁹

We also formed a steering committee consisting of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at both hospitals and Partners Healthcare. PFAC members took turns participating in quarterly steering committee meetings.

Evolution of the Intervention and Implementation

The intervention was iteratively refined during the course of the study in response to input from the PFAC, steering committee, and members of the intervention team; cases of adverse events and readmissions from patients despite being in the intervention arm; exit interviews of patients who had recently completed the intervention; and informal feedback from inpatient and outpatient clinicians. For example, we learned that the more complicated a patient’s conditions are, the sooner the clinical team wanted them to be seen after discharge. However, these patients were also less likely to feel well enough to keep that appointment. Therefore, the timing of follow-up of appointments needed to be a negotiation among the inpatient team, the patient, any caregivers, and the outpatient provider. PFAC members also emphasized that patients wanted one person to trust and to be the “point person” during a complicated transition such as hospital discharge.

At the same time, the intervention components evolved because of factors outside our control (eg, resource limitations). In keeping with the real-world nature of the research, the aim was for the intervention to be internally supported because incentives were theoretically more aligned with improvement of care transitions under the ACO model. By design, the PCORI contract only paid for limited parts of the intervention, such as a nurse practitioner to act as the discharge advocate at one hospital, overtime costs of inpatient pharmacists, and project manager time to facilitate inpatient-outpatient provider communication. (See Table 1 of Appendix 1 for details about the modifications to the intervention.)

Lastly, in keeping with PCORI’s methodology standards for studies of complex interventions,²⁰ we strove to standardize the intervention by function across hospitals, units, and practices, while still allowing for local adaptation in the form. In other words, rather than specifying exactly how a task (eg, medication counseling) needed to be performed, the study design offered sites flexibility in how they implemented the task given their available personnel and institutional culture.

Intervention Fidelity

To determine the extent to which each patient in the intervention arm received each intervention component, a project manager unblinded to treatment arm reviewed the electronic medical record for documentation of each component implemented by providers (eg, inpatient pharmacists, outpatient nurses). Because each intervention component produced documentation, this provided an accurate assessment of intervention fidelity, ie, the extent to which the intervention was implemented as intended.

Outcome Assessment

Postdischarge Follow-up

Based on previous studies,^2,21 a trained research assistant attempted to contact all study subjects 30 days (±5 days) after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any healthcare use since discharge, and functional status in the previous week. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms to medications or other aspects of medical management. Research assistants followed up any positive responses with directed medical record review for objective findings, diagnoses, treatments, and responses. If patients could not be reached after five attempts, the research assistant instead conducted a thorough review of the outpatient medical record alone for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results in all patients for evidence of postdischarge renal failure, elevated liver function tests, or new/worsening anemia.

Hospital Readmissions

We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for hospitalizations within the ACO network plus patient report during the 30-day phone call for all other readmissions.²²

Adjudication of Outcomes

Adverse events and preventable adverse events: All cases of new or worsening symptoms or signs, along with all supporting documentation, were then presented to teams of two trained blinded physician adjudicators through application of methods established in previous studies.^4,21 Each of the two adjudicators independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury due to medical management) and to classify the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, and whether the event was preventable or ameliorable. The two adjudicators then met to resolve any differences in their findings and come to consensus.

Preventable readmissions: If patients were readmitted to either study hospital, we conducted an evaluation, based on previous studies,²³ to determine if and how the readmission could have been prevented including (a) a standardized patient and caregiver interview to identify possible problems with the transitions process and (b) an email questionnaire to the patient’s PCP and the inpatient teams who cared for the patient during the index admission and readmission regarding possible deficiencies with the transitions process. As with adverse event adjudications, two physician adjudicators worked independently to classify the preventability of the readmission and then met to come to consensus. Conflicts were resolved by a third adjudicator.

Analysis Plan

To evaluate the effects of the intervention on the primary outcome, the number of postdischarge adverse events per patient, we used multivariable Poisson regression, with study arm as the main predictor. A similar approach was used to evaluate the number of new or worsening postdischarge signs or symptoms and the number of preventable adverse events per patient. We used an intention-to-treat analysis: If a practice did not start the intervention when they were scheduled to, based on our randomization, we counted all patients in that practice admitted after that point as intervention patients. We adjusted for patient demographics, clinical characteristics, month, inpatient unit, and primary care practice as fixed effects. We clustered by PCP using general linear models. Intervention effects were expressed as both unadjusted and adjusted incidence rate ratios (IRRs). We also conducted a limited number of subgroup analyses, determined a priori, to determine whether the intervention was more effective in certain patient populations; we used interaction terms (intervention × subgroup) to determine the statistical significance of any effect modification.

To evaluate the effects of the intervention on nonelective readmissions and preventable readmissions, we used a similar approach, using multivariable logistic regression. Postdischarge functional status, adjusted for status prior to admission, was analyzed using multivariable linear regression and random effects by primary care practice. The general linear mixed model (GLIMMIX) procedure in the SAS 9.3 statistical package (SAS Institute) was used to carry out all analyses.

Power and Sample Size

We assumed a baseline rate of postdischarge adverse events of 0.30 per patient.²¹ We conservatively assumed an effect size of a change from 0.30 in the control group to 0.23 in the intervention group (a relative reduction of 22%, which was based on studies of preventability rates²³ and close to the minimum clinically important difference). Based on prior studies,^4,22 we assumed an intraclass correlation coefficient of 0.01 with an average cluster size of seven patients per PCP. Assuming a 10% loss to follow-up rate and an alpha of 0.05, we targeted a sample size of 1,800 patients to achieve 80% power, with one-third of the patients in the usual care arm and two-thirds in the intervention arm.

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from Hospital A (out of 13 approached), 8 from Hospital B (out of 11), and 2 from other ACO practices (out of 9) (plus two pilot practices). Reasons for not participating included not having dedicated personnel to play the role of the responsible outpatient clinician, undergoing recent turn-over in practice leadership, and not having enough patients admitted to the two hospitals. One practice only enrolled 5 patients in the study and withdrew from participation, which left 17 practices.

Study Patients

We enrolled 1,679 patients (Figure 1). Reasons for nonenrollment included being unable to complete the screen prior to discharge, not meeting inclusion criteria or meeting exclusion criteria, being assigned to a pilot practice, and declining informed written consent. The baseline characteristics of enrolled patients are presented in Table 2. Differences between the two study arms were small. About 47% of the cohort was not reachable by phone after five attempts for the 30-day phone call, but only 69 (4.1%) were truly lost to follow-up because they were unreachable by phone and had no documentation in the electronic medical record in the 30-days after discharge.

Intervention Fidelity

The majority of patients did not receive most intervention components, even those components that were supposed to be delivered to all intervention patients (Table 3). A minority of patients were referred to visiting nurse services and to the home pharmacy program. However, 855 patients (87%) in the intervention arm received at least one intervention component.

Outcome Measures

The intervention was associated with a statistically significant reduction in several of the outcomes of interest, including the primary outcome, number of postdischarge adverse events (45% reduction), and new or worsening postdischarge signs or symptoms (22% reduction), as well as preventable postdischarge adverse events (58% reduction) (Table 4). There was a nonsignificant difference in functional status. There was no significant effect on total nonelective or on preventable readmission rates. When analyzed by type of adverse event, the intervention was associated with a reduction in adverse drug events and in procedural complications (Table 2 of Appendix 1). Of note, there was no significant difference in the proportion of patients with at least one adverse event whether the outcome was determined by phone call and medical record review (49%) or medical record review alone (51%) (P = .48).

In subgroup analyses, there was no evidence of effect modification by service, hospital, patient age, readmission risk, health literacy, or comorbidity score (Table 3 of Appendix 1). Table 4 of Appendix 1 provides examples of postdischarge adverse events seen in the usual care arm that might have been prevented in the intervention.

This intervention was associated with a reduction in postdischarge adverse events. The relative improvement in each outcome aligned with the hypothesized sensitivity to change: the smallest improvement was seen in new or worsening signs or symptoms, followed by postdischarge adverse events and then by preventable postdischarge adverse events. The intervention was not associated with a difference in readmissions. The lack of effect on hospital readmissions may have been caused by the low proportion of readmissions that are preventable, as well as low intervention fidelity and lack of resources to implement facets such as postdischarge coaching, an evidence-based intervention that was never adopted.^16,23 One lesson of this study is that it may be easier to reduce postdischarge injury (still an important outcome) than readmissions.

Putting this study in context, we should note that the literature on interventions to improve care transitions is mixed.¹⁸ While there are several reports of successful interventions, there are many reports of unsuccessful ones, often using similar components. Success is often the result of adequate resources and attention to myriad details regarding implementation.²⁴ The intervention in our study likely contributed to improvements in patient and caregiver engagement in the hospital, enhancements of communication between inpatient and outpatient clinicians, and implementation of pharmacist-led interventions to improve medication safety. Regarding the latter, several prior studies have shown the benefits of pharmacist interventions in decreasing postdischarge adverse drug events.^4,25,26 Therefore, even an intervention with incomplete intervention fidelity can reduce postdischarge adverse events, especially because adverse drug events make up the majority of adverse events.^1,2,21

Perhaps the biggest lesson we learned was regarding the limitations of the hospital-led ACO model to incentivize sufficient up-front investments in transitional care interventions. By design, we chose a real-world approach in which interventions were integrated with existing ACO efforts, which were paid for internally by the institution. As a result, many of the interventions had to be scaled back because of resource constraints. The ACO model theoretically incentivizes more integrated care, but this may not always be true in practice. Emerging evidence suggests that physician group–led ACOs are associated with lower costs and use compared with hospital-led ACOs, likely because of more aligned incentives in physician group–led ACOs to reduce use of inpatient care.^27,28

An unresolved question is whether the ideal implementation approach is to protect the time of existing clinical personnel to carry out transitional care tasks or to hire external personnel to do these tasks. We purposely spread the intervention over several clinician types to minimize the additional burden on any one of them, minimize additional costs, and play to each clinician’s expertise, but in retrospect, this may not have been the right approach. By providing additional personnel with dedicated time, interest, and training in care transitions, the intervention may be delivered with higher quantitative and qualitative fidelity, and it could create a single point of contact for patients, which was considered highly desirable by our PFAC.

This study has several limitations. A large proportion of patients (44%) were unavailable for postdischarge phone calls. However, we were able to perform medical record review for worsening signs (eg, lab abnormalities) and symptoms (as reported by patients’ providers) in the postdischarge period and adjudicate them for adverse events for all but 69 of these patients. Because all these patients had ACO-affiliated PCPs, we would expect most of their utilization to have been within the system and, therefore, to be present in the medical record. The proportion of patients with at least one adverse event did not vary by the method of follow-up, which suggests that this issue is an unlikely source of bias. Assessment of readmission was imperfect because we do not have statewide or national data. However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmission has been shown to be fairly complete in previous studies.²⁹ Adjudicators could not be fully blinded to intervention status due to the lack of blinding of admission date. We did not calculate a kappa value for interrater reliability of individual assessments of adverse events; rather, coming to consensus among the two adjudicators was part of the process. In only a handful of cases was a third adjudicator required. Lastly, this study was conducted at two academic medical centers and their affiliated primary care clinics, which potentially limits generalizability; however, the results are likely generalizable to other ACOs that include major academic medical centers.

In conclusion, in this real-world clinical trial, we designed, implemented, and iteratively refined a multifaceted intervention to improve care transitions within a hospital-based academic ACO. Evolution of the intervention components was the result of stakeholder input, experience with the intervention, and ACO resource constraints. The intervention reduced postdischarge adverse events. However, across the ACO network, intervention fidelity was low, and this may have contributed to the lack of effect on readmission rates. ACOs that implement interventions without hiring new personnel or protecting the time of existing personnel to conduct transitional tasks are likely to face the same challenges of low fidelity.

The authors would like to acknowledge the many people who worked on designing, implementing, and evaluating this intervention, including but not limited to: Natasha Isaac, Hilary Heyison, Jacqueline Minahan, Molly O’Reilly, Michelle Potter, Nailah Khoory, Maureen Fagan, David Bates, Laura Carr, Joseph Frolkis, Eric Weil, Jacqueline Somerville, Stephanie Ahmed, Marcy Bergeron, Jessica Smith, and Jane Millett. We would also like to thank the members of our Patient-Family Advisory Council: Maureen Fagan, Karen Spikes, Margie Hodges, Win Hodges, Aureldon Henderson, Dena Salzberg, and Kay Bander.

Transitions from the hospital to the ambulatory setting are high-risk periods for patients in terms of adverse events, poor clinical outcomes, and readmission. Processes of care during care transitions are suboptimal, including poor communication among inpatient providers, patients, and ambulatory providers^1,2; suboptimal communication of postdischarge plans of care to patients and their ability to carry out these plans³; medication discrepancies and nonadherence after discharge⁴; and lack of timely follow-up with ambulatory providers.⁵ Healthcare organizations continue to struggle with the question of which interventions to implement and how best to implement them.

Interventions to improve care transitions typically focus on readmission rates, but some studies have focused on postdischarge adverse events, defined as injuries in the 30 days after discharge caused by medical management rather than underlying disease processes.² These adverse events cause psychological distress, out-of-pocket expenses, decreases in functional status, and caregiver burden. An estimated 20% of hospitalized patients suffer a postdischarge adverse event.^1,2 Approximately two-thirds of these may be preventable or ameliorable.

The advent of Accountable Care Organizations (ACOs), defined as “groups of doctors, hospitals, and other health care providers who come together voluntarily to give coordinated high quality care to their patients,” creates an opportunity for improvements in patient safety during care transitions.⁶ Another opportunity has been the advent of Patient-Centered Medical Homes (PCMH), consisting of patient-oriented, comprehensive, team-based primary care enhanced by health information technology and population-based disease management tools.^7,8 In theory, a hospital-PCMH collaboration within an ACO can improve transitional interventions since optimal communication and collaboration are more likely when both inpatient and primary care providers (PCPs) share infrastructure and are similarly incentivized. The objectives of this study were to design and implement a collaborative hospital-PCMH care transitions intervention within an ACO and evaluate its effects.

This study was a two-arm, single-blind (blinded outcomes assessor), stepped-wedge, multisite cluster-randomized clinical trial (NCT02130570) approved by the institutional review board of Partners HealthCare.

Study Design and Randomization

The study employed a “stepped-wedge” design, which is a cluster-randomized study design in which an intervention is sequentially rolled out to different groups at different, prespecified, randomly determined times.⁹ Each cluster (in this case, each primary care practice) served as its own control, while still allowing for adjustment for temporal trends. Originally, 18 practices participated, but one withdrew due to the low number of patients enrolled in the study, leaving 17 clusters and 16 sequences; see Figure 1 of Appendix 1 for a full description of the sample size and timeline for each cluster. Practices were not aware of this timeline until after recruitment.

Study Setting and Participants

Conducted within a large Pioneer ACO in Boston and funded by the Patient-Centered Outcomes Research Institute (PCORI), the Partners-PCORI Transitions Study was designed as a “real-world” quality improvement project. Potential participants were adult patients who were admitted to medical and surgical services of two large academic hospitals (Hospital A and Hospital B) affiliated with an ACO, who were likely to be discharged back to the community, and whose PCP belonged to a primary care practice that was affiliated with the ACO, agreed to participate, and were designated PCMHs or on their way to being designated by meeting certain criteria: electronic health record, patient portal, team-based care, practice redesign, care management, and identification of high-risk patients. See Study Protocol (Appendix 2) for detailed patient and primary care practice inclusion criteria.

Patient Enrollment

Study staff screened participants from a daily automated list of patients admitted the day before, using medical records to determine eligibility, which was then confirmed by the patient’s nurse. Exclusion criteria included likely discharge to a location other than home, being in police custody, lack of a telephone, being homeless, previous enrollment in the study, and being unable to communicate in English or Spanish. Allocation to study arm was concealed until the patient or proxy provided informed written consent. The research assistant administered questionnaires to all study subjects to assess potential confounders and functional status 1 month prior to admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF-12]).¹⁰ Patients were recruited between March 2013 and October 2015.

Intervention

The intervention was based on a conceptual model of an ideal discharge¹¹ that we developed based on work by Naylor et al,¹² work by Coleman and Berenson,³ best practices in medication reconciliation and information transfer according to our own research,^13-15 the best examples of interventions to improve the discharge process,^12,16,17 and a systematic review of discharge interventions.¹⁸ Some of the factors necessary for an ideal care transition include complete, organized, and timely documentation of the patient’s hospital course and postdischarge plan; effective discharge planning; coordination of care among the patient’s providers; methods to ensure medication safety; advanced care planning in appropriate patients; and education and “coaching” of patients and their caregivers so they learn how to manage their conditions. The final multifaceted intervention addressed each component of the ideal discharge and included inpatient and outpatient components (Table 1 and Table 1 of Appendix 1).

Patient and Public Involvement in Research

As with all PCORI-funded studies, this study involved a patient-family advisory council (PFAC). Our PFAC included six recently hospitalized patients or caregivers of recently hospitalized patients. The PFAC participated in monthly meetings throughout the study period. They helped inform the research questions, including confirmation that the endpoints were patient centered, and provided valuable input for the design of the intervention and the patient-facing components of the data collection instruments. They also interviewed several patient participants in the study regarding their experiences with the intervention. Lastly, they helped develop plans for dissemination of study results to the public.¹⁹

We also formed a steering committee consisting of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at both hospitals and Partners Healthcare. PFAC members took turns participating in quarterly steering committee meetings.

Evolution of the Intervention and Implementation

The intervention was iteratively refined during the course of the study in response to input from the PFAC, steering committee, and members of the intervention team; cases of adverse events and readmissions from patients despite being in the intervention arm; exit interviews of patients who had recently completed the intervention; and informal feedback from inpatient and outpatient clinicians. For example, we learned that the more complicated a patient’s conditions are, the sooner the clinical team wanted them to be seen after discharge. However, these patients were also less likely to feel well enough to keep that appointment. Therefore, the timing of follow-up of appointments needed to be a negotiation among the inpatient team, the patient, any caregivers, and the outpatient provider. PFAC members also emphasized that patients wanted one person to trust and to be the “point person” during a complicated transition such as hospital discharge.

At the same time, the intervention components evolved because of factors outside our control (eg, resource limitations). In keeping with the real-world nature of the research, the aim was for the intervention to be internally supported because incentives were theoretically more aligned with improvement of care transitions under the ACO model. By design, the PCORI contract only paid for limited parts of the intervention, such as a nurse practitioner to act as the discharge advocate at one hospital, overtime costs of inpatient pharmacists, and project manager time to facilitate inpatient-outpatient provider communication. (See Table 1 of Appendix 1 for details about the modifications to the intervention.)

Lastly, in keeping with PCORI’s methodology standards for studies of complex interventions,²⁰ we strove to standardize the intervention by function across hospitals, units, and practices, while still allowing for local adaptation in the form. In other words, rather than specifying exactly how a task (eg, medication counseling) needed to be performed, the study design offered sites flexibility in how they implemented the task given their available personnel and institutional culture.

Intervention Fidelity

To determine the extent to which each patient in the intervention arm received each intervention component, a project manager unblinded to treatment arm reviewed the electronic medical record for documentation of each component implemented by providers (eg, inpatient pharmacists, outpatient nurses). Because each intervention component produced documentation, this provided an accurate assessment of intervention fidelity, ie, the extent to which the intervention was implemented as intended.

Outcome Assessment

Postdischarge Follow-up

Based on previous studies,^2,21 a trained research assistant attempted to contact all study subjects 30 days (±5 days) after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any healthcare use since discharge, and functional status in the previous week. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms to medications or other aspects of medical management. Research assistants followed up any positive responses with directed medical record review for objective findings, diagnoses, treatments, and responses. If patients could not be reached after five attempts, the research assistant instead conducted a thorough review of the outpatient medical record alone for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results in all patients for evidence of postdischarge renal failure, elevated liver function tests, or new/worsening anemia.

Hospital Readmissions

We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for hospitalizations within the ACO network plus patient report during the 30-day phone call for all other readmissions.²²

Adjudication of Outcomes

Adverse events and preventable adverse events: All cases of new or worsening symptoms or signs, along with all supporting documentation, were then presented to teams of two trained blinded physician adjudicators through application of methods established in previous studies.^4,21 Each of the two adjudicators independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury due to medical management) and to classify the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, and whether the event was preventable or ameliorable. The two adjudicators then met to resolve any differences in their findings and come to consensus.

Preventable readmissions: If patients were readmitted to either study hospital, we conducted an evaluation, based on previous studies,²³ to determine if and how the readmission could have been prevented including (a) a standardized patient and caregiver interview to identify possible problems with the transitions process and (b) an email questionnaire to the patient’s PCP and the inpatient teams who cared for the patient during the index admission and readmission regarding possible deficiencies with the transitions process. As with adverse event adjudications, two physician adjudicators worked independently to classify the preventability of the readmission and then met to come to consensus. Conflicts were resolved by a third adjudicator.

Analysis Plan

To evaluate the effects of the intervention on the primary outcome, the number of postdischarge adverse events per patient, we used multivariable Poisson regression, with study arm as the main predictor. A similar approach was used to evaluate the number of new or worsening postdischarge signs or symptoms and the number of preventable adverse events per patient. We used an intention-to-treat analysis: If a practice did not start the intervention when they were scheduled to, based on our randomization, we counted all patients in that practice admitted after that point as intervention patients. We adjusted for patient demographics, clinical characteristics, month, inpatient unit, and primary care practice as fixed effects. We clustered by PCP using general linear models. Intervention effects were expressed as both unadjusted and adjusted incidence rate ratios (IRRs). We also conducted a limited number of subgroup analyses, determined a priori, to determine whether the intervention was more effective in certain patient populations; we used interaction terms (intervention × subgroup) to determine the statistical significance of any effect modification.

To evaluate the effects of the intervention on nonelective readmissions and preventable readmissions, we used a similar approach, using multivariable logistic regression. Postdischarge functional status, adjusted for status prior to admission, was analyzed using multivariable linear regression and random effects by primary care practice. The general linear mixed model (GLIMMIX) procedure in the SAS 9.3 statistical package (SAS Institute) was used to carry out all analyses.

Power and Sample Size

We assumed a baseline rate of postdischarge adverse events of 0.30 per patient.²¹ We conservatively assumed an effect size of a change from 0.30 in the control group to 0.23 in the intervention group (a relative reduction of 22%, which was based on studies of preventability rates²³ and close to the minimum clinically important difference). Based on prior studies,^4,22 we assumed an intraclass correlation coefficient of 0.01 with an average cluster size of seven patients per PCP. Assuming a 10% loss to follow-up rate and an alpha of 0.05, we targeted a sample size of 1,800 patients to achieve 80% power, with one-third of the patients in the usual care arm and two-thirds in the intervention arm.

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from Hospital A (out of 13 approached), 8 from Hospital B (out of 11), and 2 from other ACO practices (out of 9) (plus two pilot practices). Reasons for not participating included not having dedicated personnel to play the role of the responsible outpatient clinician, undergoing recent turn-over in practice leadership, and not having enough patients admitted to the two hospitals. One practice only enrolled 5 patients in the study and withdrew from participation, which left 17 practices.

Study Patients

We enrolled 1,679 patients (Figure 1). Reasons for nonenrollment included being unable to complete the screen prior to discharge, not meeting inclusion criteria or meeting exclusion criteria, being assigned to a pilot practice, and declining informed written consent. The baseline characteristics of enrolled patients are presented in Table 2. Differences between the two study arms were small. About 47% of the cohort was not reachable by phone after five attempts for the 30-day phone call, but only 69 (4.1%) were truly lost to follow-up because they were unreachable by phone and had no documentation in the electronic medical record in the 30-days after discharge.

Intervention Fidelity

The majority of patients did not receive most intervention components, even those components that were supposed to be delivered to all intervention patients (Table 3). A minority of patients were referred to visiting nurse services and to the home pharmacy program. However, 855 patients (87%) in the intervention arm received at least one intervention component.

Outcome Measures

The intervention was associated with a statistically significant reduction in several of the outcomes of interest, including the primary outcome, number of postdischarge adverse events (45% reduction), and new or worsening postdischarge signs or symptoms (22% reduction), as well as preventable postdischarge adverse events (58% reduction) (Table 4). There was a nonsignificant difference in functional status. There was no significant effect on total nonelective or on preventable readmission rates. When analyzed by type of adverse event, the intervention was associated with a reduction in adverse drug events and in procedural complications (Table 2 of Appendix 1). Of note, there was no significant difference in the proportion of patients with at least one adverse event whether the outcome was determined by phone call and medical record review (49%) or medical record review alone (51%) (P = .48).

In subgroup analyses, there was no evidence of effect modification by service, hospital, patient age, readmission risk, health literacy, or comorbidity score (Table 3 of Appendix 1). Table 4 of Appendix 1 provides examples of postdischarge adverse events seen in the usual care arm that might have been prevented in the intervention.

This intervention was associated with a reduction in postdischarge adverse events. The relative improvement in each outcome aligned with the hypothesized sensitivity to change: the smallest improvement was seen in new or worsening signs or symptoms, followed by postdischarge adverse events and then by preventable postdischarge adverse events. The intervention was not associated with a difference in readmissions. The lack of effect on hospital readmissions may have been caused by the low proportion of readmissions that are preventable, as well as low intervention fidelity and lack of resources to implement facets such as postdischarge coaching, an evidence-based intervention that was never adopted.^16,23 One lesson of this study is that it may be easier to reduce postdischarge injury (still an important outcome) than readmissions.

Putting this study in context, we should note that the literature on interventions to improve care transitions is mixed.¹⁸ While there are several reports of successful interventions, there are many reports of unsuccessful ones, often using similar components. Success is often the result of adequate resources and attention to myriad details regarding implementation.²⁴ The intervention in our study likely contributed to improvements in patient and caregiver engagement in the hospital, enhancements of communication between inpatient and outpatient clinicians, and implementation of pharmacist-led interventions to improve medication safety. Regarding the latter, several prior studies have shown the benefits of pharmacist interventions in decreasing postdischarge adverse drug events.^4,25,26 Therefore, even an intervention with incomplete intervention fidelity can reduce postdischarge adverse events, especially because adverse drug events make up the majority of adverse events.^1,2,21

Perhaps the biggest lesson we learned was regarding the limitations of the hospital-led ACO model to incentivize sufficient up-front investments in transitional care interventions. By design, we chose a real-world approach in which interventions were integrated with existing ACO efforts, which were paid for internally by the institution. As a result, many of the interventions had to be scaled back because of resource constraints. The ACO model theoretically incentivizes more integrated care, but this may not always be true in practice. Emerging evidence suggests that physician group–led ACOs are associated with lower costs and use compared with hospital-led ACOs, likely because of more aligned incentives in physician group–led ACOs to reduce use of inpatient care.^27,28

An unresolved question is whether the ideal implementation approach is to protect the time of existing clinical personnel to carry out transitional care tasks or to hire external personnel to do these tasks. We purposely spread the intervention over several clinician types to minimize the additional burden on any one of them, minimize additional costs, and play to each clinician’s expertise, but in retrospect, this may not have been the right approach. By providing additional personnel with dedicated time, interest, and training in care transitions, the intervention may be delivered with higher quantitative and qualitative fidelity, and it could create a single point of contact for patients, which was considered highly desirable by our PFAC.

This study has several limitations. A large proportion of patients (44%) were unavailable for postdischarge phone calls. However, we were able to perform medical record review for worsening signs (eg, lab abnormalities) and symptoms (as reported by patients’ providers) in the postdischarge period and adjudicate them for adverse events for all but 69 of these patients. Because all these patients had ACO-affiliated PCPs, we would expect most of their utilization to have been within the system and, therefore, to be present in the medical record. The proportion of patients with at least one adverse event did not vary by the method of follow-up, which suggests that this issue is an unlikely source of bias. Assessment of readmission was imperfect because we do not have statewide or national data. However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmission has been shown to be fairly complete in previous studies.²⁹ Adjudicators could not be fully blinded to intervention status due to the lack of blinding of admission date. We did not calculate a kappa value for interrater reliability of individual assessments of adverse events; rather, coming to consensus among the two adjudicators was part of the process. In only a handful of cases was a third adjudicator required. Lastly, this study was conducted at two academic medical centers and their affiliated primary care clinics, which potentially limits generalizability; however, the results are likely generalizable to other ACOs that include major academic medical centers.

In conclusion, in this real-world clinical trial, we designed, implemented, and iteratively refined a multifaceted intervention to improve care transitions within a hospital-based academic ACO. Evolution of the intervention components was the result of stakeholder input, experience with the intervention, and ACO resource constraints. The intervention reduced postdischarge adverse events. However, across the ACO network, intervention fidelity was low, and this may have contributed to the lack of effect on readmission rates. ACOs that implement interventions without hiring new personnel or protecting the time of existing personnel to conduct transitional tasks are likely to face the same challenges of low fidelity.

The authors would like to acknowledge the many people who worked on designing, implementing, and evaluating this intervention, including but not limited to: Natasha Isaac, Hilary Heyison, Jacqueline Minahan, Molly O’Reilly, Michelle Potter, Nailah Khoory, Maureen Fagan, David Bates, Laura Carr, Joseph Frolkis, Eric Weil, Jacqueline Somerville, Stephanie Ahmed, Marcy Bergeron, Jessica Smith, and Jane Millett. We would also like to thank the members of our Patient-Family Advisory Council: Maureen Fagan, Karen Spikes, Margie Hodges, Win Hodges, Aureldon Henderson, Dena Salzberg, and Kay Bander.

Unintentional medication discrepancies in the hospital setting are common and contribute to adverse drug events, resulting in patient harm.¹ Discrepancies can be resolved by implementing high-quality medication reconciliation, but there are insufficient data to guide hospitals as to which interventions are most effective at improving medication reconciliation processes and reducing harm.² We recently reported that implementation of a best practices toolkit reduced total medication discrepancies in the Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS).³ This report describes the effect of individual toolkit components on rates of medication discrepancies with the potential for patient harm.

Detailed descriptions of the intervention toolkit and study design of MARQUIS are published.^4,5 Briefly, MARQUIS was a pragmatic, mentored, quality improvement (QI) study in which five hospitals in the United States implemented interventions from a best practices toolkit to improve medication reconciliation on noncritical care medical and surgical units from September 2011 to July 2014. We used a mentored implementation approach, in which each site identified the leaders of their local quality improvement team (ie, mentees) who received mentorship from a trained physician with QI and medication safety experience.⁶ Mentors conducted monthly calls with their mentees and two site visits. Sites adapted and implemented one or more components from the MARQUIS toolkit, a compilation of evidence-based best practices in medication reconciliation.^5,7

The primary outcome was unintentional medication discrepancies in admission and discharge orders with the potential for causing harm, as previously described.⁴ Trained study pharmacists at each site took “gold standard” medication histories on a random sample of up to 22 patients per month. These medications were then compared with admission and discharge medication orders, and all unintentional discrepancies were identified. The discrepancies were then adjudicated by physicians blinded to the treatment arm, who confirmed whether discrepancies were unintentional and carried the potential for patient harm.

We employed a modification of a stepped wedge methodology to measure the incremental effect of implementing nine different intervention components, introduced at different sites over the course of the study, on the number of potentially harmful discrepancies per patient. These analyses were restricted to the postimplementation period on hospital units that implemented at least one intervention. All interventions conducted at each site were categorized by component, including dates of implementation. Each intervention component could be applied more than once per site (eg, when involving a new group of providers) or implemented on a new hospital unit or service, in which case, all dates were included in the analysis. We conducted a multivariable Poisson regression (with time divided into months) adjusted for patient factors, season, and site, with the number of potentially harmful discrepancies as the dependent variable, and the total number of gold standard medications as a model offset. The model was designed to analyze changes in the y-intercept each time an intervention component was either implemented or spread and assumed the change in the y-intercept was the same for each of these events for any given component. The model also assumes that combinations of interventions had independent additive effects.

Across the five participating sites, 1,648 patients were enrolled from September 2011 to July 2014. This number included 613 patients during the preimplementation period and 1,035 patients during the postimplementation period, of which 791 were on intervention units and comprised the study population. Table 1 displays the intervention components implemented by site. Sites implemented between one and seven components. The most frequently implemented intervention component was training existing staff to take the best possible medication histories (BPMHs), implemented at four sites. The regression results are displayed in Table 2. Three interventions were associated with significant decreases in potentially harmful discrepancy rates: (1) clearly defining roles and responsibilities and communicating this with clinical staff (hazard ratio [HR] 0.53, 95% CI: 0.32–0.87); (2) training existing staff to perform discharge medication reconciliation and patient counseling (HR 0.64, 95% CI: 0.46–0.89); and (3) hiring additional staff to perform discharge medication reconciliation and patient counseling (HR 0.48, 95% CI: 0.31–0.77). Two interventions were associated with significant increases in potentially harmful discrepancy rates: training existing staff to take BPMHs (HR 1.38, 95% CI: 1.21–1.57) and implementing a new electronic health record (EHR; HR 2.21, 95% CI: 1.64–2.97).

We noted that three intervention components were associated with decreased rates of unintentional medication discrepancies with potential for harm, whereas two were associated with increased rates. The components with a beneficial effect were not surprising. A prior qualitative study demonstrated the confusion related to clinicians’ roles and responsibilities during medication reconciliation; therefore, clear delineations should reduce rework and improve the medication reconciliation process.⁸ Other studies have shown the benefits of pharmacist involvement in the inpatient setting, particularly in reducing errors at discharge.⁹ However, we did not anticipate that training staff to take BPMHs would be detrimental. Possible reasons for this finding that are based on direct observations by mentors at site visits or noted during monthly calls include (1) training personnel on this task without certification of competency may not sufficiently improve their skills, leading instead to diffusion of responsibility; (2) training personnel without sufficient time to perform the task well (eg, frontline nurses with many other responsibilities) may be counterproductive compared with training a few personnel with time dedicated to this task; and (3) training existing personnel in history-taking may have been used to delay the necessary hiring of more staff to take BPMHs. Future studies could address several of these shortcomings in both the design and implementation of medication history-training intervention components.

Several reasons may explain the association we found between implementing a new EHR and increased rates of discrepancies. Based on mentors’ experiences, we suspect it is because sitewide EHR implementation requires significant resources, time, and effort. Therefore, sitewide EHR implementation pulls attention away from a focus on medication safety. Most large vendor EHRs have design flaws in their medication reconciliation modules, with the overarching problem being that their systems are not designed for an interdisciplinary team approach to medication reconciliation (unpublished material). In addition, problems may also exist with the local implementation of these modules and the way they are used by clinicians (eg, bypassing critical steps in the medication reconciliation process that lead to new medication errors). We have updated the MARQUIS toolkit to include pros and cons of EHR software and ideal features and functions of medication reconciliation information technology. We should note that this finding contrasts with previous studies that showed beneficial effects of dedicated medication reconciliation applications, which used proprietary technology, often combined with process redesign, in a focused QI effort.^10-13 These findings suggest the need for improvements in the design, local customization, and use of medication reconciliation modules in vendor EHRs.

Our study has several limitations. We conducted an on-treatment analysis, which may be confounded by characteristics of sites that chose to implement different intervention components; however, we adjusted for sites in the analysis. Some results are based on a limited number of sites implementing an intervention component (eg, defining roles and responsibilities). Although this was a longitudinal study, and we adjusted for seasonal effects, it is possible that temporal trends and cointerventions confounded our results. The adjudication of discrepancies for the potential for harm was somewhat subjective, although we used a rigorous process to ensure the reliability of adjudication, as in prior studies.^3,14 As in the main analysis of the MARQUIS study, this analysis did not measure intervention fidelity.

Based on these analyses and the literature base, we recommend that hospitals focus first on hiring and training dedicated staff (usually pharmacists) to assist with medication reconciliation at discharge.⁷ Hospitals should also be aware of potential increases in medication discrepancies when implementing a large vendor EHR across their institution. Further work is needed on the best ways to mitigate these adverse effects, at both the design and local site levels. Finally, the effect of medication history training on discrepancies warrants further study.

Disclosures

SK has served as a consultant to Verustat, a remote health monitoring company. All other authors have no disclosures or conflicts of interests.

Funding

This study was supported by the Agency for Healthcare Research and Quality (grant number: R18 HS019598). JLS has received funding from (1) Mallinckrodt Pharmaceuticals for an investigator-initiated study of opioid-related adverse drug events in postsurgical patients; (2) Horizon Blue Cross Blue Shield for an honorarium and travel expenses for workshop on medication reconciliation; (3) Island Peer Review Organization for honorarium and travel expenses for workshop on medication reconciliation; and, (4) Portola Pharmaceuticals for investigator-initiated study of inpatients who decline subcutaneous medications for venous thromboembolism prophylaxis. ASM was funded by a VA HSR&D Career Development Award (12-168).

Trial Registration

ClinicalTrials.gov NCT01337063

Unintentional medication discrepancies in the hospital setting are common and contribute to adverse drug events, resulting in patient harm.¹ Discrepancies can be resolved by implementing high-quality medication reconciliation, but there are insufficient data to guide hospitals as to which interventions are most effective at improving medication reconciliation processes and reducing harm.² We recently reported that implementation of a best practices toolkit reduced total medication discrepancies in the Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS).³ This report describes the effect of individual toolkit components on rates of medication discrepancies with the potential for patient harm.

Detailed descriptions of the intervention toolkit and study design of MARQUIS are published.^4,5 Briefly, MARQUIS was a pragmatic, mentored, quality improvement (QI) study in which five hospitals in the United States implemented interventions from a best practices toolkit to improve medication reconciliation on noncritical care medical and surgical units from September 2011 to July 2014. We used a mentored implementation approach, in which each site identified the leaders of their local quality improvement team (ie, mentees) who received mentorship from a trained physician with QI and medication safety experience.⁶ Mentors conducted monthly calls with their mentees and two site visits. Sites adapted and implemented one or more components from the MARQUIS toolkit, a compilation of evidence-based best practices in medication reconciliation.^5,7

The primary outcome was unintentional medication discrepancies in admission and discharge orders with the potential for causing harm, as previously described.⁴ Trained study pharmacists at each site took “gold standard” medication histories on a random sample of up to 22 patients per month. These medications were then compared with admission and discharge medication orders, and all unintentional discrepancies were identified. The discrepancies were then adjudicated by physicians blinded to the treatment arm, who confirmed whether discrepancies were unintentional and carried the potential for patient harm.

We employed a modification of a stepped wedge methodology to measure the incremental effect of implementing nine different intervention components, introduced at different sites over the course of the study, on the number of potentially harmful discrepancies per patient. These analyses were restricted to the postimplementation period on hospital units that implemented at least one intervention. All interventions conducted at each site were categorized by component, including dates of implementation. Each intervention component could be applied more than once per site (eg, when involving a new group of providers) or implemented on a new hospital unit or service, in which case, all dates were included in the analysis. We conducted a multivariable Poisson regression (with time divided into months) adjusted for patient factors, season, and site, with the number of potentially harmful discrepancies as the dependent variable, and the total number of gold standard medications as a model offset. The model was designed to analyze changes in the y-intercept each time an intervention component was either implemented or spread and assumed the change in the y-intercept was the same for each of these events for any given component. The model also assumes that combinations of interventions had independent additive effects.

Across the five participating sites, 1,648 patients were enrolled from September 2011 to July 2014. This number included 613 patients during the preimplementation period and 1,035 patients during the postimplementation period, of which 791 were on intervention units and comprised the study population. Table 1 displays the intervention components implemented by site. Sites implemented between one and seven components. The most frequently implemented intervention component was training existing staff to take the best possible medication histories (BPMHs), implemented at four sites. The regression results are displayed in Table 2. Three interventions were associated with significant decreases in potentially harmful discrepancy rates: (1) clearly defining roles and responsibilities and communicating this with clinical staff (hazard ratio [HR] 0.53, 95% CI: 0.32–0.87); (2) training existing staff to perform discharge medication reconciliation and patient counseling (HR 0.64, 95% CI: 0.46–0.89); and (3) hiring additional staff to perform discharge medication reconciliation and patient counseling (HR 0.48, 95% CI: 0.31–0.77). Two interventions were associated with significant increases in potentially harmful discrepancy rates: training existing staff to take BPMHs (HR 1.38, 95% CI: 1.21–1.57) and implementing a new electronic health record (EHR; HR 2.21, 95% CI: 1.64–2.97).

We noted that three intervention components were associated with decreased rates of unintentional medication discrepancies with potential for harm, whereas two were associated with increased rates. The components with a beneficial effect were not surprising. A prior qualitative study demonstrated the confusion related to clinicians’ roles and responsibilities during medication reconciliation; therefore, clear delineations should reduce rework and improve the medication reconciliation process.⁸ Other studies have shown the benefits of pharmacist involvement in the inpatient setting, particularly in reducing errors at discharge.⁹ However, we did not anticipate that training staff to take BPMHs would be detrimental. Possible reasons for this finding that are based on direct observations by mentors at site visits or noted during monthly calls include (1) training personnel on this task without certification of competency may not sufficiently improve their skills, leading instead to diffusion of responsibility; (2) training personnel without sufficient time to perform the task well (eg, frontline nurses with many other responsibilities) may be counterproductive compared with training a few personnel with time dedicated to this task; and (3) training existing personnel in history-taking may have been used to delay the necessary hiring of more staff to take BPMHs. Future studies could address several of these shortcomings in both the design and implementation of medication history-training intervention components.

Several reasons may explain the association we found between implementing a new EHR and increased rates of discrepancies. Based on mentors’ experiences, we suspect it is because sitewide EHR implementation requires significant resources, time, and effort. Therefore, sitewide EHR implementation pulls attention away from a focus on medication safety. Most large vendor EHRs have design flaws in their medication reconciliation modules, with the overarching problem being that their systems are not designed for an interdisciplinary team approach to medication reconciliation (unpublished material). In addition, problems may also exist with the local implementation of these modules and the way they are used by clinicians (eg, bypassing critical steps in the medication reconciliation process that lead to new medication errors). We have updated the MARQUIS toolkit to include pros and cons of EHR software and ideal features and functions of medication reconciliation information technology. We should note that this finding contrasts with previous studies that showed beneficial effects of dedicated medication reconciliation applications, which used proprietary technology, often combined with process redesign, in a focused QI effort.^10-13 These findings suggest the need for improvements in the design, local customization, and use of medication reconciliation modules in vendor EHRs.

Our study has several limitations. We conducted an on-treatment analysis, which may be confounded by characteristics of sites that chose to implement different intervention components; however, we adjusted for sites in the analysis. Some results are based on a limited number of sites implementing an intervention component (eg, defining roles and responsibilities). Although this was a longitudinal study, and we adjusted for seasonal effects, it is possible that temporal trends and cointerventions confounded our results. The adjudication of discrepancies for the potential for harm was somewhat subjective, although we used a rigorous process to ensure the reliability of adjudication, as in prior studies.^3,14 As in the main analysis of the MARQUIS study, this analysis did not measure intervention fidelity.

Based on these analyses and the literature base, we recommend that hospitals focus first on hiring and training dedicated staff (usually pharmacists) to assist with medication reconciliation at discharge.⁷ Hospitals should also be aware of potential increases in medication discrepancies when implementing a large vendor EHR across their institution. Further work is needed on the best ways to mitigate these adverse effects, at both the design and local site levels. Finally, the effect of medication history training on discrepancies warrants further study.

Disclosures

SK has served as a consultant to Verustat, a remote health monitoring company. All other authors have no disclosures or conflicts of interests.

Funding

This study was supported by the Agency for Healthcare Research and Quality (grant number: R18 HS019598). JLS has received funding from (1) Mallinckrodt Pharmaceuticals for an investigator-initiated study of opioid-related adverse drug events in postsurgical patients; (2) Horizon Blue Cross Blue Shield for an honorarium and travel expenses for workshop on medication reconciliation; (3) Island Peer Review Organization for honorarium and travel expenses for workshop on medication reconciliation; and, (4) Portola Pharmaceuticals for investigator-initiated study of inpatients who decline subcutaneous medications for venous thromboembolism prophylaxis. ASM was funded by a VA HSR&D Career Development Award (12-168).

Trial Registration

ClinicalTrials.gov NCT01337063

Unintentional medication discrepancies in the hospital setting are common and contribute to adverse drug events, resulting in patient harm.¹ Discrepancies can be resolved by implementing high-quality medication reconciliation, but there are insufficient data to guide hospitals as to which interventions are most effective at improving medication reconciliation processes and reducing harm.² We recently reported that implementation of a best practices toolkit reduced total medication discrepancies in the Multi-Center Medication Reconciliation Quality Improvement Study (MARQUIS).³ This report describes the effect of individual toolkit components on rates of medication discrepancies with the potential for patient harm.

Detailed descriptions of the intervention toolkit and study design of MARQUIS are published.^4,5 Briefly, MARQUIS was a pragmatic, mentored, quality improvement (QI) study in which five hospitals in the United States implemented interventions from a best practices toolkit to improve medication reconciliation on noncritical care medical and surgical units from September 2011 to July 2014. We used a mentored implementation approach, in which each site identified the leaders of their local quality improvement team (ie, mentees) who received mentorship from a trained physician with QI and medication safety experience.⁶ Mentors conducted monthly calls with their mentees and two site visits. Sites adapted and implemented one or more components from the MARQUIS toolkit, a compilation of evidence-based best practices in medication reconciliation.^5,7

The primary outcome was unintentional medication discrepancies in admission and discharge orders with the potential for causing harm, as previously described.⁴ Trained study pharmacists at each site took “gold standard” medication histories on a random sample of up to 22 patients per month. These medications were then compared with admission and discharge medication orders, and all unintentional discrepancies were identified. The discrepancies were then adjudicated by physicians blinded to the treatment arm, who confirmed whether discrepancies were unintentional and carried the potential for patient harm.

We employed a modification of a stepped wedge methodology to measure the incremental effect of implementing nine different intervention components, introduced at different sites over the course of the study, on the number of potentially harmful discrepancies per patient. These analyses were restricted to the postimplementation period on hospital units that implemented at least one intervention. All interventions conducted at each site were categorized by component, including dates of implementation. Each intervention component could be applied more than once per site (eg, when involving a new group of providers) or implemented on a new hospital unit or service, in which case, all dates were included in the analysis. We conducted a multivariable Poisson regression (with time divided into months) adjusted for patient factors, season, and site, with the number of potentially harmful discrepancies as the dependent variable, and the total number of gold standard medications as a model offset. The model was designed to analyze changes in the y-intercept each time an intervention component was either implemented or spread and assumed the change in the y-intercept was the same for each of these events for any given component. The model also assumes that combinations of interventions had independent additive effects.

Across the five participating sites, 1,648 patients were enrolled from September 2011 to July 2014. This number included 613 patients during the preimplementation period and 1,035 patients during the postimplementation period, of which 791 were on intervention units and comprised the study population. Table 1 displays the intervention components implemented by site. Sites implemented between one and seven components. The most frequently implemented intervention component was training existing staff to take the best possible medication histories (BPMHs), implemented at four sites. The regression results are displayed in Table 2. Three interventions were associated with significant decreases in potentially harmful discrepancy rates: (1) clearly defining roles and responsibilities and communicating this with clinical staff (hazard ratio [HR] 0.53, 95% CI: 0.32–0.87); (2) training existing staff to perform discharge medication reconciliation and patient counseling (HR 0.64, 95% CI: 0.46–0.89); and (3) hiring additional staff to perform discharge medication reconciliation and patient counseling (HR 0.48, 95% CI: 0.31–0.77). Two interventions were associated with significant increases in potentially harmful discrepancy rates: training existing staff to take BPMHs (HR 1.38, 95% CI: 1.21–1.57) and implementing a new electronic health record (EHR; HR 2.21, 95% CI: 1.64–2.97).

We noted that three intervention components were associated with decreased rates of unintentional medication discrepancies with potential for harm, whereas two were associated with increased rates. The components with a beneficial effect were not surprising. A prior qualitative study demonstrated the confusion related to clinicians’ roles and responsibilities during medication reconciliation; therefore, clear delineations should reduce rework and improve the medication reconciliation process.⁸ Other studies have shown the benefits of pharmacist involvement in the inpatient setting, particularly in reducing errors at discharge.⁹ However, we did not anticipate that training staff to take BPMHs would be detrimental. Possible reasons for this finding that are based on direct observations by mentors at site visits or noted during monthly calls include (1) training personnel on this task without certification of competency may not sufficiently improve their skills, leading instead to diffusion of responsibility; (2) training personnel without sufficient time to perform the task well (eg, frontline nurses with many other responsibilities) may be counterproductive compared with training a few personnel with time dedicated to this task; and (3) training existing personnel in history-taking may have been used to delay the necessary hiring of more staff to take BPMHs. Future studies could address several of these shortcomings in both the design and implementation of medication history-training intervention components.

Several reasons may explain the association we found between implementing a new EHR and increased rates of discrepancies. Based on mentors’ experiences, we suspect it is because sitewide EHR implementation requires significant resources, time, and effort. Therefore, sitewide EHR implementation pulls attention away from a focus on medication safety. Most large vendor EHRs have design flaws in their medication reconciliation modules, with the overarching problem being that their systems are not designed for an interdisciplinary team approach to medication reconciliation (unpublished material). In addition, problems may also exist with the local implementation of these modules and the way they are used by clinicians (eg, bypassing critical steps in the medication reconciliation process that lead to new medication errors). We have updated the MARQUIS toolkit to include pros and cons of EHR software and ideal features and functions of medication reconciliation information technology. We should note that this finding contrasts with previous studies that showed beneficial effects of dedicated medication reconciliation applications, which used proprietary technology, often combined with process redesign, in a focused QI effort.^10-13 These findings suggest the need for improvements in the design, local customization, and use of medication reconciliation modules in vendor EHRs.

Our study has several limitations. We conducted an on-treatment analysis, which may be confounded by characteristics of sites that chose to implement different intervention components; however, we adjusted for sites in the analysis. Some results are based on a limited number of sites implementing an intervention component (eg, defining roles and responsibilities). Although this was a longitudinal study, and we adjusted for seasonal effects, it is possible that temporal trends and cointerventions confounded our results. The adjudication of discrepancies for the potential for harm was somewhat subjective, although we used a rigorous process to ensure the reliability of adjudication, as in prior studies.^3,14 As in the main analysis of the MARQUIS study, this analysis did not measure intervention fidelity.

Based on these analyses and the literature base, we recommend that hospitals focus first on hiring and training dedicated staff (usually pharmacists) to assist with medication reconciliation at discharge.⁷ Hospitals should also be aware of potential increases in medication discrepancies when implementing a large vendor EHR across their institution. Further work is needed on the best ways to mitigate these adverse effects, at both the design and local site levels. Finally, the effect of medication history training on discrepancies warrants further study.

Disclosures

SK has served as a consultant to Verustat, a remote health monitoring company. All other authors have no disclosures or conflicts of interests.

Funding

This study was supported by the Agency for Healthcare Research and Quality (grant number: R18 HS019598). JLS has received funding from (1) Mallinckrodt Pharmaceuticals for an investigator-initiated study of opioid-related adverse drug events in postsurgical patients; (2) Horizon Blue Cross Blue Shield for an honorarium and travel expenses for workshop on medication reconciliation; (3) Island Peer Review Organization for honorarium and travel expenses for workshop on medication reconciliation; and, (4) Portola Pharmaceuticals for investigator-initiated study of inpatients who decline subcutaneous medications for venous thromboembolism prophylaxis. ASM was funded by a VA HSR&D Career Development Award (12-168).

Trial Registration

ClinicalTrials.gov NCT01337063