Jennifer S. Myers, MD

Over the past two decades, the field of hospital medicine has gone from relative obscurity to a viable career pathway for approximately 50,000 physicians in this country.¹ A subset of hospitalists pursue careers in academic medicine, which is a pathway that traditionally includes education and scholarship in addition to patient care. While the academic career pathway is well paved in many clinical specialties, it is still relatively underdeveloped for academic hospitalists, and thus what defines career success for this group is even less clear.

In this issue of the Journal of Hospital Medicine, Cumbler et al. performed a qualitative analysis to explore how early career academic hospitalists self-define and perceive their career success.² Drawing on interviews with 17 early-career hospitalists at 3 academic medical centers, the authors created a theoretical framework organized around a traditional conceptual model of career success that is divided into intrinsic and extrinsic motivating factors. They found that early-career academic hospitalists, (clinician-educators in first 2-5 years), defined their career success almost exclusively around factors intrinsic to their day-to-day job. These factors included such things as excitement about their daily work, developing proficiency in the delivery of high-quality clinical care, and passion for doing work that is meaningful to them. In addition to these immediate job satisfiers, many hospitalists emphasized long-term career success factors such as becoming an expert in a particular domain of hospital medicine and gaining respect and recognition within their local or national environment. Surprisingly, compensation and career advancement through promotion, two traditional external career success factors, were not uniformly valued.

These findings come at a critical time for our field in which early-career faculty outnumber mid- and late-career faculty by an order of magnitude. Indeed, how to develop, promote, sustain, and retain young hospitalists is a topic on the minds of most hospital medicine group directors. Putting aside the impact of hospitalist turnover on productivity, patient care outcomes, and morale within an individual hospital medicine group, we agree with the authors that understanding and cultivating career success for academic hospitalists is imperative for the future of our field. For this reason, we launched a formal faculty development program at Penn this year, which focuses on supporting the growth of hospitalists in their first two years on faculty. The findings of this study provide interesting new perspectives and encourage us to continue our focus on early-career academic hospitalists. We laud the previous efforts in this area and hope that the paper by Cumbler et al. encourages and inspires other programs to start or accelerate their hospitalist faculty development efforts.^3-5

However, some findings from this study are somewhat perplexing or even a bit discouraging for those who are invested in faculty development in academia. For example, the authors raise the possibility that there may be a disconnect in the minds of early-career hospitalists as it pertains to their thoughts on career success. On the one hand, the hospitalists interviewed in this study are happy doing their clinical work and cite this as a primary driver of their career success. On the other hand, they equate career success with things such as developing expertise within a particular domain of hospital medicine, acquiring leadership roles, collaborating academically with other specialties or professions, or developing new innovations. Presumably this is part of the reason that they selected a job in an academic setting as opposed to a community setting. However, in order to achieve these goals, one must devote time and effort to purposefully developing them. Therefore, identifying and developing mentors who can assist early-career hospitalists with identifying, articulating, and developing strategies to achieve both their short- and long-term career goals is critical. One mentor–mentee conversation may reveal that an individual hospitalist values being an excellent clinician and has little interest in developing a niche within hospital medicine; another may reveal a lack of awareness of available professional development resources; still another may uncover a lack of realism regarding the time or skills it takes to achieve a particular career goal. These realities highlight an imperative for our field to develop robust and sustainable mentorship programs for not only early-career hospitalists but also some mid-career hospitalists whose careers may not yet be fully developed. Indeed, one of the biggest challenges that have emerged in our experience with a faculty development program at Penn is creating meaningful mentorship and career development advice for mid-career hospitalists (late assistant or early associate professors who are typically 5-10 years into their careers).

We found it interesting that the hospitalists interviewed did not mention three of the four pillars of career satisfaction outlined in the white paper on Hospitalist Career Satisfaction from the Society for Hospital Medicine: workload schedule, autonomy control, and community/environment.⁶ Perhaps this is because hospitalists, like many other professionals, recognize that feeling satisfied in one’s career is not the same as feeling successful. Satisfaction in one’s career refers to the foundational needs that one requires in order to feel content, whereas success is more often equated with achievement, even if that achievement is simply the acquisition of one’s goals for themselves. The reality is that given the constant growth and change within teaching hospitals, and therefore academic hospital medicine groups, tending to the satisfiers for hospitalists (eg, schedule and workload) often takes a front seat to assisting faculty in achieving their individual career potential. We assert that despite the inherent difficulty, academic hospital medicine group leaders need to focus their attention on both the satisfaction and career success of their early-career faculty.

Finally, this paper raises many interesting questions for researchers interested in the professional development of hospitalists. Are the career success perspectives of an early-career academic hospitalist different from those of an early-career intensivist or emergency medicine physician in an academic setting? Hospital medicine has historically been likened to both fields given the similar intensity of clinical work and the fact that all three fields were created around the need for specialists in a care setting as opposed to a disease state. It is possible that the vision of success for young academic physicians as a whole has changed with the millennial generation entering the workforce. Do early-career hospitalists look different from early-career general internists in academic settings? The latter group has more promoted faculty in their division to serve as role models and mentors and who have demonstrated more success in a variety of replicable career pathways. The fact that the definition of career success may evolve over time also emerged as a theme from this paper. Do mid-career academic hospitalists find that the excitement for daily clinical work wanes over time leaving them feeling less successful and looking for something more?

In conclusion, the findings of Cumbler et al. should promote unrest among leaders of academic hospital medicine groups and their departments of medicine. While it is inspiring to see so many early-career hospitalists focused on their daily happiness at work, we are unsure about whether they have the knowledge, tools, and guidance to achieve their self-professed academic goals, which many equate with career success. Given the continued growth of the hospital medicine workforce, we view this important new work as a national call to arms for the purposeful development of academic hospitalist faculty development programs.

Disclosures

Dr. Myers and Dr. Greysen have nothing to disclose.

Over the past two decades, the field of hospital medicine has gone from relative obscurity to a viable career pathway for approximately 50,000 physicians in this country.¹ A subset of hospitalists pursue careers in academic medicine, which is a pathway that traditionally includes education and scholarship in addition to patient care. While the academic career pathway is well paved in many clinical specialties, it is still relatively underdeveloped for academic hospitalists, and thus what defines career success for this group is even less clear.

In this issue of the Journal of Hospital Medicine, Cumbler et al. performed a qualitative analysis to explore how early career academic hospitalists self-define and perceive their career success.² Drawing on interviews with 17 early-career hospitalists at 3 academic medical centers, the authors created a theoretical framework organized around a traditional conceptual model of career success that is divided into intrinsic and extrinsic motivating factors. They found that early-career academic hospitalists, (clinician-educators in first 2-5 years), defined their career success almost exclusively around factors intrinsic to their day-to-day job. These factors included such things as excitement about their daily work, developing proficiency in the delivery of high-quality clinical care, and passion for doing work that is meaningful to them. In addition to these immediate job satisfiers, many hospitalists emphasized long-term career success factors such as becoming an expert in a particular domain of hospital medicine and gaining respect and recognition within their local or national environment. Surprisingly, compensation and career advancement through promotion, two traditional external career success factors, were not uniformly valued.

These findings come at a critical time for our field in which early-career faculty outnumber mid- and late-career faculty by an order of magnitude. Indeed, how to develop, promote, sustain, and retain young hospitalists is a topic on the minds of most hospital medicine group directors. Putting aside the impact of hospitalist turnover on productivity, patient care outcomes, and morale within an individual hospital medicine group, we agree with the authors that understanding and cultivating career success for academic hospitalists is imperative for the future of our field. For this reason, we launched a formal faculty development program at Penn this year, which focuses on supporting the growth of hospitalists in their first two years on faculty. The findings of this study provide interesting new perspectives and encourage us to continue our focus on early-career academic hospitalists. We laud the previous efforts in this area and hope that the paper by Cumbler et al. encourages and inspires other programs to start or accelerate their hospitalist faculty development efforts.^3-5

However, some findings from this study are somewhat perplexing or even a bit discouraging for those who are invested in faculty development in academia. For example, the authors raise the possibility that there may be a disconnect in the minds of early-career hospitalists as it pertains to their thoughts on career success. On the one hand, the hospitalists interviewed in this study are happy doing their clinical work and cite this as a primary driver of their career success. On the other hand, they equate career success with things such as developing expertise within a particular domain of hospital medicine, acquiring leadership roles, collaborating academically with other specialties or professions, or developing new innovations. Presumably this is part of the reason that they selected a job in an academic setting as opposed to a community setting. However, in order to achieve these goals, one must devote time and effort to purposefully developing them. Therefore, identifying and developing mentors who can assist early-career hospitalists with identifying, articulating, and developing strategies to achieve both their short- and long-term career goals is critical. One mentor–mentee conversation may reveal that an individual hospitalist values being an excellent clinician and has little interest in developing a niche within hospital medicine; another may reveal a lack of awareness of available professional development resources; still another may uncover a lack of realism regarding the time or skills it takes to achieve a particular career goal. These realities highlight an imperative for our field to develop robust and sustainable mentorship programs for not only early-career hospitalists but also some mid-career hospitalists whose careers may not yet be fully developed. Indeed, one of the biggest challenges that have emerged in our experience with a faculty development program at Penn is creating meaningful mentorship and career development advice for mid-career hospitalists (late assistant or early associate professors who are typically 5-10 years into their careers).

We found it interesting that the hospitalists interviewed did not mention three of the four pillars of career satisfaction outlined in the white paper on Hospitalist Career Satisfaction from the Society for Hospital Medicine: workload schedule, autonomy control, and community/environment.⁶ Perhaps this is because hospitalists, like many other professionals, recognize that feeling satisfied in one’s career is not the same as feeling successful. Satisfaction in one’s career refers to the foundational needs that one requires in order to feel content, whereas success is more often equated with achievement, even if that achievement is simply the acquisition of one’s goals for themselves. The reality is that given the constant growth and change within teaching hospitals, and therefore academic hospital medicine groups, tending to the satisfiers for hospitalists (eg, schedule and workload) often takes a front seat to assisting faculty in achieving their individual career potential. We assert that despite the inherent difficulty, academic hospital medicine group leaders need to focus their attention on both the satisfaction and career success of their early-career faculty.

Finally, this paper raises many interesting questions for researchers interested in the professional development of hospitalists. Are the career success perspectives of an early-career academic hospitalist different from those of an early-career intensivist or emergency medicine physician in an academic setting? Hospital medicine has historically been likened to both fields given the similar intensity of clinical work and the fact that all three fields were created around the need for specialists in a care setting as opposed to a disease state. It is possible that the vision of success for young academic physicians as a whole has changed with the millennial generation entering the workforce. Do early-career hospitalists look different from early-career general internists in academic settings? The latter group has more promoted faculty in their division to serve as role models and mentors and who have demonstrated more success in a variety of replicable career pathways. The fact that the definition of career success may evolve over time also emerged as a theme from this paper. Do mid-career academic hospitalists find that the excitement for daily clinical work wanes over time leaving them feeling less successful and looking for something more?

In conclusion, the findings of Cumbler et al. should promote unrest among leaders of academic hospital medicine groups and their departments of medicine. While it is inspiring to see so many early-career hospitalists focused on their daily happiness at work, we are unsure about whether they have the knowledge, tools, and guidance to achieve their self-professed academic goals, which many equate with career success. Given the continued growth of the hospital medicine workforce, we view this important new work as a national call to arms for the purposeful development of academic hospitalist faculty development programs.

Disclosures

Dr. Myers and Dr. Greysen have nothing to disclose.

Over the past two decades, the field of hospital medicine has gone from relative obscurity to a viable career pathway for approximately 50,000 physicians in this country.¹ A subset of hospitalists pursue careers in academic medicine, which is a pathway that traditionally includes education and scholarship in addition to patient care. While the academic career pathway is well paved in many clinical specialties, it is still relatively underdeveloped for academic hospitalists, and thus what defines career success for this group is even less clear.

In this issue of the Journal of Hospital Medicine, Cumbler et al. performed a qualitative analysis to explore how early career academic hospitalists self-define and perceive their career success.² Drawing on interviews with 17 early-career hospitalists at 3 academic medical centers, the authors created a theoretical framework organized around a traditional conceptual model of career success that is divided into intrinsic and extrinsic motivating factors. They found that early-career academic hospitalists, (clinician-educators in first 2-5 years), defined their career success almost exclusively around factors intrinsic to their day-to-day job. These factors included such things as excitement about their daily work, developing proficiency in the delivery of high-quality clinical care, and passion for doing work that is meaningful to them. In addition to these immediate job satisfiers, many hospitalists emphasized long-term career success factors such as becoming an expert in a particular domain of hospital medicine and gaining respect and recognition within their local or national environment. Surprisingly, compensation and career advancement through promotion, two traditional external career success factors, were not uniformly valued.

These findings come at a critical time for our field in which early-career faculty outnumber mid- and late-career faculty by an order of magnitude. Indeed, how to develop, promote, sustain, and retain young hospitalists is a topic on the minds of most hospital medicine group directors. Putting aside the impact of hospitalist turnover on productivity, patient care outcomes, and morale within an individual hospital medicine group, we agree with the authors that understanding and cultivating career success for academic hospitalists is imperative for the future of our field. For this reason, we launched a formal faculty development program at Penn this year, which focuses on supporting the growth of hospitalists in their first two years on faculty. The findings of this study provide interesting new perspectives and encourage us to continue our focus on early-career academic hospitalists. We laud the previous efforts in this area and hope that the paper by Cumbler et al. encourages and inspires other programs to start or accelerate their hospitalist faculty development efforts.^3-5

However, some findings from this study are somewhat perplexing or even a bit discouraging for those who are invested in faculty development in academia. For example, the authors raise the possibility that there may be a disconnect in the minds of early-career hospitalists as it pertains to their thoughts on career success. On the one hand, the hospitalists interviewed in this study are happy doing their clinical work and cite this as a primary driver of their career success. On the other hand, they equate career success with things such as developing expertise within a particular domain of hospital medicine, acquiring leadership roles, collaborating academically with other specialties or professions, or developing new innovations. Presumably this is part of the reason that they selected a job in an academic setting as opposed to a community setting. However, in order to achieve these goals, one must devote time and effort to purposefully developing them. Therefore, identifying and developing mentors who can assist early-career hospitalists with identifying, articulating, and developing strategies to achieve both their short- and long-term career goals is critical. One mentor–mentee conversation may reveal that an individual hospitalist values being an excellent clinician and has little interest in developing a niche within hospital medicine; another may reveal a lack of awareness of available professional development resources; still another may uncover a lack of realism regarding the time or skills it takes to achieve a particular career goal. These realities highlight an imperative for our field to develop robust and sustainable mentorship programs for not only early-career hospitalists but also some mid-career hospitalists whose careers may not yet be fully developed. Indeed, one of the biggest challenges that have emerged in our experience with a faculty development program at Penn is creating meaningful mentorship and career development advice for mid-career hospitalists (late assistant or early associate professors who are typically 5-10 years into their careers).

We found it interesting that the hospitalists interviewed did not mention three of the four pillars of career satisfaction outlined in the white paper on Hospitalist Career Satisfaction from the Society for Hospital Medicine: workload schedule, autonomy control, and community/environment.⁶ Perhaps this is because hospitalists, like many other professionals, recognize that feeling satisfied in one’s career is not the same as feeling successful. Satisfaction in one’s career refers to the foundational needs that one requires in order to feel content, whereas success is more often equated with achievement, even if that achievement is simply the acquisition of one’s goals for themselves. The reality is that given the constant growth and change within teaching hospitals, and therefore academic hospital medicine groups, tending to the satisfiers for hospitalists (eg, schedule and workload) often takes a front seat to assisting faculty in achieving their individual career potential. We assert that despite the inherent difficulty, academic hospital medicine group leaders need to focus their attention on both the satisfaction and career success of their early-career faculty.

Finally, this paper raises many interesting questions for researchers interested in the professional development of hospitalists. Are the career success perspectives of an early-career academic hospitalist different from those of an early-career intensivist or emergency medicine physician in an academic setting? Hospital medicine has historically been likened to both fields given the similar intensity of clinical work and the fact that all three fields were created around the need for specialists in a care setting as opposed to a disease state. It is possible that the vision of success for young academic physicians as a whole has changed with the millennial generation entering the workforce. Do early-career hospitalists look different from early-career general internists in academic settings? The latter group has more promoted faculty in their division to serve as role models and mentors and who have demonstrated more success in a variety of replicable career pathways. The fact that the definition of career success may evolve over time also emerged as a theme from this paper. Do mid-career academic hospitalists find that the excitement for daily clinical work wanes over time leaving them feeling less successful and looking for something more?

In conclusion, the findings of Cumbler et al. should promote unrest among leaders of academic hospital medicine groups and their departments of medicine. While it is inspiring to see so many early-career hospitalists focused on their daily happiness at work, we are unsure about whether they have the knowledge, tools, and guidance to achieve their self-professed academic goals, which many equate with career success. Given the continued growth of the hospital medicine workforce, we view this important new work as a national call to arms for the purposeful development of academic hospitalist faculty development programs.

Disclosures

Dr. Myers and Dr. Greysen have nothing to disclose.

Given growing incentives to reduce readmission rates, predischarge checklists and bundles have recommended that inpatient providers schedule postdischarge follow-up visits (PDFVs) for their hospitalized patients.^1-4 PDFVs have been linked to lower readmission rates in patients with chronic conditions, including congestive heart failure, psychiatric illnesses, and chronic obstructive pulmonary disease.^5-8 In contrast, the impact of PDFVs on readmissions in hospitalized general medicine populations has been mixed.^9-12 Beyond the presence or absence of PDFVs, it may be a patient’s inability to keep scheduled PDFVs that contributes more strongly to preventable readmissions.¹¹

This challenge, dealing with the 12% to 37% of patients who miss their visits (“no-shows”), is not new.^13-17 In high-risk patient populations, such as those with substance abuse, diabetes, or human immunodeficiency virus, no-shows (NSs) have been linked to poorer short-term and long-term clinical outcomes.^16,18-20 Additionally, NSs pose a challenge for outpatient clinics and the healthcare system at large. The financial cost of NSs ranges from approximately $200 per patient in 2 analyses to $7 million in cumulative lost revenue per year at 1 large academic health system.^13,17,21 As such, increasing attendance at PDFVs is a potential target for improving both patient outcomes and clinic productivity.

Most prior PDFV research has focused on readmission risk rather than PDFV attendance as the primary outcome.^5-12 However, given the patient-oriented benefits of attending PDFVs and the clinic-oriented benefits of avoiding vacant time slots, NS PDFVs represent an important missed opportunity for our healthcare delivery system. To our knowledge, risk factors for PDFV nonattendance have not yet been systematically studied. The aim of our study was to analyze PDFV nonattendance, particularly NSs and same-day cancellations (SDCs), for hospitalizations and clinics within our healthcare system.

Study Design

We conducted an observational cohort study of adult patients from 10 medical units at the Hospital of the University of Pennsylvania (a 789-bed quaternary-care hospital within an urban, academic medical system) who were scheduled with at least 1 PDFV. Specifically, the patients included in our analysis were hospitalized on general internal medicine services or medical subspecialty services with discharge dates between April 1, 2014, and March 31, 2015. Hospitalizations included in our study had at least 1 PDFV scheduled with an outpatient provider affiliated with the University of Pennsylvania Health System (UPHS). PDFVs scheduled with unaffiliated providers were not examined.

Each PDFV was requested by a patient’s inpatient care team. Once the care team had determined that a PDFV was clinically warranted, a member of the team (generally a resident, advanced practice provider, medical student, or designee) either called the UPHS clinic to schedule an appointment time or e-mailed the outpatient UPHS provider directly to facilitate a more urgent PDFV appointment time. Once a PDFV time was confirmed, PDFV details (ie, date, time, location, and phone number) were electronically entered into the patient’s discharge instructions by the inpatient care team. At the time of discharge, nurses reviewed these instructions with their patients. All patients left the hospital with a physical copy of these instructions. As part of routine care at our institution, patients then received automated telephone reminders from their UPHS-affiliated outpatient clinic 48 hours prior to each PDFV.

Data Collection

Our study was determined to meet criteria for quality improvement by the University of Pennsylvania’s Institutional Review Board. We used our healthcare system’s integrated electronic medical record system to track the dates of initial PDFV requests, the dates of hospitalization, and actual PDFV dates. PDFVs were included if the appointment request was made while a patient was hospitalized, including the day of discharge. Our study methodology only allowed us to investigate PDFVs scheduled with UPHS outpatient providers. We did not review discharge instructions or survey non-UPHS clinics to quantify visits scheduled with other providers, for example, community health centers or external private practices.

Exclusion criteria included the following: (1) office visits with nonproviders, for example, scheduled diagnostic procedures or pharmacist appointments for warfarin dosing; (2) visits cancelled by inpatient providers prior to discharge; (3) visits for patients not otherwise eligible for UPHS outpatient care because of insurance reasons; and (4) visits scheduled for dates after a patient’s death. Our motivation for the third exclusion criterion was the infrequent and irregular process by which PDFVs were authorized for these patients. These patients and their characteristics are described in Supplementary Table 1 in more detail.

For each PDFV, we recorded age, gender, race, insurance status, driving distance, length of stay for index hospitalization, discharging service (general internal medicine vs subspecialty), postdischarge disposition (home, home with home care services such as nursing or physical therapy, or facility), the number of PDFVs scheduled per index hospitalization, PDFV specialty type (oncologic subspecialty, nononcologic medical subspecialty, nononcologic surgical subspecialty, primary care, or other specialty), PDFV season, and PDFV lead time (the number of days between the discharge date and PDFV). We consolidated oncologic specialties into 1 group given the integrated nature of our healthcare system’s comprehensive cancer center. “Other” PDFV specialty subtypes are described in Supplementary Table 2. Driving distances between patient postal codes and our hospital were calculated using Excel VBA Master (Salt Lake City, Utah) and were subsequently categorized into patient-level quartiles for further analysis. For cancelled PDFVs, we collected dates of cancellation relative to the date of the appointment itself.

Study Outcomes

The primary study outcome was PDFV attendance. Each PDFV’s status was categorized by outpatient clinic staff as attended, cancelled, or NS. For cancelled appointments, cancellation dates and reasons (if entered by clinic representatives) were collected. In keeping with prior studies investigating outpatient nonattendance,we calculated collective NS/SDC rates for the variables listed above.^17,22-25 We additionally calculated NS/SDC and attendance-as-scheduled rates stratified by the number of PDFVs per patient to assess for a “high-utilizer” effect with regard to PDFV attendance.

Statistical Analysis

We used multivariable mixed-effects regression with a logit link to assess associations between age, gender, race, insurance, driving distance quartile, length of stay, discharging service, postdischarge disposition, the number of PDFVs per hospitalization, PDFV specialty type, PDFV season, PDFV lead time, and our NS/SDC outcome. The mixed-effects approach was used to account for correlation structures induced by patients who had multiple visits and for patients with multiple hospitalizations. Specifically, our model specified 2 levels of nesting (PDFVs nested within each hospitalization, which were nested within each patient) to obtain appropriate standard error estimates for our adjusted odds ratios (ORs). Correlation matrices and multivariable variance inflation factors were used to assess collinearity among the predictor variables. These assessments did not indicate strong collinearity; hence, all predictors were included in the model. Only driving distance had a small amount of missing data (0.18% of driving distances were unavailable), so multiple imputation was not undertaken. Analyses were performed using R version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

Baseline Characteristics

During the 1-year study period, there were 11,829 discrete hospitalizations in medical units at our hospital. Of these hospitalizations, 6136 (52%) had at least 1 UPHS-affiliated PDFV meeting our inclusion and exclusion criteria, as detailed in the Figure. Across these hospitalizations, 9258 PDFVs were scheduled on behalf of 4653 patients. Demographic characteristics for these patients, hospitalizations, and visits are detailed in Table 1. The median age of patients in our cohort was 61 years old (interquartile range [IQR] 49-70, range 18-101). The median driving distance was 17 miles (IQR 4.3-38.8, range 0-2891). For hospitalizations, the median length of stay was 5 days (IQR 3-10, range 0-97). The median PDFV lead time, which is defined as the number of days between discharge and PDFV, was 12 days (IQR 6-23, range 1-60). Overall, 41% of patients (n = 1927) attended all of their PDFVs as scheduled; Supplementary Figure 1 lists patient-level PDFV attendance-as-scheduled percentages in more detail.

Incidence of NSs and SDCs

Twenty-five percent of PDFVs (n = 2303) were ultimately NS/SDCs; this included 1658 NSs (18% of all appointments) and 645 SDCs (7% of all appointments). Fifty-two percent of PDFVs (n = 4847) were kept as scheduled, while 23% (n = 2108) were cancelled before the day of the visit. Of the 2558 cancellations with valid cancellation dates, 49% (n = 1252) were cancelled 2 or fewer days beforehand, as shown in Supplementary Figure 2.

When comparing PDFV characteristics themselves, oncologic visits had the lowest NS/SDC incidence of any group analyzed in our study. This may be related to the inherent life-altering nature of a cancer diagnosis or our cancer center’s use of patient navigators.^23,30 In contrast, primary care clinics suffered from NS/SDC rates approaching 40%, which is a concerning finding given the importance of primary care coordination in the posthospitalization period.^9,31 Why are primary care appointments so commonly missed? Some studies suggest that forgetting about a primary care appointment is a leading reason.^15,32,33 For PDFVs, this phenomenon may be augmented because the visits are not scheduled by patients themselves. Additionally, patients may paradoxically undervalue the benefit of an all-encompassing primary care visit, compared to a PDFV focused on a specific problem, (eg, a cardiology follow-up appointment for a patient with congestive heart failure). In particular, patients with limited health literacy may potentially undervalue the capabilities of their primary care clinics.^34,35

The low absolute number of primary care PDFVs (only 8% of all visits) scheduled for patients at our hospital was an unexpected finding. This low percentage is likely a function of the patient population hospitalized at our large, urban quaternary-care facility. First, an unknown number of patients may have had PDFVs manually scheduled with primary care providers external to our health system; these PDFVs were not captured within our study. Second, 71% of the hospitalizations in our study occurred in subspecialty services, for which specific primary care follow-up may not be as urgent. Supporting this fact, further analysis of the 6136 hospitalizations in our study (data not shown) revealed that 28% of the hospitalizations in general internal medicine were scheduled with at least 1 primary care PDFV as opposed to only 5% of subspecialty-service hospitalizations.

In contrast to several previous studies of outpatient nonattendance,we did not find that visits scheduled for time points further in the future were more likely to be missed.^{14,24,25,36,37} Unlike other appointments, it may be that PDFV lead time does not affect attendance because of the unique manner in which PDFV times are scheduled and conveyed to patients. Unlike other appointments, patients do not schedule PDFVs themselves but instead learn about their PDFV dates as part of a large set of discharge instructions. This practice may result in poor recall of PDFV dates in recently hospitalized patients³⁸, regardless of the lead time between discharge and the visit itself.

Supplementary Table 1 details a 51% NS/SDC rate for the small number of PDFVs (n = 65) that were excluded a priori from our analysis because of general ineligibility for UPHS outpatient care. We specifically chose to exclude this population because of the infrequent and irregular process by which these PDFVs were authorized on a case-by-case basis, typically via active engagement by our hospital’s social work department. We did not study this population further but postulate that the 51% NS/SDC rate may reflect other social determinants of health that contribute to appointment nonadherence in a predominantly uninsured population.

Beyond their effect on patient outcomes, improving PDFV-related processes has the potential to boost both inpatient and outpatient provider satisfaction. From the standpoint of frontline inpatient providers (often resident physicians), calling outpatient clinics to request PDFVs is viewed as 1 of the top 5 administrative tasks that interfere with house staff education.³⁹ Future interventions that involve patients in the PDFV scheduling process may improve inpatient workflow while simultaneously engaging patients in their own care. For example, asking clinic representatives to directly schedule PDFVs with hospitalized patients, either by phone or in person, has been shown in pilot studies to improve PDFV attendance and decrease readmissions.^40-42 Conversely, NS/SDC visits harm outpatient provider productivity and decrease provider availability for other patients.^13,17,43 Strategies to mitigate the impact of unfilled appointment slots (eg, deliberately overbooking time slots in advance) carry their own risks, including provider burnout.⁴⁴ As such, preventing NSs may be superior to curing their adverse impacts. Many such strategies exist in the ambulatory setting,^13,43,45 for example, better communication with patients through texting or goal-directed, personalized phone reminders.^46-48Our study methodology has several limitations. Most importantly, we were unable to measure PDFVs made with providers unaffiliated with UPHS. As previously noted, our low proportion of primary care PDFVs may specifically reflect patients with primary care providers outside of our health system. Similarly, our low percentage of Medicaid patients receiving PDFVs may be related to follow-up visits with nonaffiliated community health centers. We were unable to measure patient acuity and health literacy as potential predictors of NS/SDC rates. Driving distances were calculated from patient postal codes to our hospital, not to individual outpatient clinics. However, the majority of our hospital-affiliated clinics are located adjacent to our hospital; additionally, we grouped driving distances into quartiles for our analysis. We had initially attempted to differentiate between clinic-initiated and patient-initiated cancellations, but unfortunately, we found that the data were too unreliable to be used for further analysis (outlined in Supplementary Table 3). Lastly, because we studied patients in medical units at a single large, urban, academic center, our results are not generalizable to other settings (eg, community hospitals, hospitals with smaller networks of outpatient providers, or patients being discharged from surgical services or observation units).

Given national efforts to enhance postdischarge transitions of care, we aimed to analyze attendance at provider-scheduled PDFV appointments. Our finding that 25% of PDFVs resulted in NS/SDCs raises both questions and opportunities for inpatient and outpatient providers. Further research is needed to understand why so many patients miss their PDFVs, and we should work as a field to develop creative solutions to improve PDFV scheduling and attendance.

Acknowledgments

The authors acknowledge Marie Synnestvedt, PhD, and Manik Chhabra, MD, for their assistance with data gathering and statistical analysis. They also acknowledge Allison DeKosky, MD, Michael Serpa, BS, Michael McFall, and Scott Schlegel, MBA, for their assistance with researching this topic. They did not receive external compensation for their assistance outside of their usual salary support.

DISCLOSURE

Nothing to report.

Given growing incentives to reduce readmission rates, predischarge checklists and bundles have recommended that inpatient providers schedule postdischarge follow-up visits (PDFVs) for their hospitalized patients.^1-4 PDFVs have been linked to lower readmission rates in patients with chronic conditions, including congestive heart failure, psychiatric illnesses, and chronic obstructive pulmonary disease.^5-8 In contrast, the impact of PDFVs on readmissions in hospitalized general medicine populations has been mixed.^9-12 Beyond the presence or absence of PDFVs, it may be a patient’s inability to keep scheduled PDFVs that contributes more strongly to preventable readmissions.¹¹

This challenge, dealing with the 12% to 37% of patients who miss their visits (“no-shows”), is not new.^13-17 In high-risk patient populations, such as those with substance abuse, diabetes, or human immunodeficiency virus, no-shows (NSs) have been linked to poorer short-term and long-term clinical outcomes.^16,18-20 Additionally, NSs pose a challenge for outpatient clinics and the healthcare system at large. The financial cost of NSs ranges from approximately $200 per patient in 2 analyses to $7 million in cumulative lost revenue per year at 1 large academic health system.^13,17,21 As such, increasing attendance at PDFVs is a potential target for improving both patient outcomes and clinic productivity.

Most prior PDFV research has focused on readmission risk rather than PDFV attendance as the primary outcome.^5-12 However, given the patient-oriented benefits of attending PDFVs and the clinic-oriented benefits of avoiding vacant time slots, NS PDFVs represent an important missed opportunity for our healthcare delivery system. To our knowledge, risk factors for PDFV nonattendance have not yet been systematically studied. The aim of our study was to analyze PDFV nonattendance, particularly NSs and same-day cancellations (SDCs), for hospitalizations and clinics within our healthcare system.

Study Design

We conducted an observational cohort study of adult patients from 10 medical units at the Hospital of the University of Pennsylvania (a 789-bed quaternary-care hospital within an urban, academic medical system) who were scheduled with at least 1 PDFV. Specifically, the patients included in our analysis were hospitalized on general internal medicine services or medical subspecialty services with discharge dates between April 1, 2014, and March 31, 2015. Hospitalizations included in our study had at least 1 PDFV scheduled with an outpatient provider affiliated with the University of Pennsylvania Health System (UPHS). PDFVs scheduled with unaffiliated providers were not examined.

Each PDFV was requested by a patient’s inpatient care team. Once the care team had determined that a PDFV was clinically warranted, a member of the team (generally a resident, advanced practice provider, medical student, or designee) either called the UPHS clinic to schedule an appointment time or e-mailed the outpatient UPHS provider directly to facilitate a more urgent PDFV appointment time. Once a PDFV time was confirmed, PDFV details (ie, date, time, location, and phone number) were electronically entered into the patient’s discharge instructions by the inpatient care team. At the time of discharge, nurses reviewed these instructions with their patients. All patients left the hospital with a physical copy of these instructions. As part of routine care at our institution, patients then received automated telephone reminders from their UPHS-affiliated outpatient clinic 48 hours prior to each PDFV.

Data Collection

Our study was determined to meet criteria for quality improvement by the University of Pennsylvania’s Institutional Review Board. We used our healthcare system’s integrated electronic medical record system to track the dates of initial PDFV requests, the dates of hospitalization, and actual PDFV dates. PDFVs were included if the appointment request was made while a patient was hospitalized, including the day of discharge. Our study methodology only allowed us to investigate PDFVs scheduled with UPHS outpatient providers. We did not review discharge instructions or survey non-UPHS clinics to quantify visits scheduled with other providers, for example, community health centers or external private practices.

Exclusion criteria included the following: (1) office visits with nonproviders, for example, scheduled diagnostic procedures or pharmacist appointments for warfarin dosing; (2) visits cancelled by inpatient providers prior to discharge; (3) visits for patients not otherwise eligible for UPHS outpatient care because of insurance reasons; and (4) visits scheduled for dates after a patient’s death. Our motivation for the third exclusion criterion was the infrequent and irregular process by which PDFVs were authorized for these patients. These patients and their characteristics are described in Supplementary Table 1 in more detail.

For each PDFV, we recorded age, gender, race, insurance status, driving distance, length of stay for index hospitalization, discharging service (general internal medicine vs subspecialty), postdischarge disposition (home, home with home care services such as nursing or physical therapy, or facility), the number of PDFVs scheduled per index hospitalization, PDFV specialty type (oncologic subspecialty, nononcologic medical subspecialty, nononcologic surgical subspecialty, primary care, or other specialty), PDFV season, and PDFV lead time (the number of days between the discharge date and PDFV). We consolidated oncologic specialties into 1 group given the integrated nature of our healthcare system’s comprehensive cancer center. “Other” PDFV specialty subtypes are described in Supplementary Table 2. Driving distances between patient postal codes and our hospital were calculated using Excel VBA Master (Salt Lake City, Utah) and were subsequently categorized into patient-level quartiles for further analysis. For cancelled PDFVs, we collected dates of cancellation relative to the date of the appointment itself.

Study Outcomes

The primary study outcome was PDFV attendance. Each PDFV’s status was categorized by outpatient clinic staff as attended, cancelled, or NS. For cancelled appointments, cancellation dates and reasons (if entered by clinic representatives) were collected. In keeping with prior studies investigating outpatient nonattendance,we calculated collective NS/SDC rates for the variables listed above.^17,22-25 We additionally calculated NS/SDC and attendance-as-scheduled rates stratified by the number of PDFVs per patient to assess for a “high-utilizer” effect with regard to PDFV attendance.

Statistical Analysis

We used multivariable mixed-effects regression with a logit link to assess associations between age, gender, race, insurance, driving distance quartile, length of stay, discharging service, postdischarge disposition, the number of PDFVs per hospitalization, PDFV specialty type, PDFV season, PDFV lead time, and our NS/SDC outcome. The mixed-effects approach was used to account for correlation structures induced by patients who had multiple visits and for patients with multiple hospitalizations. Specifically, our model specified 2 levels of nesting (PDFVs nested within each hospitalization, which were nested within each patient) to obtain appropriate standard error estimates for our adjusted odds ratios (ORs). Correlation matrices and multivariable variance inflation factors were used to assess collinearity among the predictor variables. These assessments did not indicate strong collinearity; hence, all predictors were included in the model. Only driving distance had a small amount of missing data (0.18% of driving distances were unavailable), so multiple imputation was not undertaken. Analyses were performed using R version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

Baseline Characteristics

During the 1-year study period, there were 11,829 discrete hospitalizations in medical units at our hospital. Of these hospitalizations, 6136 (52%) had at least 1 UPHS-affiliated PDFV meeting our inclusion and exclusion criteria, as detailed in the Figure. Across these hospitalizations, 9258 PDFVs were scheduled on behalf of 4653 patients. Demographic characteristics for these patients, hospitalizations, and visits are detailed in Table 1. The median age of patients in our cohort was 61 years old (interquartile range [IQR] 49-70, range 18-101). The median driving distance was 17 miles (IQR 4.3-38.8, range 0-2891). For hospitalizations, the median length of stay was 5 days (IQR 3-10, range 0-97). The median PDFV lead time, which is defined as the number of days between discharge and PDFV, was 12 days (IQR 6-23, range 1-60). Overall, 41% of patients (n = 1927) attended all of their PDFVs as scheduled; Supplementary Figure 1 lists patient-level PDFV attendance-as-scheduled percentages in more detail.

Incidence of NSs and SDCs

Twenty-five percent of PDFVs (n = 2303) were ultimately NS/SDCs; this included 1658 NSs (18% of all appointments) and 645 SDCs (7% of all appointments). Fifty-two percent of PDFVs (n = 4847) were kept as scheduled, while 23% (n = 2108) were cancelled before the day of the visit. Of the 2558 cancellations with valid cancellation dates, 49% (n = 1252) were cancelled 2 or fewer days beforehand, as shown in Supplementary Figure 2.

When comparing PDFV characteristics themselves, oncologic visits had the lowest NS/SDC incidence of any group analyzed in our study. This may be related to the inherent life-altering nature of a cancer diagnosis or our cancer center’s use of patient navigators.^23,30 In contrast, primary care clinics suffered from NS/SDC rates approaching 40%, which is a concerning finding given the importance of primary care coordination in the posthospitalization period.^9,31 Why are primary care appointments so commonly missed? Some studies suggest that forgetting about a primary care appointment is a leading reason.^15,32,33 For PDFVs, this phenomenon may be augmented because the visits are not scheduled by patients themselves. Additionally, patients may paradoxically undervalue the benefit of an all-encompassing primary care visit, compared to a PDFV focused on a specific problem, (eg, a cardiology follow-up appointment for a patient with congestive heart failure). In particular, patients with limited health literacy may potentially undervalue the capabilities of their primary care clinics.^34,35

The low absolute number of primary care PDFVs (only 8% of all visits) scheduled for patients at our hospital was an unexpected finding. This low percentage is likely a function of the patient population hospitalized at our large, urban quaternary-care facility. First, an unknown number of patients may have had PDFVs manually scheduled with primary care providers external to our health system; these PDFVs were not captured within our study. Second, 71% of the hospitalizations in our study occurred in subspecialty services, for which specific primary care follow-up may not be as urgent. Supporting this fact, further analysis of the 6136 hospitalizations in our study (data not shown) revealed that 28% of the hospitalizations in general internal medicine were scheduled with at least 1 primary care PDFV as opposed to only 5% of subspecialty-service hospitalizations.

In contrast to several previous studies of outpatient nonattendance,we did not find that visits scheduled for time points further in the future were more likely to be missed.^{14,24,25,36,37} Unlike other appointments, it may be that PDFV lead time does not affect attendance because of the unique manner in which PDFV times are scheduled and conveyed to patients. Unlike other appointments, patients do not schedule PDFVs themselves but instead learn about their PDFV dates as part of a large set of discharge instructions. This practice may result in poor recall of PDFV dates in recently hospitalized patients³⁸, regardless of the lead time between discharge and the visit itself.

Supplementary Table 1 details a 51% NS/SDC rate for the small number of PDFVs (n = 65) that were excluded a priori from our analysis because of general ineligibility for UPHS outpatient care. We specifically chose to exclude this population because of the infrequent and irregular process by which these PDFVs were authorized on a case-by-case basis, typically via active engagement by our hospital’s social work department. We did not study this population further but postulate that the 51% NS/SDC rate may reflect other social determinants of health that contribute to appointment nonadherence in a predominantly uninsured population.

Beyond their effect on patient outcomes, improving PDFV-related processes has the potential to boost both inpatient and outpatient provider satisfaction. From the standpoint of frontline inpatient providers (often resident physicians), calling outpatient clinics to request PDFVs is viewed as 1 of the top 5 administrative tasks that interfere with house staff education.³⁹ Future interventions that involve patients in the PDFV scheduling process may improve inpatient workflow while simultaneously engaging patients in their own care. For example, asking clinic representatives to directly schedule PDFVs with hospitalized patients, either by phone or in person, has been shown in pilot studies to improve PDFV attendance and decrease readmissions.^40-42 Conversely, NS/SDC visits harm outpatient provider productivity and decrease provider availability for other patients.^13,17,43 Strategies to mitigate the impact of unfilled appointment slots (eg, deliberately overbooking time slots in advance) carry their own risks, including provider burnout.⁴⁴ As such, preventing NSs may be superior to curing their adverse impacts. Many such strategies exist in the ambulatory setting,^13,43,45 for example, better communication with patients through texting or goal-directed, personalized phone reminders.^46-48Our study methodology has several limitations. Most importantly, we were unable to measure PDFVs made with providers unaffiliated with UPHS. As previously noted, our low proportion of primary care PDFVs may specifically reflect patients with primary care providers outside of our health system. Similarly, our low percentage of Medicaid patients receiving PDFVs may be related to follow-up visits with nonaffiliated community health centers. We were unable to measure patient acuity and health literacy as potential predictors of NS/SDC rates. Driving distances were calculated from patient postal codes to our hospital, not to individual outpatient clinics. However, the majority of our hospital-affiliated clinics are located adjacent to our hospital; additionally, we grouped driving distances into quartiles for our analysis. We had initially attempted to differentiate between clinic-initiated and patient-initiated cancellations, but unfortunately, we found that the data were too unreliable to be used for further analysis (outlined in Supplementary Table 3). Lastly, because we studied patients in medical units at a single large, urban, academic center, our results are not generalizable to other settings (eg, community hospitals, hospitals with smaller networks of outpatient providers, or patients being discharged from surgical services or observation units).

Given national efforts to enhance postdischarge transitions of care, we aimed to analyze attendance at provider-scheduled PDFV appointments. Our finding that 25% of PDFVs resulted in NS/SDCs raises both questions and opportunities for inpatient and outpatient providers. Further research is needed to understand why so many patients miss their PDFVs, and we should work as a field to develop creative solutions to improve PDFV scheduling and attendance.

Acknowledgments

The authors acknowledge Marie Synnestvedt, PhD, and Manik Chhabra, MD, for their assistance with data gathering and statistical analysis. They also acknowledge Allison DeKosky, MD, Michael Serpa, BS, Michael McFall, and Scott Schlegel, MBA, for their assistance with researching this topic. They did not receive external compensation for their assistance outside of their usual salary support.

DISCLOSURE

Nothing to report.

Given growing incentives to reduce readmission rates, predischarge checklists and bundles have recommended that inpatient providers schedule postdischarge follow-up visits (PDFVs) for their hospitalized patients.^1-4 PDFVs have been linked to lower readmission rates in patients with chronic conditions, including congestive heart failure, psychiatric illnesses, and chronic obstructive pulmonary disease.^5-8 In contrast, the impact of PDFVs on readmissions in hospitalized general medicine populations has been mixed.^9-12 Beyond the presence or absence of PDFVs, it may be a patient’s inability to keep scheduled PDFVs that contributes more strongly to preventable readmissions.¹¹

This challenge, dealing with the 12% to 37% of patients who miss their visits (“no-shows”), is not new.^13-17 In high-risk patient populations, such as those with substance abuse, diabetes, or human immunodeficiency virus, no-shows (NSs) have been linked to poorer short-term and long-term clinical outcomes.^16,18-20 Additionally, NSs pose a challenge for outpatient clinics and the healthcare system at large. The financial cost of NSs ranges from approximately $200 per patient in 2 analyses to $7 million in cumulative lost revenue per year at 1 large academic health system.^13,17,21 As such, increasing attendance at PDFVs is a potential target for improving both patient outcomes and clinic productivity.

Most prior PDFV research has focused on readmission risk rather than PDFV attendance as the primary outcome.^5-12 However, given the patient-oriented benefits of attending PDFVs and the clinic-oriented benefits of avoiding vacant time slots, NS PDFVs represent an important missed opportunity for our healthcare delivery system. To our knowledge, risk factors for PDFV nonattendance have not yet been systematically studied. The aim of our study was to analyze PDFV nonattendance, particularly NSs and same-day cancellations (SDCs), for hospitalizations and clinics within our healthcare system.

Study Design

We conducted an observational cohort study of adult patients from 10 medical units at the Hospital of the University of Pennsylvania (a 789-bed quaternary-care hospital within an urban, academic medical system) who were scheduled with at least 1 PDFV. Specifically, the patients included in our analysis were hospitalized on general internal medicine services or medical subspecialty services with discharge dates between April 1, 2014, and March 31, 2015. Hospitalizations included in our study had at least 1 PDFV scheduled with an outpatient provider affiliated with the University of Pennsylvania Health System (UPHS). PDFVs scheduled with unaffiliated providers were not examined.

Each PDFV was requested by a patient’s inpatient care team. Once the care team had determined that a PDFV was clinically warranted, a member of the team (generally a resident, advanced practice provider, medical student, or designee) either called the UPHS clinic to schedule an appointment time or e-mailed the outpatient UPHS provider directly to facilitate a more urgent PDFV appointment time. Once a PDFV time was confirmed, PDFV details (ie, date, time, location, and phone number) were electronically entered into the patient’s discharge instructions by the inpatient care team. At the time of discharge, nurses reviewed these instructions with their patients. All patients left the hospital with a physical copy of these instructions. As part of routine care at our institution, patients then received automated telephone reminders from their UPHS-affiliated outpatient clinic 48 hours prior to each PDFV.

Data Collection

Our study was determined to meet criteria for quality improvement by the University of Pennsylvania’s Institutional Review Board. We used our healthcare system’s integrated electronic medical record system to track the dates of initial PDFV requests, the dates of hospitalization, and actual PDFV dates. PDFVs were included if the appointment request was made while a patient was hospitalized, including the day of discharge. Our study methodology only allowed us to investigate PDFVs scheduled with UPHS outpatient providers. We did not review discharge instructions or survey non-UPHS clinics to quantify visits scheduled with other providers, for example, community health centers or external private practices.

Exclusion criteria included the following: (1) office visits with nonproviders, for example, scheduled diagnostic procedures or pharmacist appointments for warfarin dosing; (2) visits cancelled by inpatient providers prior to discharge; (3) visits for patients not otherwise eligible for UPHS outpatient care because of insurance reasons; and (4) visits scheduled for dates after a patient’s death. Our motivation for the third exclusion criterion was the infrequent and irregular process by which PDFVs were authorized for these patients. These patients and their characteristics are described in Supplementary Table 1 in more detail.

For each PDFV, we recorded age, gender, race, insurance status, driving distance, length of stay for index hospitalization, discharging service (general internal medicine vs subspecialty), postdischarge disposition (home, home with home care services such as nursing or physical therapy, or facility), the number of PDFVs scheduled per index hospitalization, PDFV specialty type (oncologic subspecialty, nononcologic medical subspecialty, nononcologic surgical subspecialty, primary care, or other specialty), PDFV season, and PDFV lead time (the number of days between the discharge date and PDFV). We consolidated oncologic specialties into 1 group given the integrated nature of our healthcare system’s comprehensive cancer center. “Other” PDFV specialty subtypes are described in Supplementary Table 2. Driving distances between patient postal codes and our hospital were calculated using Excel VBA Master (Salt Lake City, Utah) and were subsequently categorized into patient-level quartiles for further analysis. For cancelled PDFVs, we collected dates of cancellation relative to the date of the appointment itself.

Study Outcomes

The primary study outcome was PDFV attendance. Each PDFV’s status was categorized by outpatient clinic staff as attended, cancelled, or NS. For cancelled appointments, cancellation dates and reasons (if entered by clinic representatives) were collected. In keeping with prior studies investigating outpatient nonattendance,we calculated collective NS/SDC rates for the variables listed above.^17,22-25 We additionally calculated NS/SDC and attendance-as-scheduled rates stratified by the number of PDFVs per patient to assess for a “high-utilizer” effect with regard to PDFV attendance.

Statistical Analysis

We used multivariable mixed-effects regression with a logit link to assess associations between age, gender, race, insurance, driving distance quartile, length of stay, discharging service, postdischarge disposition, the number of PDFVs per hospitalization, PDFV specialty type, PDFV season, PDFV lead time, and our NS/SDC outcome. The mixed-effects approach was used to account for correlation structures induced by patients who had multiple visits and for patients with multiple hospitalizations. Specifically, our model specified 2 levels of nesting (PDFVs nested within each hospitalization, which were nested within each patient) to obtain appropriate standard error estimates for our adjusted odds ratios (ORs). Correlation matrices and multivariable variance inflation factors were used to assess collinearity among the predictor variables. These assessments did not indicate strong collinearity; hence, all predictors were included in the model. Only driving distance had a small amount of missing data (0.18% of driving distances were unavailable), so multiple imputation was not undertaken. Analyses were performed using R version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).

Baseline Characteristics

During the 1-year study period, there were 11,829 discrete hospitalizations in medical units at our hospital. Of these hospitalizations, 6136 (52%) had at least 1 UPHS-affiliated PDFV meeting our inclusion and exclusion criteria, as detailed in the Figure. Across these hospitalizations, 9258 PDFVs were scheduled on behalf of 4653 patients. Demographic characteristics for these patients, hospitalizations, and visits are detailed in Table 1. The median age of patients in our cohort was 61 years old (interquartile range [IQR] 49-70, range 18-101). The median driving distance was 17 miles (IQR 4.3-38.8, range 0-2891). For hospitalizations, the median length of stay was 5 days (IQR 3-10, range 0-97). The median PDFV lead time, which is defined as the number of days between discharge and PDFV, was 12 days (IQR 6-23, range 1-60). Overall, 41% of patients (n = 1927) attended all of their PDFVs as scheduled; Supplementary Figure 1 lists patient-level PDFV attendance-as-scheduled percentages in more detail.

Incidence of NSs and SDCs

Twenty-five percent of PDFVs (n = 2303) were ultimately NS/SDCs; this included 1658 NSs (18% of all appointments) and 645 SDCs (7% of all appointments). Fifty-two percent of PDFVs (n = 4847) were kept as scheduled, while 23% (n = 2108) were cancelled before the day of the visit. Of the 2558 cancellations with valid cancellation dates, 49% (n = 1252) were cancelled 2 or fewer days beforehand, as shown in Supplementary Figure 2.

When comparing PDFV characteristics themselves, oncologic visits had the lowest NS/SDC incidence of any group analyzed in our study. This may be related to the inherent life-altering nature of a cancer diagnosis or our cancer center’s use of patient navigators.^23,30 In contrast, primary care clinics suffered from NS/SDC rates approaching 40%, which is a concerning finding given the importance of primary care coordination in the posthospitalization period.^9,31 Why are primary care appointments so commonly missed? Some studies suggest that forgetting about a primary care appointment is a leading reason.^15,32,33 For PDFVs, this phenomenon may be augmented because the visits are not scheduled by patients themselves. Additionally, patients may paradoxically undervalue the benefit of an all-encompassing primary care visit, compared to a PDFV focused on a specific problem, (eg, a cardiology follow-up appointment for a patient with congestive heart failure). In particular, patients with limited health literacy may potentially undervalue the capabilities of their primary care clinics.^34,35

The low absolute number of primary care PDFVs (only 8% of all visits) scheduled for patients at our hospital was an unexpected finding. This low percentage is likely a function of the patient population hospitalized at our large, urban quaternary-care facility. First, an unknown number of patients may have had PDFVs manually scheduled with primary care providers external to our health system; these PDFVs were not captured within our study. Second, 71% of the hospitalizations in our study occurred in subspecialty services, for which specific primary care follow-up may not be as urgent. Supporting this fact, further analysis of the 6136 hospitalizations in our study (data not shown) revealed that 28% of the hospitalizations in general internal medicine were scheduled with at least 1 primary care PDFV as opposed to only 5% of subspecialty-service hospitalizations.

In contrast to several previous studies of outpatient nonattendance,we did not find that visits scheduled for time points further in the future were more likely to be missed.^{14,24,25,36,37} Unlike other appointments, it may be that PDFV lead time does not affect attendance because of the unique manner in which PDFV times are scheduled and conveyed to patients. Unlike other appointments, patients do not schedule PDFVs themselves but instead learn about their PDFV dates as part of a large set of discharge instructions. This practice may result in poor recall of PDFV dates in recently hospitalized patients³⁸, regardless of the lead time between discharge and the visit itself.

Supplementary Table 1 details a 51% NS/SDC rate for the small number of PDFVs (n = 65) that were excluded a priori from our analysis because of general ineligibility for UPHS outpatient care. We specifically chose to exclude this population because of the infrequent and irregular process by which these PDFVs were authorized on a case-by-case basis, typically via active engagement by our hospital’s social work department. We did not study this population further but postulate that the 51% NS/SDC rate may reflect other social determinants of health that contribute to appointment nonadherence in a predominantly uninsured population.

Beyond their effect on patient outcomes, improving PDFV-related processes has the potential to boost both inpatient and outpatient provider satisfaction. From the standpoint of frontline inpatient providers (often resident physicians), calling outpatient clinics to request PDFVs is viewed as 1 of the top 5 administrative tasks that interfere with house staff education.³⁹ Future interventions that involve patients in the PDFV scheduling process may improve inpatient workflow while simultaneously engaging patients in their own care. For example, asking clinic representatives to directly schedule PDFVs with hospitalized patients, either by phone or in person, has been shown in pilot studies to improve PDFV attendance and decrease readmissions.^40-42 Conversely, NS/SDC visits harm outpatient provider productivity and decrease provider availability for other patients.^13,17,43 Strategies to mitigate the impact of unfilled appointment slots (eg, deliberately overbooking time slots in advance) carry their own risks, including provider burnout.⁴⁴ As such, preventing NSs may be superior to curing their adverse impacts. Many such strategies exist in the ambulatory setting,^13,43,45 for example, better communication with patients through texting or goal-directed, personalized phone reminders.^46-48Our study methodology has several limitations. Most importantly, we were unable to measure PDFVs made with providers unaffiliated with UPHS. As previously noted, our low proportion of primary care PDFVs may specifically reflect patients with primary care providers outside of our health system. Similarly, our low percentage of Medicaid patients receiving PDFVs may be related to follow-up visits with nonaffiliated community health centers. We were unable to measure patient acuity and health literacy as potential predictors of NS/SDC rates. Driving distances were calculated from patient postal codes to our hospital, not to individual outpatient clinics. However, the majority of our hospital-affiliated clinics are located adjacent to our hospital; additionally, we grouped driving distances into quartiles for our analysis. We had initially attempted to differentiate between clinic-initiated and patient-initiated cancellations, but unfortunately, we found that the data were too unreliable to be used for further analysis (outlined in Supplementary Table 3). Lastly, because we studied patients in medical units at a single large, urban, academic center, our results are not generalizable to other settings (eg, community hospitals, hospitals with smaller networks of outpatient providers, or patients being discharged from surgical services or observation units).

Given national efforts to enhance postdischarge transitions of care, we aimed to analyze attendance at provider-scheduled PDFV appointments. Our finding that 25% of PDFVs resulted in NS/SDCs raises both questions and opportunities for inpatient and outpatient providers. Further research is needed to understand why so many patients miss their PDFVs, and we should work as a field to develop creative solutions to improve PDFV scheduling and attendance.

Acknowledgments

The authors acknowledge Marie Synnestvedt, PhD, and Manik Chhabra, MD, for their assistance with data gathering and statistical analysis. They also acknowledge Allison DeKosky, MD, Michael Serpa, BS, Michael McFall, and Scott Schlegel, MBA, for their assistance with researching this topic. They did not receive external compensation for their assistance outside of their usual salary support.

DISCLOSURE

Nothing to report.

In Reply: We thank Dr. Field for his insights and personal observations related to diagnosis and biases that contribute to diagnostic errors.

Dr. Field’s comment about the importance of revisiting one’s initial working diagnosis is consistent with our proposed diagnostic time out. A diagnostic time out can incorporate a short checklist and aid in debiasing clinicians when findings do not fit the case presentation, such as lack of response to diuretic therapy. Being mindful of slowing down and not necessarily rushing to judgment is another important component.¹ Of note, the residents in our case did revisit their initial working diagnosis, as suggested by Dr. Field. Questions from learners have great potential to serve as debiasing instruments and should always be encouraged. Those who do not work with students can do the same by speaking with nurses or other members of the healthcare team, who offer observations that busy physicians might miss.

Our case highlights the problem that we lack objective criteria to diagnose symptomatic heart failure. While B-type natriuretic factor (BNP) has a strong negative predictive value, serial BNP measurements have not been established to be helpful in the management of heart failure.² Although certain findings on chest radiography have strong positive and negative likelihood associations, the role of serial chest radiographs is less clear.³ Thus, heart failure remains a clinical diagnosis in current practice.

As Dr. Field points out, the accuracy and performance characteristics of diagnostic testing, such as the respiratory rate, need to be considered in conjunction with debiasing strategies to achieve higher diagnostic accuracy. Multiple factors can contribute to low-performing or misinterpreted diagnostic tests, and inaccurate vital signs have been shown to be similarly prone to potential error.⁴

Finally, we wholeheartedly agree with Dr. Field’s comment on unnecessary testing.  High-value care is appropriate care. Using Bayesian reasoning to guide testing, monitoring the treatment course appropriately, and eliminating waste is highly likely to improve both value and diagnostic accuracy. Automated, ritual ordering of daily tests can indicate that thinking has been shut off, leaving clinicians susceptible to premature closure of the diagnostic process as well as the potential for “incidentalomas” to distract them from the right diagnosis, all the while leading to low-value care such as wasteful spending, patient dissatisfaction, and hospital-acquired anemia.⁵ We believe that deciding on a daily basis what the next day’s tests will be can be another powerful debiasing habit, one with benefits beyond diagnosis.

In Reply: We thank Dr. Field for his insights and personal observations related to diagnosis and biases that contribute to diagnostic errors.

Dr. Field’s comment about the importance of revisiting one’s initial working diagnosis is consistent with our proposed diagnostic time out. A diagnostic time out can incorporate a short checklist and aid in debiasing clinicians when findings do not fit the case presentation, such as lack of response to diuretic therapy. Being mindful of slowing down and not necessarily rushing to judgment is another important component.¹ Of note, the residents in our case did revisit their initial working diagnosis, as suggested by Dr. Field. Questions from learners have great potential to serve as debiasing instruments and should always be encouraged. Those who do not work with students can do the same by speaking with nurses or other members of the healthcare team, who offer observations that busy physicians might miss.

Our case highlights the problem that we lack objective criteria to diagnose symptomatic heart failure. While B-type natriuretic factor (BNP) has a strong negative predictive value, serial BNP measurements have not been established to be helpful in the management of heart failure.² Although certain findings on chest radiography have strong positive and negative likelihood associations, the role of serial chest radiographs is less clear.³ Thus, heart failure remains a clinical diagnosis in current practice.

As Dr. Field points out, the accuracy and performance characteristics of diagnostic testing, such as the respiratory rate, need to be considered in conjunction with debiasing strategies to achieve higher diagnostic accuracy. Multiple factors can contribute to low-performing or misinterpreted diagnostic tests, and inaccurate vital signs have been shown to be similarly prone to potential error.⁴

Finally, we wholeheartedly agree with Dr. Field’s comment on unnecessary testing.  High-value care is appropriate care. Using Bayesian reasoning to guide testing, monitoring the treatment course appropriately, and eliminating waste is highly likely to improve both value and diagnostic accuracy. Automated, ritual ordering of daily tests can indicate that thinking has been shut off, leaving clinicians susceptible to premature closure of the diagnostic process as well as the potential for “incidentalomas” to distract them from the right diagnosis, all the while leading to low-value care such as wasteful spending, patient dissatisfaction, and hospital-acquired anemia.⁵ We believe that deciding on a daily basis what the next day’s tests will be can be another powerful debiasing habit, one with benefits beyond diagnosis.

In Reply: We thank Dr. Field for his insights and personal observations related to diagnosis and biases that contribute to diagnostic errors.

Dr. Field’s comment about the importance of revisiting one’s initial working diagnosis is consistent with our proposed diagnostic time out. A diagnostic time out can incorporate a short checklist and aid in debiasing clinicians when findings do not fit the case presentation, such as lack of response to diuretic therapy. Being mindful of slowing down and not necessarily rushing to judgment is another important component.¹ Of note, the residents in our case did revisit their initial working diagnosis, as suggested by Dr. Field. Questions from learners have great potential to serve as debiasing instruments and should always be encouraged. Those who do not work with students can do the same by speaking with nurses or other members of the healthcare team, who offer observations that busy physicians might miss.

Our case highlights the problem that we lack objective criteria to diagnose symptomatic heart failure. While B-type natriuretic factor (BNP) has a strong negative predictive value, serial BNP measurements have not been established to be helpful in the management of heart failure.² Although certain findings on chest radiography have strong positive and negative likelihood associations, the role of serial chest radiographs is less clear.³ Thus, heart failure remains a clinical diagnosis in current practice.

As Dr. Field points out, the accuracy and performance characteristics of diagnostic testing, such as the respiratory rate, need to be considered in conjunction with debiasing strategies to achieve higher diagnostic accuracy. Multiple factors can contribute to low-performing or misinterpreted diagnostic tests, and inaccurate vital signs have been shown to be similarly prone to potential error.⁴

Finally, we wholeheartedly agree with Dr. Field’s comment on unnecessary testing.  High-value care is appropriate care. Using Bayesian reasoning to guide testing, monitoring the treatment course appropriately, and eliminating waste is highly likely to improve both value and diagnostic accuracy. Automated, ritual ordering of daily tests can indicate that thinking has been shut off, leaving clinicians susceptible to premature closure of the diagnostic process as well as the potential for “incidentalomas” to distract them from the right diagnosis, all the while leading to low-value care such as wasteful spending, patient dissatisfaction, and hospital-acquired anemia.⁵ We believe that deciding on a daily basis what the next day’s tests will be can be another powerful debiasing habit, one with benefits beyond diagnosis.

An elderly Spanish-speaking woman with morbid obesity, diabetes, hypertension, and rheumatoid arthritis presents to the emergency department with worsening shortness of breath and cough. She speaks only Spanish, so her son provides the history without the aid of an interpreter.

Her shortness of breath is most noticeable with exertion and has increased gradually over the past 2 months. She has a nonproductive cough. Her son has noticed decreased oral intake and weight loss over the past few weeks.  She has neither traveled recently nor been in contact with anyone known to have an infectious disease.

A review of systems is otherwise negative: specifically, she denies chest pain, fevers, or chills. She saw her primary care physician 3 weeks ago for these complaints and was prescribed a 3-day course of azithromycin with no improvement.

Her medications include lisinopril, atenolol, glipizide, and metformin; her son believes she may be taking others as well but is not sure. He is also unsure of what treatment his mother has received for her rheumatoid arthritis, and most of her medical records are within another health system.

The patient’s son believes she may be taking other medications but is not sure; her records are at another institution

On physical examination, the patient is coughing and appears ill. Her temperature is 99.9°F (37.7°C), heart rate 105 beats per minute, blood pressure 140/70 mm Hg, res­piratory rate 24 per minute, and oxygen saturation by pulse oximetry 89% on room air. Heart sounds are normal, jugular venous pressure cannot be assessed because of her obese body habitus, pulmonary examination demonstrates crackles in all lung fields, and lower-extremity edema is not present. Her extremities are warm and well perfused. Musculoskeletal examination reveals deformities of the joints in both hands consistent with rheumatoid arthritis.

Laboratory data:

• White blood cell count 13.0 × 10⁹/L (reference range 3.7–11.0)
• Hemoglobin level 10 g/dL (11.5–15)
• Serum creatinine 1.0 mg/dL (0.7–1.4)
• Pro-brain-type natriuretic peptide (pro-BNP) level greater than the upper limit of normal.

A chest radiograph is obtained, and the resident radiologist’s preliminary impression is that it is consistent with pulmonary vascular congestion.

The patient is admitted for further diagnostic evaluation. The emergency department resident orders intravenous furosemide and signs out to the night float medicine resident that this is an “elderly woman with hypertension, diabetes, and heart failure being admitted for a heart failure exacerbation.”

What is the accuracy of a physician’s initial working diagnosis?

Diagnostic accuracy requires both clinical knowledge and problem-solving skills.¹

A decade ago, a National Patient Safety Foundation survey² found that one in six patients had suffered a medical error related to misdiagnosis. In a large systematic review of autopsy-based diagnostic errors, the theorized rate of major errors ranged from 8.4% to as high as 24.4%.³ A study by Neale et al⁴ found that admitting diagnoses were incorrect in 6% of cases. In emergency departments, inaccuracy rates of up to 12% have been described.⁵

What factors influence the prevalence of diagnostic errors?

Initial empiric treatments, such as intravenous furosemide in the above scenario, add to the challenge of diagnosis in acute care settings and can influence clinical decisions made by subsequent providers.⁶

Nonspecific or vague symptoms make diagnosis especially challenging. Shortness of breath, for example, is a common chief complaint in medical patients, as in this case. Green et al⁷ found emergency department physicians reported clinical uncertainty for a diagnosis of heart failure in 31% of patients evaluated for “dyspnea.” Pulmonary embolism and pulmonary tuberculosis are also in the differential diagnosis for our patient, with studies reporting a misdiagnosis rate of 55% for pulmonary embolism⁸ and 50% for pulmonary tuberculosis.⁹

Hertwig et al,¹⁰ describing the diagnostic process in patients presenting to emergency departments with a nonspecific constellation of symptoms, found particularly low rates of agreement between the initial diagnostic impression and the final, correct one. In fact, the actual diagnosis was only in the physician’s initial “top three” differential diagnoses 29% to 83% of the time.

Atypical presentations of common diseases, initial nonspecific presentations of common diseases, and confounding comorbid conditions have also been associated with misdiagnosis.¹¹ Our case scenario illustrates the frequent challenges physicians face when diagnosing patients who present with nonspecific symptoms and signs on a background of multiple, chronic comorbidities.

Contextual factors in the system and environment contribute to the potential for error.¹² Examples include frequent interruptions, time pressure, poor handoffs, insufficient data, and multitasking.

In our scenario, incomplete data, time constraints, and multitasking in a busy work environment compelled the emergency department resident to rapidly synthesize information to establish a working diagnosis. Interpretations of radiographs by on-call radiology residents are similarly at risk of diagnostic error for the same reasons.¹³

Physician factors also influence diagnosis. Interestingly, physician certainty or uncertainty at the time of initial diagnosis does not uniformly appear to correlate with diagnostic accuracy. A recent study showed that physician confidence remained high regardless of the degree of difficulty in a given case, and degree of confidence also correlated poorly with whether the physician’s diagnosis was accurate.¹⁴

For patients admitted with a chief complaint of dyspnea, as in our scenario, Zwaan et al¹⁵ showed that “inappropriate selectivity” in reasoning contributed to an inaccurate diagnosis 23% of the time. Inappropriate selectivity, as defined by these authors, occurs when a probable diagnosis is not sufficiently considered and therefore is neither confirmed nor ruled out.

In our patient scenario, the failure to consider diagnoses other than heart failure and the inability to confirm a prior diagnosis of heart failure in the emergency department may contribute to a diagnostic error.

CASE CONTINUED: NO IMPROVEMENT OVER 3 DAYS

The night float resident, who has six other admissions this night, cannot ask the resident who evaluated this patient in the emergency department for further information because the shift has ended. The patient’s son left at the time of admission and is not available when the patient arrives on the medical ward.

The night float resident quickly examines the patient, enters admission orders, and signs the patient out to the intern and resident who will be caring for her during her hospitalization. The verbal handoff notes that the history was limited due to a language barrier. The initial problem list includes heart failure without a differential diagnosis, but notes that an elevated pro-BNP and chest radiograph confirm heart failure as the likely diagnosis.

Several hours after the night float resident has left, the resident presents this history to the attending physician, and together they decide to order her regular at-home medications, as well as deep vein thrombosis prophylaxis and echocardiography. In writing the orders, subcutaneous heparin once daily is erroneously entered instead of low-molecular-weight heparin daily, as this is the default in the medical record system. The tired resident fails to recognize this, and the pharmacist does not question it.

Over the next 2 days, the patient’s cough and shortness of breath persist.

After the attending physician dismisses their concerns, the residents do not bring up their idea again

On hospital day 3, two junior residents on the team (who finished their internship 2 weeks ago) review the attending radiologist’s interpretation of the chest radiograph. Unflagged, it confirms the resident’s interpretation but notes ill-defined, scattered, faint opacities. The residents believe that an interstitial pattern may be present and suggest that the patient may not have heart failure but rather a primary pulmonary disease. They bring this to the attention of their attending physician, who dismisses their concerns and comments that heart failure is a clinical diagnosis. The residents do not bring this idea up again to the attending physician.

That night, the float team is called by the nursing staff because of worsening oxygenation and cough. They add an intravenous corticosteroid, a broad-spectrum antibiotic, and an inhaled bronchodilator to the patient’s drug regimen.

How do cognitive errors predispose physicians to diagnostic errors?

When errors in diagnosis are reviewed retrospectively, cognitive or “thinking” errors are generally found, especially in nonprocedural or primary care specialties such as internal medicine, pediatrics, and emergency medicine.^16,17

A widely accepted theory on how humans make decisions was described by the psychologists Tversky and Kahneman in 1974¹⁸ and has been applied more recently to physicians’ diagnostic processes.¹⁹ Their dual process model theory states that persons with a requisite level of expertise use either the intuitive “system 1” process of thinking, based on pattern-recognition and heuristics, or the slower, more analytical “system 2” process.²⁰ Experts disagree as to whether in medicine these processes represent a binary either-or model or a continuum²¹ with relative contributions of each process determined by the physician and the task.

What are some common types of cognitive error?

Experts agree that many diagnostic errors in medicine stem from decisions arrived at by inappropriate system 1 thinking due to biases. These biases have been identified and described as they relate to medicine, most notably by Croskerry.²²

Several cognitive biases are illustrated in our clinical scenario:

The framing effect occurred when the emergency department resident listed the patient’s admitting diagnosis as heart failure during the clinical handoff of care.

Anchoring bias, as defined by Croskerry,²² is the tendency to lock onto salient features of the case too early in the diagnostic process and then to fail to adjust this initial diagnostic impression. This bias affected the admitting night float resident, primary intern, resident, and attending physician.

Diagnostic momentum, in turn, is a well-described phenomenon that clinical providers are especially vulnerable to in today’s environment of “copy-and-paste” medical records and numerous handovers of care as a consequence of residency duty-hour restrictions.²³

Availability bias refers to commonly seen diagnoses like heart failure or recently seen diagnoses, which are more “available” to the human memory. These diagnoses, which spring to mind quickly, often trick providers into thinking that because they are more easily recalled, they are also more common or more likely.

Confirmation bias. The initial working diagnosis of heart failure may have led the medical team to place greater emphasis on the elevated pro-BNP and the chest radiograph to support the initial impression while ignoring findings such as weight loss that do not support this impression.

Blind obedience. Although the residents recognized the possibility of a primary pulmonary disease, they did not investigate this further. And when the attending physician dismissed their suggestion, they thus deferred to the person in authority or with a reputation of expertise.

Overconfidence bias. Despite minimal improvement in the patient’s clinical status after effective diuresis and the suggestion of alternative diagnoses by the residents, the attending physician remained confident—perhaps overconfident—in the diagnosis of heart failure and would not consider alternatives. Overconfidence bias has been well described and occurs when a medical provider believes too strongly in his or her ability to be correct and therefore fails to consider alternative diagnoses.²⁴

Despite succumbing to overconfidence bias, the attending physician was able to overcome base-rate neglect, ie, failure to consider the prevalence of potential diagnoses in diagnostic reasoning.

Each of these biases, and others not mentioned, can lead to premature closure, which is the unfortunate root cause of many diagnostic errors and delays. We have illustrated several biases in our case scenario that led several physicians on the medical team to prematurely “close” on the diagnosis of heart failure (Table 1).

CASE CONTINUED: SURPRISES AND REASSESSMENT

On hospital day 4, the patient’s medication lists from her previous hospitalizations arrive, and the team is surprised to discover that she has been receiving infliximab for the past 3 to 4 months for her rheumatoid arthritis.

Additionally, an echocardiogram that was ordered on hospital day 1 but was lost in the cardiologist’s reading queue comes in and shows a normal ejection fraction with no evidence of elevated filling pressures.

Computed tomography of the chest reveals a reticular pattern with innumerable, tiny, 1- to 2-mm pulmonary nodules. The differential diagnosis is expanded to include hypersensitivity pneumonitis, lymphoma, fungal infection, and miliary tuberculosis.

How do faulty systems contribute to diagnostic error?

It is increasingly recognized that diagnostic errors can occur as a result of cognitive error, systems-based error, or quite commonly, both. Graber et al¹⁷ analyzed 100 cases of diagnostic error and determined that while cognitive errors did occur in most of them, nearly half the time both cognitive and systems-based errors contributed simultaneously.¹⁷ Observers have further delineated the importance of the systems context and how it affects our thinking.²⁵

In this case, the language barrier, lack of availability of family, and inability to promptly utilize interpreter services contributed to early problems in acquiring a detailed history and a complete medication list that included the immunosuppressant infliximab. Later, a systems error led to a delay in the interpretation of an echocardiogram. Each of these factors, if prevented, would have presumably resulted in expansion of the differential diagnosis and earlier arrival at the correct diagnosis.

CASE CONTINUED: THE PATIENT DIES OF TUBERCULOSIS

The patient is moved to a negative pressure room, and the pulmonary consultants recommend bronchoscopy. During the procedure, the patient suffers acute respiratory failure, is intubated, and is transferred to the medical intensive care unit, where a saddle pulmonary embolism is diagnosed by computed tomographic angiography.

One day later, the sputum culture from the bronchoscopy returns as positive for acid-fast bacilli. A four-drug regimen for tuberculosis is started. The patient continues to have a downward course and expires 2 weeks later. Autopsy reveals miliary tuberculosis.

What is the frequency of diagnostic error in medicine?

Diagnostic error is estimated to have a frequency of 10% to 20%.²⁴ Rates of diagnostic error are similar irrespective of method of determination, eg, from autopsy,³ standardized patients (ie, actors presenting with scripted scenarios),²⁶ or case reviews.²⁷ Patient surveys report patient-perceived harm from diagnostic error at a rate of 35% to 42%.^28,29 The landmark Harvard Medical Practice Study found that 17% of all adverse events were attributable to diagnostic error.³⁰

Diagnostic error is the most common type of medical error in nonprocedural medical fields.³¹ It causes a disproportionately large amount of morbidity and death.

Diagnostic error is the most common cause of malpractice claims in the United States. In inpatient and outpatient settings, for both medical and surgical patients, it accounted for 45.9% of all outpatient malpractice claims in 2009, making it the most common reason for medical malpractice litigation.³² A 2013 study indicated that diagnostic error is more common, more expensive, and two times more likely to result in death than any other category of error.³³

CASE CONTINUED: MORBIDITY AND MORTALITY CONFERENCE

The patient’s case is brought to a morbidity and mortality conference for discussion. The systems issues in the case—including medication reconciliation, availability of interpreters, and timing and process of echocardiogram readings—are all discussed, but clinical reasoning and cognitive errors made in the case are avoided.

Why are cognitive errors often neglected in discussions of medical error?

Historically, openly discussing error in medicine has been difficult. Over the past decade, however, and fueled by the landmark Institute of Medicine report To Err is Human,³⁴ the healthcare community has made substantial strides in identifying and talking about systems factors as a cause of preventable medical error.^34,35

While systems contributions to medical error are inherently “external” to physicians and other healthcare providers, the cognitive contributions to error are inherently “internal” and are often considered personal. This has led to diagnostic error being kept out of many patient safety conversations. Further, while the solutions to systems errors are often tangible, such as implementing a fall prevention program or changing the physical packaging of a medication to reduce a medication dispensing or administration error, solutions to cognitive errors are generally considered more challenging to address by organizations trying to improve patient safety.

How can hospitals and department leaders do better?

Healthcare organizations and leaders of clinical teams or departments can implement several strategies.³⁶

First, they can seek out and analyze the causes of diagnostic errors that are occurring locally in their institution and learn from their diagnostic errors, such as the one in our clinical scenario.

Trainees, physicians, and nurses should be comfortable questioning each other

Second, they can promote a culture of open communication and questioning around diagnosis. Trainees, physicians, and nurses should be comfortable questioning each other, including those higher up in the hierarchy, by saying, “I’m not sure” or “What else could this be?” to help reduce cognitive bias and expand the diagnostic possibilities.

Similarly, developing strategies to promote feedback on diagnosis among physicians will allow us all to learn from our diagnostic mistakes.

Use of the electronic medical record to assist in follow-up of pending diagnostic studies and patient return visits is yet another strategy.

Finally, healthcare organizations can adopt strategies to promote patient involvement in diagnosis, such as providing patients with copies of their test results and discharge summaries, encouraging the use of electronic patient communication portals, and empowering patients to ask questions related to their diagnosis. Prioritizing potential solutions to reduce diagnostic errors may be helpful in situations, depending on the context and environment, in which all proposed interventions may not be possible.

CASE CONTINUED: LEARNING FROM MISTAKES

The attending physician and resident in the case meet after the conference to review their clinical decision-making. Both are interested in learning from this case and improving their diagnostic skills in the future.

What specific steps can clinicians take to mitigate cognitive bias in daily practice?

In addition to continuing to expand one’s medical knowledge and gain more clinical experience, we can suggest several small steps to busy clinicians, taken individually or in combination with others that may improve diagnostic skills by reducing the potential for biased thinking in clinical practice.

From Croskerry P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ 2009; 14:27–35. With kind permission from Springer Science and Business Media.

Think about your thinking. Our first recommendation would be to become more familiar with the dual process theory of clinical cognition (Figure 1).^37,38 This theoretical framework may be very helpful as a foundation from which to build better thinking skills. Physicians, especially residents, and students can be taught these concepts and their potential to contribute to diagnostic errors, and can use these skills to recognize those contributions in others’ diagnostic practices and even in their own.³⁹

Facilitating metacognition, or “thinking about one’s thinking,” may help clinicians catch themselves in thinking traps and provide the opportunity to reflect on biases retrospectively, as a double check or an opportunity to learn from a mistake.

Recognize your emotions. Gaining an understanding of the effect of one’s emotions on decision-making also can help clinicians free themselves of bias. As human beings, healthcare professionals are  susceptible to emotion, and the best approach to mitigate the emotional influences may be to consciously name them and adjust for them.⁴⁰

Because it is impractical to apply slow, analytical system 2 approaches to every case, skills that hone and develop more accurate, reliable system 1 thinking are crucial. Gaining broad exposure to increased numbers of cases may be the most reliable way to build an experiential repertoire of “illness scripts,” but there are ways to increase the experiential value of any case with a few techniques that have potential to promote better intuition.⁴¹

Embracing uncertainty in the early diagnostic process and envisioning the worst-case scenario in a case allows the consideration of additional diagnostic paths outside of the current working diagnosis, potentially priming the clinician to look for and recognize early warning signs that could argue against the initial diagnosis at a time when an adjustment could be made to prevent a bad outcome.

Practice progressive problem-solving,⁴² a technique in which the physician creates additional challenges to increase the cognitive burden of a “routine” case in an effort to train his or her mind and sharpen intuition. An example of this practice is contemplating a backup treatment plan in advance in the event of a poor response to or an adverse effect of treatment. Highly rated physicians and teachers perform this regularly.^43,44 Other ways to maximize the learning value of an individual case include seeking feedback on patient outcomes, especially when a patient has been discharged or transferred to another provider’s care, or when the physician goes off service.

Simulation, traditionally used for procedural training, has potential as well. Cognitive simulation, such as case reports or virtual patient modules, have potential to enhance clinical reasoning skills as well, though possibly at greater cost of time and expense.

Decreased reliance on memory is likely to improve diagnostic reasoning. Systems tools such as checklists⁴⁵ and health information technology⁴⁶ have potential to reduce diagnostic errors, not by taking thinking away from the clinician but by relieving the cognitive load enough to facilitate greater effort toward reasoning.

Slow down. Finally, and perhaps most important, recent models of clinical expertise have suggested that mastery comes from having a robust intuitive method, with a sense of the limitations of the intuitive approach, an ability to recognize the need to perform more analytical reasoning in select cases, and the willingness to do so. In short, it may well be that the hallmark of a master clinician is the propensity to slow down when necessary.⁴⁷

A ‘diagnostic time-out’ for safety might catch opportunities to recognize and mitigate biases and errors

If one considers diagnosis a cognitive procedure, perhaps a brief “diagnostic time-out” for safety might afford an opportunity to recognize and mitigate biases and errors. There are likely many potential scripts for a good diagnostic time-out, but to be functional it should be brief and simple to facilitate consistent use. We have recommended the following four questions to our residents as a starting point, any of which could signal the need to switch to a slower, analytic approach.

Four-step diagnostic time-out

• What else can it be?
• Is there anything about the case that does not fit?
• Is it possible that multiple processes are going on?
• Do I need to slow down?

These questions can serve as a double check for an intuitively formed initial working diagnosis, incorporating many of the principles discussed above, in a way that would hopefully avoid undue burden on a busy clinician. These techniques, it must be acknowledged, have not yet been directly tied to reductions in diagnostic errors. However, diagnostic errors, as discussed, are very difficult to identify and study, and these techniques will serve mainly to improve habits that are likely to show benefits over much longer time periods than most studies can measure.

An elderly Spanish-speaking woman with morbid obesity, diabetes, hypertension, and rheumatoid arthritis presents to the emergency department with worsening shortness of breath and cough. She speaks only Spanish, so her son provides the history without the aid of an interpreter.

Her shortness of breath is most noticeable with exertion and has increased gradually over the past 2 months. She has a nonproductive cough. Her son has noticed decreased oral intake and weight loss over the past few weeks.  She has neither traveled recently nor been in contact with anyone known to have an infectious disease.

A review of systems is otherwise negative: specifically, she denies chest pain, fevers, or chills. She saw her primary care physician 3 weeks ago for these complaints and was prescribed a 3-day course of azithromycin with no improvement.

Her medications include lisinopril, atenolol, glipizide, and metformin; her son believes she may be taking others as well but is not sure. He is also unsure of what treatment his mother has received for her rheumatoid arthritis, and most of her medical records are within another health system.

The patient’s son believes she may be taking other medications but is not sure; her records are at another institution

On physical examination, the patient is coughing and appears ill. Her temperature is 99.9°F (37.7°C), heart rate 105 beats per minute, blood pressure 140/70 mm Hg, res­piratory rate 24 per minute, and oxygen saturation by pulse oximetry 89% on room air. Heart sounds are normal, jugular venous pressure cannot be assessed because of her obese body habitus, pulmonary examination demonstrates crackles in all lung fields, and lower-extremity edema is not present. Her extremities are warm and well perfused. Musculoskeletal examination reveals deformities of the joints in both hands consistent with rheumatoid arthritis.

Laboratory data:

• White blood cell count 13.0 × 10⁹/L (reference range 3.7–11.0)
• Hemoglobin level 10 g/dL (11.5–15)
• Serum creatinine 1.0 mg/dL (0.7–1.4)
• Pro-brain-type natriuretic peptide (pro-BNP) level greater than the upper limit of normal.

A chest radiograph is obtained, and the resident radiologist’s preliminary impression is that it is consistent with pulmonary vascular congestion.

The patient is admitted for further diagnostic evaluation. The emergency department resident orders intravenous furosemide and signs out to the night float medicine resident that this is an “elderly woman with hypertension, diabetes, and heart failure being admitted for a heart failure exacerbation.”

What is the accuracy of a physician’s initial working diagnosis?

Diagnostic accuracy requires both clinical knowledge and problem-solving skills.¹

A decade ago, a National Patient Safety Foundation survey² found that one in six patients had suffered a medical error related to misdiagnosis. In a large systematic review of autopsy-based diagnostic errors, the theorized rate of major errors ranged from 8.4% to as high as 24.4%.³ A study by Neale et al⁴ found that admitting diagnoses were incorrect in 6% of cases. In emergency departments, inaccuracy rates of up to 12% have been described.⁵

What factors influence the prevalence of diagnostic errors?

Initial empiric treatments, such as intravenous furosemide in the above scenario, add to the challenge of diagnosis in acute care settings and can influence clinical decisions made by subsequent providers.⁶

Nonspecific or vague symptoms make diagnosis especially challenging. Shortness of breath, for example, is a common chief complaint in medical patients, as in this case. Green et al⁷ found emergency department physicians reported clinical uncertainty for a diagnosis of heart failure in 31% of patients evaluated for “dyspnea.” Pulmonary embolism and pulmonary tuberculosis are also in the differential diagnosis for our patient, with studies reporting a misdiagnosis rate of 55% for pulmonary embolism⁸ and 50% for pulmonary tuberculosis.⁹

Hertwig et al,¹⁰ describing the diagnostic process in patients presenting to emergency departments with a nonspecific constellation of symptoms, found particularly low rates of agreement between the initial diagnostic impression and the final, correct one. In fact, the actual diagnosis was only in the physician’s initial “top three” differential diagnoses 29% to 83% of the time.

Atypical presentations of common diseases, initial nonspecific presentations of common diseases, and confounding comorbid conditions have also been associated with misdiagnosis.¹¹ Our case scenario illustrates the frequent challenges physicians face when diagnosing patients who present with nonspecific symptoms and signs on a background of multiple, chronic comorbidities.

Contextual factors in the system and environment contribute to the potential for error.¹² Examples include frequent interruptions, time pressure, poor handoffs, insufficient data, and multitasking.

In our scenario, incomplete data, time constraints, and multitasking in a busy work environment compelled the emergency department resident to rapidly synthesize information to establish a working diagnosis. Interpretations of radiographs by on-call radiology residents are similarly at risk of diagnostic error for the same reasons.¹³

Physician factors also influence diagnosis. Interestingly, physician certainty or uncertainty at the time of initial diagnosis does not uniformly appear to correlate with diagnostic accuracy. A recent study showed that physician confidence remained high regardless of the degree of difficulty in a given case, and degree of confidence also correlated poorly with whether the physician’s diagnosis was accurate.¹⁴

For patients admitted with a chief complaint of dyspnea, as in our scenario, Zwaan et al¹⁵ showed that “inappropriate selectivity” in reasoning contributed to an inaccurate diagnosis 23% of the time. Inappropriate selectivity, as defined by these authors, occurs when a probable diagnosis is not sufficiently considered and therefore is neither confirmed nor ruled out.

In our patient scenario, the failure to consider diagnoses other than heart failure and the inability to confirm a prior diagnosis of heart failure in the emergency department may contribute to a diagnostic error.

CASE CONTINUED: NO IMPROVEMENT OVER 3 DAYS

The night float resident, who has six other admissions this night, cannot ask the resident who evaluated this patient in the emergency department for further information because the shift has ended. The patient’s son left at the time of admission and is not available when the patient arrives on the medical ward.

The night float resident quickly examines the patient, enters admission orders, and signs the patient out to the intern and resident who will be caring for her during her hospitalization. The verbal handoff notes that the history was limited due to a language barrier. The initial problem list includes heart failure without a differential diagnosis, but notes that an elevated pro-BNP and chest radiograph confirm heart failure as the likely diagnosis.

Several hours after the night float resident has left, the resident presents this history to the attending physician, and together they decide to order her regular at-home medications, as well as deep vein thrombosis prophylaxis and echocardiography. In writing the orders, subcutaneous heparin once daily is erroneously entered instead of low-molecular-weight heparin daily, as this is the default in the medical record system. The tired resident fails to recognize this, and the pharmacist does not question it.

Over the next 2 days, the patient’s cough and shortness of breath persist.

After the attending physician dismisses their concerns, the residents do not bring up their idea again

On hospital day 3, two junior residents on the team (who finished their internship 2 weeks ago) review the attending radiologist’s interpretation of the chest radiograph. Unflagged, it confirms the resident’s interpretation but notes ill-defined, scattered, faint opacities. The residents believe that an interstitial pattern may be present and suggest that the patient may not have heart failure but rather a primary pulmonary disease. They bring this to the attention of their attending physician, who dismisses their concerns and comments that heart failure is a clinical diagnosis. The residents do not bring this idea up again to the attending physician.

That night, the float team is called by the nursing staff because of worsening oxygenation and cough. They add an intravenous corticosteroid, a broad-spectrum antibiotic, and an inhaled bronchodilator to the patient’s drug regimen.

How do cognitive errors predispose physicians to diagnostic errors?

When errors in diagnosis are reviewed retrospectively, cognitive or “thinking” errors are generally found, especially in nonprocedural or primary care specialties such as internal medicine, pediatrics, and emergency medicine.^16,17

A widely accepted theory on how humans make decisions was described by the psychologists Tversky and Kahneman in 1974¹⁸ and has been applied more recently to physicians’ diagnostic processes.¹⁹ Their dual process model theory states that persons with a requisite level of expertise use either the intuitive “system 1” process of thinking, based on pattern-recognition and heuristics, or the slower, more analytical “system 2” process.²⁰ Experts disagree as to whether in medicine these processes represent a binary either-or model or a continuum²¹ with relative contributions of each process determined by the physician and the task.

What are some common types of cognitive error?

Experts agree that many diagnostic errors in medicine stem from decisions arrived at by inappropriate system 1 thinking due to biases. These biases have been identified and described as they relate to medicine, most notably by Croskerry.²²

Several cognitive biases are illustrated in our clinical scenario:

The framing effect occurred when the emergency department resident listed the patient’s admitting diagnosis as heart failure during the clinical handoff of care.

Anchoring bias, as defined by Croskerry,²² is the tendency to lock onto salient features of the case too early in the diagnostic process and then to fail to adjust this initial diagnostic impression. This bias affected the admitting night float resident, primary intern, resident, and attending physician.

Diagnostic momentum, in turn, is a well-described phenomenon that clinical providers are especially vulnerable to in today’s environment of “copy-and-paste” medical records and numerous handovers of care as a consequence of residency duty-hour restrictions.²³

Availability bias refers to commonly seen diagnoses like heart failure or recently seen diagnoses, which are more “available” to the human memory. These diagnoses, which spring to mind quickly, often trick providers into thinking that because they are more easily recalled, they are also more common or more likely.

Confirmation bias. The initial working diagnosis of heart failure may have led the medical team to place greater emphasis on the elevated pro-BNP and the chest radiograph to support the initial impression while ignoring findings such as weight loss that do not support this impression.

Blind obedience. Although the residents recognized the possibility of a primary pulmonary disease, they did not investigate this further. And when the attending physician dismissed their suggestion, they thus deferred to the person in authority or with a reputation of expertise.

Overconfidence bias. Despite minimal improvement in the patient’s clinical status after effective diuresis and the suggestion of alternative diagnoses by the residents, the attending physician remained confident—perhaps overconfident—in the diagnosis of heart failure and would not consider alternatives. Overconfidence bias has been well described and occurs when a medical provider believes too strongly in his or her ability to be correct and therefore fails to consider alternative diagnoses.²⁴

Despite succumbing to overconfidence bias, the attending physician was able to overcome base-rate neglect, ie, failure to consider the prevalence of potential diagnoses in diagnostic reasoning.

Each of these biases, and others not mentioned, can lead to premature closure, which is the unfortunate root cause of many diagnostic errors and delays. We have illustrated several biases in our case scenario that led several physicians on the medical team to prematurely “close” on the diagnosis of heart failure (Table 1).

CASE CONTINUED: SURPRISES AND REASSESSMENT

On hospital day 4, the patient’s medication lists from her previous hospitalizations arrive, and the team is surprised to discover that she has been receiving infliximab for the past 3 to 4 months for her rheumatoid arthritis.

Additionally, an echocardiogram that was ordered on hospital day 1 but was lost in the cardiologist’s reading queue comes in and shows a normal ejection fraction with no evidence of elevated filling pressures.

Computed tomography of the chest reveals a reticular pattern with innumerable, tiny, 1- to 2-mm pulmonary nodules. The differential diagnosis is expanded to include hypersensitivity pneumonitis, lymphoma, fungal infection, and miliary tuberculosis.

How do faulty systems contribute to diagnostic error?

It is increasingly recognized that diagnostic errors can occur as a result of cognitive error, systems-based error, or quite commonly, both. Graber et al¹⁷ analyzed 100 cases of diagnostic error and determined that while cognitive errors did occur in most of them, nearly half the time both cognitive and systems-based errors contributed simultaneously.¹⁷ Observers have further delineated the importance of the systems context and how it affects our thinking.²⁵

In this case, the language barrier, lack of availability of family, and inability to promptly utilize interpreter services contributed to early problems in acquiring a detailed history and a complete medication list that included the immunosuppressant infliximab. Later, a systems error led to a delay in the interpretation of an echocardiogram. Each of these factors, if prevented, would have presumably resulted in expansion of the differential diagnosis and earlier arrival at the correct diagnosis.

CASE CONTINUED: THE PATIENT DIES OF TUBERCULOSIS

The patient is moved to a negative pressure room, and the pulmonary consultants recommend bronchoscopy. During the procedure, the patient suffers acute respiratory failure, is intubated, and is transferred to the medical intensive care unit, where a saddle pulmonary embolism is diagnosed by computed tomographic angiography.

One day later, the sputum culture from the bronchoscopy returns as positive for acid-fast bacilli. A four-drug regimen for tuberculosis is started. The patient continues to have a downward course and expires 2 weeks later. Autopsy reveals miliary tuberculosis.

What is the frequency of diagnostic error in medicine?

Diagnostic error is estimated to have a frequency of 10% to 20%.²⁴ Rates of diagnostic error are similar irrespective of method of determination, eg, from autopsy,³ standardized patients (ie, actors presenting with scripted scenarios),²⁶ or case reviews.²⁷ Patient surveys report patient-perceived harm from diagnostic error at a rate of 35% to 42%.^28,29 The landmark Harvard Medical Practice Study found that 17% of all adverse events were attributable to diagnostic error.³⁰

Diagnostic error is the most common type of medical error in nonprocedural medical fields.³¹ It causes a disproportionately large amount of morbidity and death.

Diagnostic error is the most common cause of malpractice claims in the United States. In inpatient and outpatient settings, for both medical and surgical patients, it accounted for 45.9% of all outpatient malpractice claims in 2009, making it the most common reason for medical malpractice litigation.³² A 2013 study indicated that diagnostic error is more common, more expensive, and two times more likely to result in death than any other category of error.³³

CASE CONTINUED: MORBIDITY AND MORTALITY CONFERENCE

The patient’s case is brought to a morbidity and mortality conference for discussion. The systems issues in the case—including medication reconciliation, availability of interpreters, and timing and process of echocardiogram readings—are all discussed, but clinical reasoning and cognitive errors made in the case are avoided.

Why are cognitive errors often neglected in discussions of medical error?

Historically, openly discussing error in medicine has been difficult. Over the past decade, however, and fueled by the landmark Institute of Medicine report To Err is Human,³⁴ the healthcare community has made substantial strides in identifying and talking about systems factors as a cause of preventable medical error.^34,35

While systems contributions to medical error are inherently “external” to physicians and other healthcare providers, the cognitive contributions to error are inherently “internal” and are often considered personal. This has led to diagnostic error being kept out of many patient safety conversations. Further, while the solutions to systems errors are often tangible, such as implementing a fall prevention program or changing the physical packaging of a medication to reduce a medication dispensing or administration error, solutions to cognitive errors are generally considered more challenging to address by organizations trying to improve patient safety.

How can hospitals and department leaders do better?

Healthcare organizations and leaders of clinical teams or departments can implement several strategies.³⁶

First, they can seek out and analyze the causes of diagnostic errors that are occurring locally in their institution and learn from their diagnostic errors, such as the one in our clinical scenario.

Trainees, physicians, and nurses should be comfortable questioning each other

Second, they can promote a culture of open communication and questioning around diagnosis. Trainees, physicians, and nurses should be comfortable questioning each other, including those higher up in the hierarchy, by saying, “I’m not sure” or “What else could this be?” to help reduce cognitive bias and expand the diagnostic possibilities.

Similarly, developing strategies to promote feedback on diagnosis among physicians will allow us all to learn from our diagnostic mistakes.

Use of the electronic medical record to assist in follow-up of pending diagnostic studies and patient return visits is yet another strategy.

Finally, healthcare organizations can adopt strategies to promote patient involvement in diagnosis, such as providing patients with copies of their test results and discharge summaries, encouraging the use of electronic patient communication portals, and empowering patients to ask questions related to their diagnosis. Prioritizing potential solutions to reduce diagnostic errors may be helpful in situations, depending on the context and environment, in which all proposed interventions may not be possible.

CASE CONTINUED: LEARNING FROM MISTAKES

The attending physician and resident in the case meet after the conference to review their clinical decision-making. Both are interested in learning from this case and improving their diagnostic skills in the future.

What specific steps can clinicians take to mitigate cognitive bias in daily practice?

In addition to continuing to expand one’s medical knowledge and gain more clinical experience, we can suggest several small steps to busy clinicians, taken individually or in combination with others that may improve diagnostic skills by reducing the potential for biased thinking in clinical practice.

From Croskerry P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ 2009; 14:27–35. With kind permission from Springer Science and Business Media.

Think about your thinking. Our first recommendation would be to become more familiar with the dual process theory of clinical cognition (Figure 1).^37,38 This theoretical framework may be very helpful as a foundation from which to build better thinking skills. Physicians, especially residents, and students can be taught these concepts and their potential to contribute to diagnostic errors, and can use these skills to recognize those contributions in others’ diagnostic practices and even in their own.³⁹

Facilitating metacognition, or “thinking about one’s thinking,” may help clinicians catch themselves in thinking traps and provide the opportunity to reflect on biases retrospectively, as a double check or an opportunity to learn from a mistake.

Recognize your emotions. Gaining an understanding of the effect of one’s emotions on decision-making also can help clinicians free themselves of bias. As human beings, healthcare professionals are  susceptible to emotion, and the best approach to mitigate the emotional influences may be to consciously name them and adjust for them.⁴⁰

Because it is impractical to apply slow, analytical system 2 approaches to every case, skills that hone and develop more accurate, reliable system 1 thinking are crucial. Gaining broad exposure to increased numbers of cases may be the most reliable way to build an experiential repertoire of “illness scripts,” but there are ways to increase the experiential value of any case with a few techniques that have potential to promote better intuition.⁴¹

Embracing uncertainty in the early diagnostic process and envisioning the worst-case scenario in a case allows the consideration of additional diagnostic paths outside of the current working diagnosis, potentially priming the clinician to look for and recognize early warning signs that could argue against the initial diagnosis at a time when an adjustment could be made to prevent a bad outcome.

Practice progressive problem-solving,⁴² a technique in which the physician creates additional challenges to increase the cognitive burden of a “routine” case in an effort to train his or her mind and sharpen intuition. An example of this practice is contemplating a backup treatment plan in advance in the event of a poor response to or an adverse effect of treatment. Highly rated physicians and teachers perform this regularly.^43,44 Other ways to maximize the learning value of an individual case include seeking feedback on patient outcomes, especially when a patient has been discharged or transferred to another provider’s care, or when the physician goes off service.

Simulation, traditionally used for procedural training, has potential as well. Cognitive simulation, such as case reports or virtual patient modules, have potential to enhance clinical reasoning skills as well, though possibly at greater cost of time and expense.

Decreased reliance on memory is likely to improve diagnostic reasoning. Systems tools such as checklists⁴⁵ and health information technology⁴⁶ have potential to reduce diagnostic errors, not by taking thinking away from the clinician but by relieving the cognitive load enough to facilitate greater effort toward reasoning.

Slow down. Finally, and perhaps most important, recent models of clinical expertise have suggested that mastery comes from having a robust intuitive method, with a sense of the limitations of the intuitive approach, an ability to recognize the need to perform more analytical reasoning in select cases, and the willingness to do so. In short, it may well be that the hallmark of a master clinician is the propensity to slow down when necessary.⁴⁷

A ‘diagnostic time-out’ for safety might catch opportunities to recognize and mitigate biases and errors

If one considers diagnosis a cognitive procedure, perhaps a brief “diagnostic time-out” for safety might afford an opportunity to recognize and mitigate biases and errors. There are likely many potential scripts for a good diagnostic time-out, but to be functional it should be brief and simple to facilitate consistent use. We have recommended the following four questions to our residents as a starting point, any of which could signal the need to switch to a slower, analytic approach.

Four-step diagnostic time-out

• What else can it be?
• Is there anything about the case that does not fit?
• Is it possible that multiple processes are going on?
• Do I need to slow down?

These questions can serve as a double check for an intuitively formed initial working diagnosis, incorporating many of the principles discussed above, in a way that would hopefully avoid undue burden on a busy clinician. These techniques, it must be acknowledged, have not yet been directly tied to reductions in diagnostic errors. However, diagnostic errors, as discussed, are very difficult to identify and study, and these techniques will serve mainly to improve habits that are likely to show benefits over much longer time periods than most studies can measure.

An elderly Spanish-speaking woman with morbid obesity, diabetes, hypertension, and rheumatoid arthritis presents to the emergency department with worsening shortness of breath and cough. She speaks only Spanish, so her son provides the history without the aid of an interpreter.

Her shortness of breath is most noticeable with exertion and has increased gradually over the past 2 months. She has a nonproductive cough. Her son has noticed decreased oral intake and weight loss over the past few weeks.  She has neither traveled recently nor been in contact with anyone known to have an infectious disease.

A review of systems is otherwise negative: specifically, she denies chest pain, fevers, or chills. She saw her primary care physician 3 weeks ago for these complaints and was prescribed a 3-day course of azithromycin with no improvement.

Her medications include lisinopril, atenolol, glipizide, and metformin; her son believes she may be taking others as well but is not sure. He is also unsure of what treatment his mother has received for her rheumatoid arthritis, and most of her medical records are within another health system.

The patient’s son believes she may be taking other medications but is not sure; her records are at another institution

On physical examination, the patient is coughing and appears ill. Her temperature is 99.9°F (37.7°C), heart rate 105 beats per minute, blood pressure 140/70 mm Hg, res­piratory rate 24 per minute, and oxygen saturation by pulse oximetry 89% on room air. Heart sounds are normal, jugular venous pressure cannot be assessed because of her obese body habitus, pulmonary examination demonstrates crackles in all lung fields, and lower-extremity edema is not present. Her extremities are warm and well perfused. Musculoskeletal examination reveals deformities of the joints in both hands consistent with rheumatoid arthritis.

Laboratory data:

• White blood cell count 13.0 × 10⁹/L (reference range 3.7–11.0)
• Hemoglobin level 10 g/dL (11.5–15)
• Serum creatinine 1.0 mg/dL (0.7–1.4)
• Pro-brain-type natriuretic peptide (pro-BNP) level greater than the upper limit of normal.

A chest radiograph is obtained, and the resident radiologist’s preliminary impression is that it is consistent with pulmonary vascular congestion.

The patient is admitted for further diagnostic evaluation. The emergency department resident orders intravenous furosemide and signs out to the night float medicine resident that this is an “elderly woman with hypertension, diabetes, and heart failure being admitted for a heart failure exacerbation.”

What is the accuracy of a physician’s initial working diagnosis?

Diagnostic accuracy requires both clinical knowledge and problem-solving skills.¹

A decade ago, a National Patient Safety Foundation survey² found that one in six patients had suffered a medical error related to misdiagnosis. In a large systematic review of autopsy-based diagnostic errors, the theorized rate of major errors ranged from 8.4% to as high as 24.4%.³ A study by Neale et al⁴ found that admitting diagnoses were incorrect in 6% of cases. In emergency departments, inaccuracy rates of up to 12% have been described.⁵

What factors influence the prevalence of diagnostic errors?

Initial empiric treatments, such as intravenous furosemide in the above scenario, add to the challenge of diagnosis in acute care settings and can influence clinical decisions made by subsequent providers.⁶

Nonspecific or vague symptoms make diagnosis especially challenging. Shortness of breath, for example, is a common chief complaint in medical patients, as in this case. Green et al⁷ found emergency department physicians reported clinical uncertainty for a diagnosis of heart failure in 31% of patients evaluated for “dyspnea.” Pulmonary embolism and pulmonary tuberculosis are also in the differential diagnosis for our patient, with studies reporting a misdiagnosis rate of 55% for pulmonary embolism⁸ and 50% for pulmonary tuberculosis.⁹

Hertwig et al,¹⁰ describing the diagnostic process in patients presenting to emergency departments with a nonspecific constellation of symptoms, found particularly low rates of agreement between the initial diagnostic impression and the final, correct one. In fact, the actual diagnosis was only in the physician’s initial “top three” differential diagnoses 29% to 83% of the time.

Atypical presentations of common diseases, initial nonspecific presentations of common diseases, and confounding comorbid conditions have also been associated with misdiagnosis.¹¹ Our case scenario illustrates the frequent challenges physicians face when diagnosing patients who present with nonspecific symptoms and signs on a background of multiple, chronic comorbidities.

Contextual factors in the system and environment contribute to the potential for error.¹² Examples include frequent interruptions, time pressure, poor handoffs, insufficient data, and multitasking.

In our scenario, incomplete data, time constraints, and multitasking in a busy work environment compelled the emergency department resident to rapidly synthesize information to establish a working diagnosis. Interpretations of radiographs by on-call radiology residents are similarly at risk of diagnostic error for the same reasons.¹³

Physician factors also influence diagnosis. Interestingly, physician certainty or uncertainty at the time of initial diagnosis does not uniformly appear to correlate with diagnostic accuracy. A recent study showed that physician confidence remained high regardless of the degree of difficulty in a given case, and degree of confidence also correlated poorly with whether the physician’s diagnosis was accurate.¹⁴

For patients admitted with a chief complaint of dyspnea, as in our scenario, Zwaan et al¹⁵ showed that “inappropriate selectivity” in reasoning contributed to an inaccurate diagnosis 23% of the time. Inappropriate selectivity, as defined by these authors, occurs when a probable diagnosis is not sufficiently considered and therefore is neither confirmed nor ruled out.

In our patient scenario, the failure to consider diagnoses other than heart failure and the inability to confirm a prior diagnosis of heart failure in the emergency department may contribute to a diagnostic error.

CASE CONTINUED: NO IMPROVEMENT OVER 3 DAYS

The night float resident, who has six other admissions this night, cannot ask the resident who evaluated this patient in the emergency department for further information because the shift has ended. The patient’s son left at the time of admission and is not available when the patient arrives on the medical ward.

The night float resident quickly examines the patient, enters admission orders, and signs the patient out to the intern and resident who will be caring for her during her hospitalization. The verbal handoff notes that the history was limited due to a language barrier. The initial problem list includes heart failure without a differential diagnosis, but notes that an elevated pro-BNP and chest radiograph confirm heart failure as the likely diagnosis.

Several hours after the night float resident has left, the resident presents this history to the attending physician, and together they decide to order her regular at-home medications, as well as deep vein thrombosis prophylaxis and echocardiography. In writing the orders, subcutaneous heparin once daily is erroneously entered instead of low-molecular-weight heparin daily, as this is the default in the medical record system. The tired resident fails to recognize this, and the pharmacist does not question it.

Over the next 2 days, the patient’s cough and shortness of breath persist.

After the attending physician dismisses their concerns, the residents do not bring up their idea again

On hospital day 3, two junior residents on the team (who finished their internship 2 weeks ago) review the attending radiologist’s interpretation of the chest radiograph. Unflagged, it confirms the resident’s interpretation but notes ill-defined, scattered, faint opacities. The residents believe that an interstitial pattern may be present and suggest that the patient may not have heart failure but rather a primary pulmonary disease. They bring this to the attention of their attending physician, who dismisses their concerns and comments that heart failure is a clinical diagnosis. The residents do not bring this idea up again to the attending physician.

That night, the float team is called by the nursing staff because of worsening oxygenation and cough. They add an intravenous corticosteroid, a broad-spectrum antibiotic, and an inhaled bronchodilator to the patient’s drug regimen.

How do cognitive errors predispose physicians to diagnostic errors?

When errors in diagnosis are reviewed retrospectively, cognitive or “thinking” errors are generally found, especially in nonprocedural or primary care specialties such as internal medicine, pediatrics, and emergency medicine.^16,17

A widely accepted theory on how humans make decisions was described by the psychologists Tversky and Kahneman in 1974¹⁸ and has been applied more recently to physicians’ diagnostic processes.¹⁹ Their dual process model theory states that persons with a requisite level of expertise use either the intuitive “system 1” process of thinking, based on pattern-recognition and heuristics, or the slower, more analytical “system 2” process.²⁰ Experts disagree as to whether in medicine these processes represent a binary either-or model or a continuum²¹ with relative contributions of each process determined by the physician and the task.

What are some common types of cognitive error?

Experts agree that many diagnostic errors in medicine stem from decisions arrived at by inappropriate system 1 thinking due to biases. These biases have been identified and described as they relate to medicine, most notably by Croskerry.²²

Several cognitive biases are illustrated in our clinical scenario:

The framing effect occurred when the emergency department resident listed the patient’s admitting diagnosis as heart failure during the clinical handoff of care.

Anchoring bias, as defined by Croskerry,²² is the tendency to lock onto salient features of the case too early in the diagnostic process and then to fail to adjust this initial diagnostic impression. This bias affected the admitting night float resident, primary intern, resident, and attending physician.

Diagnostic momentum, in turn, is a well-described phenomenon that clinical providers are especially vulnerable to in today’s environment of “copy-and-paste” medical records and numerous handovers of care as a consequence of residency duty-hour restrictions.²³

Availability bias refers to commonly seen diagnoses like heart failure or recently seen diagnoses, which are more “available” to the human memory. These diagnoses, which spring to mind quickly, often trick providers into thinking that because they are more easily recalled, they are also more common or more likely.

Confirmation bias. The initial working diagnosis of heart failure may have led the medical team to place greater emphasis on the elevated pro-BNP and the chest radiograph to support the initial impression while ignoring findings such as weight loss that do not support this impression.

Blind obedience. Although the residents recognized the possibility of a primary pulmonary disease, they did not investigate this further. And when the attending physician dismissed their suggestion, they thus deferred to the person in authority or with a reputation of expertise.

Overconfidence bias. Despite minimal improvement in the patient’s clinical status after effective diuresis and the suggestion of alternative diagnoses by the residents, the attending physician remained confident—perhaps overconfident—in the diagnosis of heart failure and would not consider alternatives. Overconfidence bias has been well described and occurs when a medical provider believes too strongly in his or her ability to be correct and therefore fails to consider alternative diagnoses.²⁴

Despite succumbing to overconfidence bias, the attending physician was able to overcome base-rate neglect, ie, failure to consider the prevalence of potential diagnoses in diagnostic reasoning.

Each of these biases, and others not mentioned, can lead to premature closure, which is the unfortunate root cause of many diagnostic errors and delays. We have illustrated several biases in our case scenario that led several physicians on the medical team to prematurely “close” on the diagnosis of heart failure (Table 1).

CASE CONTINUED: SURPRISES AND REASSESSMENT

On hospital day 4, the patient’s medication lists from her previous hospitalizations arrive, and the team is surprised to discover that she has been receiving infliximab for the past 3 to 4 months for her rheumatoid arthritis.

Additionally, an echocardiogram that was ordered on hospital day 1 but was lost in the cardiologist’s reading queue comes in and shows a normal ejection fraction with no evidence of elevated filling pressures.

Computed tomography of the chest reveals a reticular pattern with innumerable, tiny, 1- to 2-mm pulmonary nodules. The differential diagnosis is expanded to include hypersensitivity pneumonitis, lymphoma, fungal infection, and miliary tuberculosis.

How do faulty systems contribute to diagnostic error?

It is increasingly recognized that diagnostic errors can occur as a result of cognitive error, systems-based error, or quite commonly, both. Graber et al¹⁷ analyzed 100 cases of diagnostic error and determined that while cognitive errors did occur in most of them, nearly half the time both cognitive and systems-based errors contributed simultaneously.¹⁷ Observers have further delineated the importance of the systems context and how it affects our thinking.²⁵

In this case, the language barrier, lack of availability of family, and inability to promptly utilize interpreter services contributed to early problems in acquiring a detailed history and a complete medication list that included the immunosuppressant infliximab. Later, a systems error led to a delay in the interpretation of an echocardiogram. Each of these factors, if prevented, would have presumably resulted in expansion of the differential diagnosis and earlier arrival at the correct diagnosis.

CASE CONTINUED: THE PATIENT DIES OF TUBERCULOSIS

The patient is moved to a negative pressure room, and the pulmonary consultants recommend bronchoscopy. During the procedure, the patient suffers acute respiratory failure, is intubated, and is transferred to the medical intensive care unit, where a saddle pulmonary embolism is diagnosed by computed tomographic angiography.

One day later, the sputum culture from the bronchoscopy returns as positive for acid-fast bacilli. A four-drug regimen for tuberculosis is started. The patient continues to have a downward course and expires 2 weeks later. Autopsy reveals miliary tuberculosis.

What is the frequency of diagnostic error in medicine?

Diagnostic error is estimated to have a frequency of 10% to 20%.²⁴ Rates of diagnostic error are similar irrespective of method of determination, eg, from autopsy,³ standardized patients (ie, actors presenting with scripted scenarios),²⁶ or case reviews.²⁷ Patient surveys report patient-perceived harm from diagnostic error at a rate of 35% to 42%.^28,29 The landmark Harvard Medical Practice Study found that 17% of all adverse events were attributable to diagnostic error.³⁰

Diagnostic error is the most common type of medical error in nonprocedural medical fields.³¹ It causes a disproportionately large amount of morbidity and death.

Diagnostic error is the most common cause of malpractice claims in the United States. In inpatient and outpatient settings, for both medical and surgical patients, it accounted for 45.9% of all outpatient malpractice claims in 2009, making it the most common reason for medical malpractice litigation.³² A 2013 study indicated that diagnostic error is more common, more expensive, and two times more likely to result in death than any other category of error.³³

CASE CONTINUED: MORBIDITY AND MORTALITY CONFERENCE

The patient’s case is brought to a morbidity and mortality conference for discussion. The systems issues in the case—including medication reconciliation, availability of interpreters, and timing and process of echocardiogram readings—are all discussed, but clinical reasoning and cognitive errors made in the case are avoided.

Why are cognitive errors often neglected in discussions of medical error?

Historically, openly discussing error in medicine has been difficult. Over the past decade, however, and fueled by the landmark Institute of Medicine report To Err is Human,³⁴ the healthcare community has made substantial strides in identifying and talking about systems factors as a cause of preventable medical error.^34,35

While systems contributions to medical error are inherently “external” to physicians and other healthcare providers, the cognitive contributions to error are inherently “internal” and are often considered personal. This has led to diagnostic error being kept out of many patient safety conversations. Further, while the solutions to systems errors are often tangible, such as implementing a fall prevention program or changing the physical packaging of a medication to reduce a medication dispensing or administration error, solutions to cognitive errors are generally considered more challenging to address by organizations trying to improve patient safety.

How can hospitals and department leaders do better?

Healthcare organizations and leaders of clinical teams or departments can implement several strategies.³⁶

First, they can seek out and analyze the causes of diagnostic errors that are occurring locally in their institution and learn from their diagnostic errors, such as the one in our clinical scenario.

Trainees, physicians, and nurses should be comfortable questioning each other

Second, they can promote a culture of open communication and questioning around diagnosis. Trainees, physicians, and nurses should be comfortable questioning each other, including those higher up in the hierarchy, by saying, “I’m not sure” or “What else could this be?” to help reduce cognitive bias and expand the diagnostic possibilities.

Similarly, developing strategies to promote feedback on diagnosis among physicians will allow us all to learn from our diagnostic mistakes.

Use of the electronic medical record to assist in follow-up of pending diagnostic studies and patient return visits is yet another strategy.

Finally, healthcare organizations can adopt strategies to promote patient involvement in diagnosis, such as providing patients with copies of their test results and discharge summaries, encouraging the use of electronic patient communication portals, and empowering patients to ask questions related to their diagnosis. Prioritizing potential solutions to reduce diagnostic errors may be helpful in situations, depending on the context and environment, in which all proposed interventions may not be possible.

CASE CONTINUED: LEARNING FROM MISTAKES

The attending physician and resident in the case meet after the conference to review their clinical decision-making. Both are interested in learning from this case and improving their diagnostic skills in the future.

What specific steps can clinicians take to mitigate cognitive bias in daily practice?

In addition to continuing to expand one’s medical knowledge and gain more clinical experience, we can suggest several small steps to busy clinicians, taken individually or in combination with others that may improve diagnostic skills by reducing the potential for biased thinking in clinical practice.

From Croskerry P. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Adv Health Sci Educ 2009; 14:27–35. With kind permission from Springer Science and Business Media.

Think about your thinking. Our first recommendation would be to become more familiar with the dual process theory of clinical cognition (Figure 1).^37,38 This theoretical framework may be very helpful as a foundation from which to build better thinking skills. Physicians, especially residents, and students can be taught these concepts and their potential to contribute to diagnostic errors, and can use these skills to recognize those contributions in others’ diagnostic practices and even in their own.³⁹

Facilitating metacognition, or “thinking about one’s thinking,” may help clinicians catch themselves in thinking traps and provide the opportunity to reflect on biases retrospectively, as a double check or an opportunity to learn from a mistake.

Recognize your emotions. Gaining an understanding of the effect of one’s emotions on decision-making also can help clinicians free themselves of bias. As human beings, healthcare professionals are  susceptible to emotion, and the best approach to mitigate the emotional influences may be to consciously name them and adjust for them.⁴⁰

Because it is impractical to apply slow, analytical system 2 approaches to every case, skills that hone and develop more accurate, reliable system 1 thinking are crucial. Gaining broad exposure to increased numbers of cases may be the most reliable way to build an experiential repertoire of “illness scripts,” but there are ways to increase the experiential value of any case with a few techniques that have potential to promote better intuition.⁴¹

Embracing uncertainty in the early diagnostic process and envisioning the worst-case scenario in a case allows the consideration of additional diagnostic paths outside of the current working diagnosis, potentially priming the clinician to look for and recognize early warning signs that could argue against the initial diagnosis at a time when an adjustment could be made to prevent a bad outcome.

Practice progressive problem-solving,⁴² a technique in which the physician creates additional challenges to increase the cognitive burden of a “routine” case in an effort to train his or her mind and sharpen intuition. An example of this practice is contemplating a backup treatment plan in advance in the event of a poor response to or an adverse effect of treatment. Highly rated physicians and teachers perform this regularly.^43,44 Other ways to maximize the learning value of an individual case include seeking feedback on patient outcomes, especially when a patient has been discharged or transferred to another provider’s care, or when the physician goes off service.

Simulation, traditionally used for procedural training, has potential as well. Cognitive simulation, such as case reports or virtual patient modules, have potential to enhance clinical reasoning skills as well, though possibly at greater cost of time and expense.

Decreased reliance on memory is likely to improve diagnostic reasoning. Systems tools such as checklists⁴⁵ and health information technology⁴⁶ have potential to reduce diagnostic errors, not by taking thinking away from the clinician but by relieving the cognitive load enough to facilitate greater effort toward reasoning.

Slow down. Finally, and perhaps most important, recent models of clinical expertise have suggested that mastery comes from having a robust intuitive method, with a sense of the limitations of the intuitive approach, an ability to recognize the need to perform more analytical reasoning in select cases, and the willingness to do so. In short, it may well be that the hallmark of a master clinician is the propensity to slow down when necessary.⁴⁷

A ‘diagnostic time-out’ for safety might catch opportunities to recognize and mitigate biases and errors

If one considers diagnosis a cognitive procedure, perhaps a brief “diagnostic time-out” for safety might afford an opportunity to recognize and mitigate biases and errors. There are likely many potential scripts for a good diagnostic time-out, but to be functional it should be brief and simple to facilitate consistent use. We have recommended the following four questions to our residents as a starting point, any of which could signal the need to switch to a slower, analytic approach.

Four-step diagnostic time-out

• What else can it be?
• Is there anything about the case that does not fit?
• Is it possible that multiple processes are going on?
• Do I need to slow down?

These questions can serve as a double check for an intuitively formed initial working diagnosis, incorporating many of the principles discussed above, in a way that would hopefully avoid undue burden on a busy clinician. These techniques, it must be acknowledged, have not yet been directly tied to reductions in diagnostic errors. However, diagnostic errors, as discussed, are very difficult to identify and study, and these techniques will serve mainly to improve habits that are likely to show benefits over much longer time periods than most studies can measure.