Hardeep Singh, MD, MPH

As the death toll from the coronavirus disease 2019 (COVID-19) pandemic rapidly increases, the need to make a timely and accurate diagnosis has never been greater. Even before the pandemic, diagnostic errors (ie, missed, delayed, and incorrect diagnoses) had been one of the leading contributors to harm in health care.¹ The COVID-19 pandemic is likely to increase the risk of such errors for several reasons. The disease itself is new and knowledge of its clinical manifestations is still evolving. Both physical and psychological safety of clinicians and health system capacity are compromised and can affect clinical decision-making.² Situational factors such as staffing shortages and workarounds are more common, and clinicians in certain geographic areas are experiencing epic levels of stress, fatigue, and burnout. Finally, decisions in busy, chaotic and time-pressured healthcare systems with disrupted and/or newly designed care processes will be error prone.¹

Based on emerging literature and collaborative discussions across the globe, we propose a new typology of diagnostic errors of concern in the COVID-19 era (Table). These errors span the entire continuum of care and have both systems-based and cognitive origins. While some errors arise from previously described clinical reasoning fallacies, others are unique to the pandemic. We provide a user-friendly nomenclature while describing eight types of diagnostic errors and highlight mitigation strategies to reduce potential preventable harm caused by those errors.

The classic COVID-19 presentation of a febrile respiratory illness warrants confirmatory testing, but testing may not be available or produce a false-negative result, leading to an error we termed “Classic.” In the United States, efforts to develop and implement testing protocols are still evolving. There is wide local and regional variation in type and availability of tests, as well as accessibility of information regarding test performance characteristics or diagnostic yield.³ Test results that are false negatives or testing that is not performed can lead to delayed diagnosis of the disease, as well as continued spread.

Testing is similarly relevant when patients present with unusual or nonrespiratory symptoms. Both predominantly olfactory⁴ and gastrointestinal manifestations⁵ have now been described, and mysterious new associations, such as multisystem inflammatory syndromes, continue to emerge. A failure to recognize atypical presentations and associations, either because of testing problems or knowledge gaps, could lead to overlooking underlying COVID-19 diagnosis, an error we termed “Anomalous.”

Another error emerging in the pandemic is mislabeling patients who do not have COVID-19 as having the disease, particularly those with respiratory symptoms. This usually occurs in absence of testing in an overwhelmed health system with limited capacity to test or treat (eg, clinicians just assume it must be COVID-19 when the test is not available). This type of labeling error, called “Anchor,” introduces the risk of missing other respiratory infections such as bacterial sinusitis and pneumonia, as well as nonrespiratory conditions.

In patients with known COVID-19, a second underlying or concurrent condition may be missed, an error we termed “Secondary.” For instance, reports of coagulopathy-related pulmonary embolism⁶ and strokes in young patients with minimal symptoms⁷ have emerged just recently. Respiratory compromise may be mistakenly attributed to COVID-19 rather than looking for a new source of worsening, such as pulmonary embolism. Similarly, clinicians may not recognize subtle stroke symptoms in patients who were otherwise feeling well at home. Such cognitive errors will likely increase as it becomes harder for clinicians or health systems to keep up with new knowledge.

Collateral effects of the COVID-19 pandemic are also emerging. For instance, patients with symptoms of new acute conditions may be unwilling to visit acute care for evaluation because of infection risk, an error we termed “Acute Collateral.” Concerns are already being raised that patients with acute myocardial infarction⁸ and stroke⁹ are not coming in for evaluation. Similarly, there may be delays in diagnosis of important ambulatory conditions, including cancer,¹⁰ when appointments or elective procedures are canceled (“Chronic Collateral”). In the United Kingdom, referrals under the 2-week wait system–in which suspected cancer patients referred by general practitioners are seen within 2-weeks–fell by 70% over March to April, 2020.

Diagnosis of non–COVID-19 patients coming into the hospital may also be affected because of the understandably heightened state of attention to COVID-19 patients, capacity, and staffing issues, an error we termed “Strain.” Physicians, including surgeons, pediatricians, and radiologists, have been “redeployed” into acute care medical specialties. Cognitive errors increase when clinicians in new roles face unfamiliar situations and disease manifestations. Although these clinicians may be highly experienced previously, they may have insufficient skills and experience in their new roles and may not feel comfortable asking for guidance.¹¹

Lastly, clinicians are increasingly using intermediary mechanisms, such as PPE and telemedicine technologies, to interact with patients. This is new for both parties and could introduce new types of errors, which we termed “Unintended.” Furthermore, interactions mediated via telemedicine technologies or PPE, as well as PPE conservation measures such as reduced room entries and e-consultation, may reduce the ability of even well-trained clinicians to take effective histories, perform physical exams, and monitor symptoms. In fact, infection-prevention isolation has been shown to put patients at risk of preventable adverse events in hospitalized patients.¹²

There are many strategies that health systems could deploy to try to minimize these eight types of diagnostic errors. We organize mitigation strategies using the Safer Dx framework, which proposes sociotechnical approaches (ie, both technology and other systems-based approaches) to reduce diagnostic error.¹³

Technology for Cognitive Support

Up-to-date electronic decision support is needed to optimize test interpretation. Technology can also help scale and facilitate rapid adoption of standardized safety practices and protocols to address emerging risks areas. For instance, there are early efforts to create, implement, and disseminate smart algorithms to predict risks of non–COVID-19 diagnoses such as venous thromboembolism, patient transfer protocols on how best to reduce the burden at overstressed hospitals, protocols to triage rescheduling of elective procedures based on potential risk as determined from data in the electronic health record, new rules for creating outreach to patients who have missed appointments to prevent delays in their evaluation and diagnosis, and triage protocols and follow-up systems to optimize telemedicine.¹⁴

Optimized Workflow and Communication

When in-person contact is limited, specific practices (eg, providing patients with iPads, use of reflective listening, and use of optimal nonverbal communication strategies such as eye-contact) can still facilitate comprehensive discussions with patients and families about symptoms and encourage them to speak up if and when they have concerns.¹⁵ For patients reached through telemedicine, follow-up appointments and surveys should be done to ensure that symptoms and concerns have been addressed. For clinicians working in new clinical areas unfamiliar to them (eg, surgeons on medical floors, hospitalists in ICUs), buddy systems can pair these clinicians with more experienced clinicians to make it easier for them to ask for help. Visual aids, decision support, and reliable error-prevention resources can also be helpful.¹⁶

People-Focused Interventions

Some clinicians are used to practicing solo, but this is the time to start “diagnostic huddles” for discussion of challenging cases with symptoms that are unusual or not improving as expected or for determining whether anything has been missed. In addition to encouraging patients to use reliable digital tools for self-triage, outreach to patients and the public must also advise them (with the help of public health authorities and the media) to seek medical assistance for certain important conditions such as acute myocardial infarction and stroke.

Organizational Strategies

Fundamental safety strategies must be ensured. First, it is critical to have a strong safety culture in which staff feel empowered to speak up, ask questions or ask for help, and report concerns without fear of repercussions or judgement. Culture can take years to develop, but due to rapidly changing circumstances in a crisis, there are ways for healthcare leaders to create changes more quickly. In addition to having daily huddles, leaders should be visible and communicate clearly about the behaviors and norms they are supporting. In particular, frequent leadership rounding (either virtually or in person)—during which leaders ask questions and encourage discussions of concerns in a supportive way—can foster the kind of culture that is needed. All organizations should implement peer support, counseling, limits on hours worked, and other support strategies for all clinicians to minimize the fatigue, stress, and anxiety that can impair cognitive function.¹⁷

Organizations must also be able to identify these errors to help understand root causes and prioritize interventions.¹⁸ For example, streamlined reporting systems that use apps and hotlines could be developed quickly to ensure that clinicians and patients/families can easily report these errors. Electronic triggers can help detect specific situations indicative of error or delay (eg, patient not on precautions gets switched to precautions during a hospitalization; absence of follow-up on abnormal tests).¹⁹

Learning systems—both within and across hospitals—should continue to share diagnostic challenges, the most up-to-date information, and best practices/protocols, and identify opportunities for improvement together. Many hospitals are having virtual grand rounds, journals are rapidly sharing new information via open access, regional and national cross-organizational and multidisciplinary learning networks of various groups have emerged (such as networks of oncologists, infectious disease specialists, and hospitalists), and new and transparent communication channels have developed between state and local health departments, government leaders, health systems, and the public. These forums should discuss emerging knowledge on diagnosis and strategies for risk reduction, many of which will unfold over the next few months.

State/Federal Policies and Regulations

While there is progress, additional challenges with accessibility, accuracy and performance of testing should be addressed at a national level. Guidance is needed on which asymptomatic people should be tested, both within and outside hospitals. Standardized metrics should be developed to monitor diagnostic performance and outcomes and evaluate how COVID-19 diagnosis errors affect different demographics. For instance, black and Hispanic individuals are disproportionately represented in COVID-19 cases and deaths, so metrics could be further stratified by race and ethnicity to ensure that we can understand and eliminate inequities, such as lack of access to care or testing.²⁰

Clinicians must be provided with both cognitive and system support so they can do what they do best—diagnose and treat patients and save lives. Intermittent epidemic spikes based on location and season, including a potentially bigger spike of cases later this year, are now projected. Risks and recommendations discussed herein should therefore be rapidly shared to help redesign and strengthen the work system and protect patients from preventable diagnosis-related harm.

The views expressed in this article do not represent the views of the U.S. Department of Veterans Affairs or the United States government.

As the death toll from the coronavirus disease 2019 (COVID-19) pandemic rapidly increases, the need to make a timely and accurate diagnosis has never been greater. Even before the pandemic, diagnostic errors (ie, missed, delayed, and incorrect diagnoses) had been one of the leading contributors to harm in health care.¹ The COVID-19 pandemic is likely to increase the risk of such errors for several reasons. The disease itself is new and knowledge of its clinical manifestations is still evolving. Both physical and psychological safety of clinicians and health system capacity are compromised and can affect clinical decision-making.² Situational factors such as staffing shortages and workarounds are more common, and clinicians in certain geographic areas are experiencing epic levels of stress, fatigue, and burnout. Finally, decisions in busy, chaotic and time-pressured healthcare systems with disrupted and/or newly designed care processes will be error prone.¹

Based on emerging literature and collaborative discussions across the globe, we propose a new typology of diagnostic errors of concern in the COVID-19 era (Table). These errors span the entire continuum of care and have both systems-based and cognitive origins. While some errors arise from previously described clinical reasoning fallacies, others are unique to the pandemic. We provide a user-friendly nomenclature while describing eight types of diagnostic errors and highlight mitigation strategies to reduce potential preventable harm caused by those errors.

The classic COVID-19 presentation of a febrile respiratory illness warrants confirmatory testing, but testing may not be available or produce a false-negative result, leading to an error we termed “Classic.” In the United States, efforts to develop and implement testing protocols are still evolving. There is wide local and regional variation in type and availability of tests, as well as accessibility of information regarding test performance characteristics or diagnostic yield.³ Test results that are false negatives or testing that is not performed can lead to delayed diagnosis of the disease, as well as continued spread.

Testing is similarly relevant when patients present with unusual or nonrespiratory symptoms. Both predominantly olfactory⁴ and gastrointestinal manifestations⁵ have now been described, and mysterious new associations, such as multisystem inflammatory syndromes, continue to emerge. A failure to recognize atypical presentations and associations, either because of testing problems or knowledge gaps, could lead to overlooking underlying COVID-19 diagnosis, an error we termed “Anomalous.”

Another error emerging in the pandemic is mislabeling patients who do not have COVID-19 as having the disease, particularly those with respiratory symptoms. This usually occurs in absence of testing in an overwhelmed health system with limited capacity to test or treat (eg, clinicians just assume it must be COVID-19 when the test is not available). This type of labeling error, called “Anchor,” introduces the risk of missing other respiratory infections such as bacterial sinusitis and pneumonia, as well as nonrespiratory conditions.

In patients with known COVID-19, a second underlying or concurrent condition may be missed, an error we termed “Secondary.” For instance, reports of coagulopathy-related pulmonary embolism⁶ and strokes in young patients with minimal symptoms⁷ have emerged just recently. Respiratory compromise may be mistakenly attributed to COVID-19 rather than looking for a new source of worsening, such as pulmonary embolism. Similarly, clinicians may not recognize subtle stroke symptoms in patients who were otherwise feeling well at home. Such cognitive errors will likely increase as it becomes harder for clinicians or health systems to keep up with new knowledge.

Collateral effects of the COVID-19 pandemic are also emerging. For instance, patients with symptoms of new acute conditions may be unwilling to visit acute care for evaluation because of infection risk, an error we termed “Acute Collateral.” Concerns are already being raised that patients with acute myocardial infarction⁸ and stroke⁹ are not coming in for evaluation. Similarly, there may be delays in diagnosis of important ambulatory conditions, including cancer,¹⁰ when appointments or elective procedures are canceled (“Chronic Collateral”). In the United Kingdom, referrals under the 2-week wait system–in which suspected cancer patients referred by general practitioners are seen within 2-weeks–fell by 70% over March to April, 2020.

Diagnosis of non–COVID-19 patients coming into the hospital may also be affected because of the understandably heightened state of attention to COVID-19 patients, capacity, and staffing issues, an error we termed “Strain.” Physicians, including surgeons, pediatricians, and radiologists, have been “redeployed” into acute care medical specialties. Cognitive errors increase when clinicians in new roles face unfamiliar situations and disease manifestations. Although these clinicians may be highly experienced previously, they may have insufficient skills and experience in their new roles and may not feel comfortable asking for guidance.¹¹

Lastly, clinicians are increasingly using intermediary mechanisms, such as PPE and telemedicine technologies, to interact with patients. This is new for both parties and could introduce new types of errors, which we termed “Unintended.” Furthermore, interactions mediated via telemedicine technologies or PPE, as well as PPE conservation measures such as reduced room entries and e-consultation, may reduce the ability of even well-trained clinicians to take effective histories, perform physical exams, and monitor symptoms. In fact, infection-prevention isolation has been shown to put patients at risk of preventable adverse events in hospitalized patients.¹²

There are many strategies that health systems could deploy to try to minimize these eight types of diagnostic errors. We organize mitigation strategies using the Safer Dx framework, which proposes sociotechnical approaches (ie, both technology and other systems-based approaches) to reduce diagnostic error.¹³

Technology for Cognitive Support

Up-to-date electronic decision support is needed to optimize test interpretation. Technology can also help scale and facilitate rapid adoption of standardized safety practices and protocols to address emerging risks areas. For instance, there are early efforts to create, implement, and disseminate smart algorithms to predict risks of non–COVID-19 diagnoses such as venous thromboembolism, patient transfer protocols on how best to reduce the burden at overstressed hospitals, protocols to triage rescheduling of elective procedures based on potential risk as determined from data in the electronic health record, new rules for creating outreach to patients who have missed appointments to prevent delays in their evaluation and diagnosis, and triage protocols and follow-up systems to optimize telemedicine.¹⁴

Optimized Workflow and Communication

When in-person contact is limited, specific practices (eg, providing patients with iPads, use of reflective listening, and use of optimal nonverbal communication strategies such as eye-contact) can still facilitate comprehensive discussions with patients and families about symptoms and encourage them to speak up if and when they have concerns.¹⁵ For patients reached through telemedicine, follow-up appointments and surveys should be done to ensure that symptoms and concerns have been addressed. For clinicians working in new clinical areas unfamiliar to them (eg, surgeons on medical floors, hospitalists in ICUs), buddy systems can pair these clinicians with more experienced clinicians to make it easier for them to ask for help. Visual aids, decision support, and reliable error-prevention resources can also be helpful.¹⁶

People-Focused Interventions

Some clinicians are used to practicing solo, but this is the time to start “diagnostic huddles” for discussion of challenging cases with symptoms that are unusual or not improving as expected or for determining whether anything has been missed. In addition to encouraging patients to use reliable digital tools for self-triage, outreach to patients and the public must also advise them (with the help of public health authorities and the media) to seek medical assistance for certain important conditions such as acute myocardial infarction and stroke.

Organizational Strategies

Fundamental safety strategies must be ensured. First, it is critical to have a strong safety culture in which staff feel empowered to speak up, ask questions or ask for help, and report concerns without fear of repercussions or judgement. Culture can take years to develop, but due to rapidly changing circumstances in a crisis, there are ways for healthcare leaders to create changes more quickly. In addition to having daily huddles, leaders should be visible and communicate clearly about the behaviors and norms they are supporting. In particular, frequent leadership rounding (either virtually or in person)—during which leaders ask questions and encourage discussions of concerns in a supportive way—can foster the kind of culture that is needed. All organizations should implement peer support, counseling, limits on hours worked, and other support strategies for all clinicians to minimize the fatigue, stress, and anxiety that can impair cognitive function.¹⁷

Organizations must also be able to identify these errors to help understand root causes and prioritize interventions.¹⁸ For example, streamlined reporting systems that use apps and hotlines could be developed quickly to ensure that clinicians and patients/families can easily report these errors. Electronic triggers can help detect specific situations indicative of error or delay (eg, patient not on precautions gets switched to precautions during a hospitalization; absence of follow-up on abnormal tests).¹⁹

Learning systems—both within and across hospitals—should continue to share diagnostic challenges, the most up-to-date information, and best practices/protocols, and identify opportunities for improvement together. Many hospitals are having virtual grand rounds, journals are rapidly sharing new information via open access, regional and national cross-organizational and multidisciplinary learning networks of various groups have emerged (such as networks of oncologists, infectious disease specialists, and hospitalists), and new and transparent communication channels have developed between state and local health departments, government leaders, health systems, and the public. These forums should discuss emerging knowledge on diagnosis and strategies for risk reduction, many of which will unfold over the next few months.

State/Federal Policies and Regulations

While there is progress, additional challenges with accessibility, accuracy and performance of testing should be addressed at a national level. Guidance is needed on which asymptomatic people should be tested, both within and outside hospitals. Standardized metrics should be developed to monitor diagnostic performance and outcomes and evaluate how COVID-19 diagnosis errors affect different demographics. For instance, black and Hispanic individuals are disproportionately represented in COVID-19 cases and deaths, so metrics could be further stratified by race and ethnicity to ensure that we can understand and eliminate inequities, such as lack of access to care or testing.²⁰

Clinicians must be provided with both cognitive and system support so they can do what they do best—diagnose and treat patients and save lives. Intermittent epidemic spikes based on location and season, including a potentially bigger spike of cases later this year, are now projected. Risks and recommendations discussed herein should therefore be rapidly shared to help redesign and strengthen the work system and protect patients from preventable diagnosis-related harm.

The views expressed in this article do not represent the views of the U.S. Department of Veterans Affairs or the United States government.

As the death toll from the coronavirus disease 2019 (COVID-19) pandemic rapidly increases, the need to make a timely and accurate diagnosis has never been greater. Even before the pandemic, diagnostic errors (ie, missed, delayed, and incorrect diagnoses) had been one of the leading contributors to harm in health care.¹ The COVID-19 pandemic is likely to increase the risk of such errors for several reasons. The disease itself is new and knowledge of its clinical manifestations is still evolving. Both physical and psychological safety of clinicians and health system capacity are compromised and can affect clinical decision-making.² Situational factors such as staffing shortages and workarounds are more common, and clinicians in certain geographic areas are experiencing epic levels of stress, fatigue, and burnout. Finally, decisions in busy, chaotic and time-pressured healthcare systems with disrupted and/or newly designed care processes will be error prone.¹

Based on emerging literature and collaborative discussions across the globe, we propose a new typology of diagnostic errors of concern in the COVID-19 era (Table). These errors span the entire continuum of care and have both systems-based and cognitive origins. While some errors arise from previously described clinical reasoning fallacies, others are unique to the pandemic. We provide a user-friendly nomenclature while describing eight types of diagnostic errors and highlight mitigation strategies to reduce potential preventable harm caused by those errors.

The classic COVID-19 presentation of a febrile respiratory illness warrants confirmatory testing, but testing may not be available or produce a false-negative result, leading to an error we termed “Classic.” In the United States, efforts to develop and implement testing protocols are still evolving. There is wide local and regional variation in type and availability of tests, as well as accessibility of information regarding test performance characteristics or diagnostic yield.³ Test results that are false negatives or testing that is not performed can lead to delayed diagnosis of the disease, as well as continued spread.

Testing is similarly relevant when patients present with unusual or nonrespiratory symptoms. Both predominantly olfactory⁴ and gastrointestinal manifestations⁵ have now been described, and mysterious new associations, such as multisystem inflammatory syndromes, continue to emerge. A failure to recognize atypical presentations and associations, either because of testing problems or knowledge gaps, could lead to overlooking underlying COVID-19 diagnosis, an error we termed “Anomalous.”

Another error emerging in the pandemic is mislabeling patients who do not have COVID-19 as having the disease, particularly those with respiratory symptoms. This usually occurs in absence of testing in an overwhelmed health system with limited capacity to test or treat (eg, clinicians just assume it must be COVID-19 when the test is not available). This type of labeling error, called “Anchor,” introduces the risk of missing other respiratory infections such as bacterial sinusitis and pneumonia, as well as nonrespiratory conditions.

In patients with known COVID-19, a second underlying or concurrent condition may be missed, an error we termed “Secondary.” For instance, reports of coagulopathy-related pulmonary embolism⁶ and strokes in young patients with minimal symptoms⁷ have emerged just recently. Respiratory compromise may be mistakenly attributed to COVID-19 rather than looking for a new source of worsening, such as pulmonary embolism. Similarly, clinicians may not recognize subtle stroke symptoms in patients who were otherwise feeling well at home. Such cognitive errors will likely increase as it becomes harder for clinicians or health systems to keep up with new knowledge.

Collateral effects of the COVID-19 pandemic are also emerging. For instance, patients with symptoms of new acute conditions may be unwilling to visit acute care for evaluation because of infection risk, an error we termed “Acute Collateral.” Concerns are already being raised that patients with acute myocardial infarction⁸ and stroke⁹ are not coming in for evaluation. Similarly, there may be delays in diagnosis of important ambulatory conditions, including cancer,¹⁰ when appointments or elective procedures are canceled (“Chronic Collateral”). In the United Kingdom, referrals under the 2-week wait system–in which suspected cancer patients referred by general practitioners are seen within 2-weeks–fell by 70% over March to April, 2020.

Diagnosis of non–COVID-19 patients coming into the hospital may also be affected because of the understandably heightened state of attention to COVID-19 patients, capacity, and staffing issues, an error we termed “Strain.” Physicians, including surgeons, pediatricians, and radiologists, have been “redeployed” into acute care medical specialties. Cognitive errors increase when clinicians in new roles face unfamiliar situations and disease manifestations. Although these clinicians may be highly experienced previously, they may have insufficient skills and experience in their new roles and may not feel comfortable asking for guidance.¹¹

Lastly, clinicians are increasingly using intermediary mechanisms, such as PPE and telemedicine technologies, to interact with patients. This is new for both parties and could introduce new types of errors, which we termed “Unintended.” Furthermore, interactions mediated via telemedicine technologies or PPE, as well as PPE conservation measures such as reduced room entries and e-consultation, may reduce the ability of even well-trained clinicians to take effective histories, perform physical exams, and monitor symptoms. In fact, infection-prevention isolation has been shown to put patients at risk of preventable adverse events in hospitalized patients.¹²

There are many strategies that health systems could deploy to try to minimize these eight types of diagnostic errors. We organize mitigation strategies using the Safer Dx framework, which proposes sociotechnical approaches (ie, both technology and other systems-based approaches) to reduce diagnostic error.¹³

Technology for Cognitive Support

Up-to-date electronic decision support is needed to optimize test interpretation. Technology can also help scale and facilitate rapid adoption of standardized safety practices and protocols to address emerging risks areas. For instance, there are early efforts to create, implement, and disseminate smart algorithms to predict risks of non–COVID-19 diagnoses such as venous thromboembolism, patient transfer protocols on how best to reduce the burden at overstressed hospitals, protocols to triage rescheduling of elective procedures based on potential risk as determined from data in the electronic health record, new rules for creating outreach to patients who have missed appointments to prevent delays in their evaluation and diagnosis, and triage protocols and follow-up systems to optimize telemedicine.¹⁴

Optimized Workflow and Communication

When in-person contact is limited, specific practices (eg, providing patients with iPads, use of reflective listening, and use of optimal nonverbal communication strategies such as eye-contact) can still facilitate comprehensive discussions with patients and families about symptoms and encourage them to speak up if and when they have concerns.¹⁵ For patients reached through telemedicine, follow-up appointments and surveys should be done to ensure that symptoms and concerns have been addressed. For clinicians working in new clinical areas unfamiliar to them (eg, surgeons on medical floors, hospitalists in ICUs), buddy systems can pair these clinicians with more experienced clinicians to make it easier for them to ask for help. Visual aids, decision support, and reliable error-prevention resources can also be helpful.¹⁶

People-Focused Interventions

Some clinicians are used to practicing solo, but this is the time to start “diagnostic huddles” for discussion of challenging cases with symptoms that are unusual or not improving as expected or for determining whether anything has been missed. In addition to encouraging patients to use reliable digital tools for self-triage, outreach to patients and the public must also advise them (with the help of public health authorities and the media) to seek medical assistance for certain important conditions such as acute myocardial infarction and stroke.

Organizational Strategies

Fundamental safety strategies must be ensured. First, it is critical to have a strong safety culture in which staff feel empowered to speak up, ask questions or ask for help, and report concerns without fear of repercussions or judgement. Culture can take years to develop, but due to rapidly changing circumstances in a crisis, there are ways for healthcare leaders to create changes more quickly. In addition to having daily huddles, leaders should be visible and communicate clearly about the behaviors and norms they are supporting. In particular, frequent leadership rounding (either virtually or in person)—during which leaders ask questions and encourage discussions of concerns in a supportive way—can foster the kind of culture that is needed. All organizations should implement peer support, counseling, limits on hours worked, and other support strategies for all clinicians to minimize the fatigue, stress, and anxiety that can impair cognitive function.¹⁷

Organizations must also be able to identify these errors to help understand root causes and prioritize interventions.¹⁸ For example, streamlined reporting systems that use apps and hotlines could be developed quickly to ensure that clinicians and patients/families can easily report these errors. Electronic triggers can help detect specific situations indicative of error or delay (eg, patient not on precautions gets switched to precautions during a hospitalization; absence of follow-up on abnormal tests).¹⁹

Learning systems—both within and across hospitals—should continue to share diagnostic challenges, the most up-to-date information, and best practices/protocols, and identify opportunities for improvement together. Many hospitals are having virtual grand rounds, journals are rapidly sharing new information via open access, regional and national cross-organizational and multidisciplinary learning networks of various groups have emerged (such as networks of oncologists, infectious disease specialists, and hospitalists), and new and transparent communication channels have developed between state and local health departments, government leaders, health systems, and the public. These forums should discuss emerging knowledge on diagnosis and strategies for risk reduction, many of which will unfold over the next few months.

State/Federal Policies and Regulations

While there is progress, additional challenges with accessibility, accuracy and performance of testing should be addressed at a national level. Guidance is needed on which asymptomatic people should be tested, both within and outside hospitals. Standardized metrics should be developed to monitor diagnostic performance and outcomes and evaluate how COVID-19 diagnosis errors affect different demographics. For instance, black and Hispanic individuals are disproportionately represented in COVID-19 cases and deaths, so metrics could be further stratified by race and ethnicity to ensure that we can understand and eliminate inequities, such as lack of access to care or testing.²⁰

Clinicians must be provided with both cognitive and system support so they can do what they do best—diagnose and treat patients and save lives. Intermittent epidemic spikes based on location and season, including a potentially bigger spike of cases later this year, are now projected. Risks and recommendations discussed herein should therefore be rapidly shared to help redesign and strengthen the work system and protect patients from preventable diagnosis-related harm.

The views expressed in this article do not represent the views of the U.S. Department of Veterans Affairs or the United States government.

Diagnostic error—defined as a failure to establish an accurate and timely explanation of the patient’s health problem—is an important source of patient harm.¹ Data suggest that all patients will experience at least 1 diagnostic error in their lifetime.^2-4 Not surprisingly, diagnostic errors are among the leading categories of paid malpractice claims in the United States.⁵

Despite diagnostic errors being morbid and sometimes deadly in the hospital,^6,7 little is known about how residents and learners approach diagnostic decision making. Errors in diagnosis are believed to stem from cognitive or system failures,⁸ with errors in cognition believed to occur due to rapid, reflexive thinking operating in the absence of a more analytical, deliberate process. System-based problems (eg, lack of expert availability, technology barriers, and access to data) have also been cited as contributors.⁹ However, whether and how these apply to trainees is not known.

Therefore, we conducted a focused ethnography of inpatient medicine teams (ie, attendings, residents, interns, and medical students) in 2 affiliated teaching hospitals, aiming to (a) observe the process of diagnosis by trainees and (b) identify methods to improve the diagnostic process and prevent errors.

We designed a multimethod, focused ethnographic study to examine diagnostic decision making in hospital settings.^10,11 In contrast to anthropologic ethnographies that study entire fields using open-ended questions, our study was designed to examine the process of diagnosis from the perspective of clinicians engaged in this activity.¹¹ This approach allowed us to capture diagnostic decisions and cognitive and system-based factors in a manner currently lacking in the literature.¹²

Setting and Participants

Between January 2016 and May 2016, we observed the members of four inpatient internal medicine teaching teams at 2 affiliated teaching hospitals. We purposefully selected teaching teams for observation because they are the primary model of care in academic settings and we have expertise in carrying out similar studies.^13,14 Teaching teams typically consisted of a medical attending (senior-level physician), 1 senior resident (a second- or third-year postgraduate trainee), two interns (a trainee in their first postgraduate year), and two to four  medical students. Teams were selected at random using existing schedules and followed Monday to Friday so as to permit observation of work on call and noncall days. Owing to manpower limitations, weekend and night shifts were not observed. However, overnight events were captured during morning rounds.

Most of the teams began rounds at 8:30 AM. Typically, rounds lasted for 90–120 min and concluded with a recap (ie, “running the list”) with a review of explicit plans for patients after they had been evaluated by the attending. This discussion often occurred in the team rooms, with the attending leading the discussion with the trainees.

Data Collection

A multidisciplinary team, including clinicians (eg, physicians, nurses), nonclinicians (eg, qualitative researchers, social scientists), and healthcare engineers, conducted the observations. We observed preround activities of interns and residents before arrival of the attending (7:00 AM - 8:30 AM), followed by morning rounds with the entire team, and afternoon work that included senior residents, interns, and students.

To capture multiple aspects of the diagnostic process, we collected data using field notes modeled on components of the National Academy of Science model for diagnosis (Appendix).^1,15 This model encompasses phases of the diagnostic process (eg, data gathering, integration, formulation of a working diagnosis, treatment delivery, and outcomes) and the work system (team members, organization, technology and tools, physical environment, tasks).

Focus Groups and Interviews

At the end of weekly observations, we conducted focus groups with the residents and one-on- one interviews with the attendings. Focus groups with the residents were conducted to encourage a group discussion about the diagnostic process. Separate interviews with the attendings were performed to ensure that power differentials did not influence discussions. During focus groups, we specifically asked about challenges and possible solutions to improve diagnosis. Experienced qualitative methodologists (J.F., M.H., M.Q.) used semistructured interview guides for discussions (Appendix).

Data Analysis

After aggregating and reading the data, three reviewers (V.C., S.K., S.S.) began inductive analysis by handwriting notes and initial reflective thoughts to create preliminary codes. Multiple team members then reread the original field notes and the focus group/interview data to refine the preliminary codes and develop additional codes. Next, relationships between codes were identified and used to develop key themes. Triangulation of data collected from observations and interview/focus group sessions was carried out to compare data that we surmised with data that were verbalized by the team. The developed themes were discussed as a group to ensure consistency of major findings.

Ethical and Regulatory Oversight

This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).

Four teaching teams (4 attendings, 4 senior residents, 9 interns, and 14 medical students) were observed over 33 distinct shifts and 168 hours. Observations included morning rounds (96 h), postround call days (52 h), and postround non-call days (20 h). Morning rounds lasted an average of 127 min (range: 48-232 min) and included an average of 9 patients (range: 4-16 patients).

Themes Regarding the Diagnostic Process

We identified the following 4 primary themes related to the diagnostic process in teaching hospitals: (1) diagnosis is a social phenomenon; (2) data necessary to make diagnoses are fragmented; (3) distractions undermine the diagnostic process; and (4) time pressures interfere with diagnostic decision making (Appendix Table 1).

(1) Diagnosis is a Social Phenomenon.

Team members viewed the process of diagnosis as a social exchange of facts, findings, and strategies within a defined structure. The opportunity to discuss impressions with others was valued as a means to share, test, and process assumptions.

“Rounds are the most important part of the process. That is where we make most decisions in a collective, collaborative way with the attending present. We bounce ideas off each other.” (Intern)

Typical of social processes, variations based on time of day and schedule were observed. For instance, during call days, learners gathered data and formed working diagnosis and treatment plans with minimal attending interaction. This separation of roles and responsibilities introduced a hierarchy within diagnosis as follows:

“The interns would not call me first; they would talk to the senior resident and then if the senior thought he should chat with me, then they would call. But for the most part, they gather information and come up with the plan.” (Attending).

The work system was suited to facilitate social interactions. For instance, designated rooms (with team members informally assigned to a computer) provided physical proximity of the resident to interns and medical students. In this space, numerous informal discussions between team members (eg, “What do you think about this test?” “I’m not sure what to do about this finding.” “Should I call a [consult] on this patient?”) were observed. Although proximity to each other was viewed as beneficial, dangers to the social nature of diagnosis in the form of anchoring (ie, a cognitive bias where emphasis is placed on the first piece of data)¹⁶ were also mentioned. Similarly, the paradox associated with social proof (ie, the pressure to assume conformity within a group) was also observed as disagreement between team members and attendings rarely occurred during observations.

“I mean, they’re the attending, right? It’s hard to argue with them when they want a test or something done. When I do push back, it’s rare that others will support me–so it’s usually me and the attending.” (Resident)

“I would push back if I think it’s really bad for the patient or could cause harm–but the truth is, it doesn’t happen much.” (Intern)

(2) Data Necessary to Make Diagnoses are Fragmented

Team members universally cited fragmentation in data delivery, retrieval, and processing as a barrier to diagnosis. Team members indicated that test results might not be looked at or acted upon in a timely manner, and participants pointed to the electronic medical record as a source of this challenge.

“Before I knew about [the app for Epic], I would literally sit on the computer to get all the information we would need on rounds. Its key to making decisions. We often say we will do something, only to find the test result doesn’t support it–and then we’re back to square 1.” (Intern)

Information used by teams came from myriad sources (eg, patients, family members, electronic records) and from various settings (eg, emergency department, patient rooms, discussions with consultants). Additionally, test results often appeared without warning. Thus, availability of information was poorly aligned with clinical duties.

“They (the lab) will call us when a blood culture is positive or something is off. That is very helpful but it often comes later in the day, when we’re done with rounds.” (Resident)

The work system was highlighted as a key contributor to data fragmentation. Peculiarities of our electronic medical record (EMR) and how data were collected, stored, or presented were described as “frustrating,” and “unsafe,” by team members. Correspondingly, we frequently observed interns asking for assistance for tasks such as ordering tests or finding information despite being “trained” to use the EMR.

“People have to learn how to filter, how to recognize the most important points and link data streams together in terms of causality. But we assume they know where to find that information. It’s actually a very hard thing to do, for both the house staff and me.” (Attending)

(3) Distractions Undermine the Diagnostic Process

Distractions often created cognitive difficulties. For example, ambient noise and interruptions from neighbors working on other teams were cited as barriers to diagnosis. In addition, we observed several team members using headphones to drown out ambient noise while working on the computer.

“I know I shouldn’t do it (wear headphones), but I have no other way of turning down the noise so I can concentrate.” (Intern)

Similarly, the unpredictable nature and the volume of pages often interrupted thinking about diagnosis.

“Sometimes the pager just goes off all the time and (after making sure its not an urgent issue), I will just ignore it for a bit, especially if I am in the middle of something. It would be great if I could finish my thought process knowing I would not be interrupted.” (Resident)

To mitigate this problem, 1 attending described how he would proactively seek out nurses caring for his patients to “head off” questions (eg, “I will renew the restraints and medications this morning,” and “Is there anything you need in terms of orders for this patient that I can take care of now?”) that might lead to pages. Another resident described his approach as follows:

“I make it a point to tell the nurses where I will be hanging out and where they can find me if they have any questions. I tell them to come talk to me rather than page me since that will be less distracting.” (Resident).

Most of the interns described documentation work such as writing admission and progress notes in negative terms (“an academic exercise,” “part of the billing activity”). However, in the context of interruptions, some described this as helpful.

“The most valuable part of the thinking process was writing the assessment and plan because that’s actually my schema for all problems. It literally is the only time where I can sit and collect my thoughts to formulate a diagnosis and plan.” (Intern)

(4) Time Pressures Interfere With Diagnostic Decision Making

All team members spoke about the challenge of finding time for diagnosis during the workday. Often, they had to skip learning sessions for this purpose.

“They tell us we should go to morning report or noon conference but when I’m running around trying to get things done. I hate having to choose between my education and doing what’s best for the patient–but that’s often what it comes down to.” (Intern)

When specifically asked whether setting aside dedicated time to specifically review and formulate diagnoses would be valuable, respondents were uniformly enthusiastic. Team members described attentional conflicts as being the worst when “cross covering” other teams on call days, as their patient load effectively doubled during this time. Of note, cross-covering occurred when teams were also on call—and thus took them away from important diagnostic activities such as data gathering or synthesis for patients they were admitting.

“If you were to ever design a system where errors were likely–this is how you would design it: take a team with little supervision, double their patient load, keep them busy with new challenging cases and then ask questions about patients they know little about.” (Resident)

Although diagnostic errors have been called “the next frontier for patient safety,”¹⁷ little is known about the process, barriers, and facilitators to diagnosis in teaching hospitals. In this focused ethnography conducted at 2 academic medical centers, we identified multiple cognitive and system-level challenges and potential strategies to improve diagnosis from trainees engaged in this activity. Key themes identified by those we observed included the social nature of diagnosis, fragmented information delivery, constant distractions and interruptions, and time pressures. In turn, these insights allow us to generate strategies that can be applied to improve the diagnostic process in teaching hospitals.

Our study underscores the importance of social interactions in diagnosis. In contrast, most of the interventions to prevent diagnostic errors target individual providers through practices such as metacognition and “thinking about thinking.”^18-20 These interventions are based on Daniel Kahnemann’s work on dual thought process. Type 1 thought processes are fast, subconscious, reflexive, largely intuitive, and more vulnerable to error. In contrast, Type 2 processes are slower, deliberate, analytic, and less prone to error.²¹ Although an individual’s Type 2 thought capacity is limited, a major goal of cognitive interventions is to encourage Type 2 over Type 1 thinking, an approach termed “de-biasing.”^22-24 Unfortunately, cognitive interventions testing such approaches have suffered mixed results–perhaps because of lack of focus on collective wisdom or group thinking, which may be key to diagnosis from our findings.^9,25 In this sense, morning rounds were a social gathering used to strategize and develop care plans, but with limited time to think about diagnosis.²⁶ Introduction of defined periods for individuals to engage in diagnostic activities such as de-biasing (ie, asking “what else could this be)²⁷ before or after rounds may provide an opportunity for reflection and improving diagnosis. In addition, embedding tools such as diagnosis expanders and checklists within these defined time slots^28,29 may prove to be useful in reflecting on diagnosis and preventing diagnostic errors.

An unexpected yet important finding from this study were the challenges posed by distractions and the physical environment. Potentially maladaptive workarounds to these interruptions included use of headphones; more productive strategies included updating nurses with plans to avert pages and creating a list of activities to ensure that key tasks were not forgotten.^30,31 Applying lessons from aviation, a focused effort to limit distractions during key portions of the day, might be worth considering for diagnostic safety.³² Similarly, improving the environment in which diagnosis occurs—including creating spaces that are quiet, orderly, and optimized for thinking—may be valuable.³³Our study has limitations. First, our findings are limited to direct observations; we are thus unable to comment on how unobserved aspects of care (eg, cognitive processes) might have influenced our findings. Our observations of clinical care might also have introduced a Hawthorne effect. However, because we were closely integrated with teams and conducted focus groups to corroborate our assessments, we believe that this was not the case. Second, we did not identify diagnostic errors or link processes we observed to errors. Third, our approach is limited to 2 teaching centers, thereby limiting the generalizability of findings. Relatedly, we were only able to conduct observations during weekdays; differences in weekend and night resources might affect our insights.

The cognitive and system-based barriers faced by clinicians in teaching hospitals suggest that new methods to improve diagnosis are needed. Future interventions such as defined “time-outs” for diagnosis, strategies focused on limiting distractions, and methods to improve communication between team members are novel and have parallels in other industries. As challenges to quantify diagnostic errors abound,³⁴ improving cognitive- and system-based factors via reflection through communication, concentration, and organization is necessary to improve medical decision making in academic medical centers.

Disclosures

None declared for all coauthors.

Funding

This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11-222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.

Diagnostic error—defined as a failure to establish an accurate and timely explanation of the patient’s health problem—is an important source of patient harm.¹ Data suggest that all patients will experience at least 1 diagnostic error in their lifetime.^2-4 Not surprisingly, diagnostic errors are among the leading categories of paid malpractice claims in the United States.⁵

Despite diagnostic errors being morbid and sometimes deadly in the hospital,^6,7 little is known about how residents and learners approach diagnostic decision making. Errors in diagnosis are believed to stem from cognitive or system failures,⁸ with errors in cognition believed to occur due to rapid, reflexive thinking operating in the absence of a more analytical, deliberate process. System-based problems (eg, lack of expert availability, technology barriers, and access to data) have also been cited as contributors.⁹ However, whether and how these apply to trainees is not known.

Therefore, we conducted a focused ethnography of inpatient medicine teams (ie, attendings, residents, interns, and medical students) in 2 affiliated teaching hospitals, aiming to (a) observe the process of diagnosis by trainees and (b) identify methods to improve the diagnostic process and prevent errors.

We designed a multimethod, focused ethnographic study to examine diagnostic decision making in hospital settings.^10,11 In contrast to anthropologic ethnographies that study entire fields using open-ended questions, our study was designed to examine the process of diagnosis from the perspective of clinicians engaged in this activity.¹¹ This approach allowed us to capture diagnostic decisions and cognitive and system-based factors in a manner currently lacking in the literature.¹²

Setting and Participants

Between January 2016 and May 2016, we observed the members of four inpatient internal medicine teaching teams at 2 affiliated teaching hospitals. We purposefully selected teaching teams for observation because they are the primary model of care in academic settings and we have expertise in carrying out similar studies.^13,14 Teaching teams typically consisted of a medical attending (senior-level physician), 1 senior resident (a second- or third-year postgraduate trainee), two interns (a trainee in their first postgraduate year), and two to four  medical students. Teams were selected at random using existing schedules and followed Monday to Friday so as to permit observation of work on call and noncall days. Owing to manpower limitations, weekend and night shifts were not observed. However, overnight events were captured during morning rounds.

Most of the teams began rounds at 8:30 AM. Typically, rounds lasted for 90–120 min and concluded with a recap (ie, “running the list”) with a review of explicit plans for patients after they had been evaluated by the attending. This discussion often occurred in the team rooms, with the attending leading the discussion with the trainees.

Data Collection

A multidisciplinary team, including clinicians (eg, physicians, nurses), nonclinicians (eg, qualitative researchers, social scientists), and healthcare engineers, conducted the observations. We observed preround activities of interns and residents before arrival of the attending (7:00 AM - 8:30 AM), followed by morning rounds with the entire team, and afternoon work that included senior residents, interns, and students.

To capture multiple aspects of the diagnostic process, we collected data using field notes modeled on components of the National Academy of Science model for diagnosis (Appendix).^1,15 This model encompasses phases of the diagnostic process (eg, data gathering, integration, formulation of a working diagnosis, treatment delivery, and outcomes) and the work system (team members, organization, technology and tools, physical environment, tasks).

Focus Groups and Interviews

At the end of weekly observations, we conducted focus groups with the residents and one-on- one interviews with the attendings. Focus groups with the residents were conducted to encourage a group discussion about the diagnostic process. Separate interviews with the attendings were performed to ensure that power differentials did not influence discussions. During focus groups, we specifically asked about challenges and possible solutions to improve diagnosis. Experienced qualitative methodologists (J.F., M.H., M.Q.) used semistructured interview guides for discussions (Appendix).

Data Analysis

After aggregating and reading the data, three reviewers (V.C., S.K., S.S.) began inductive analysis by handwriting notes and initial reflective thoughts to create preliminary codes. Multiple team members then reread the original field notes and the focus group/interview data to refine the preliminary codes and develop additional codes. Next, relationships between codes were identified and used to develop key themes. Triangulation of data collected from observations and interview/focus group sessions was carried out to compare data that we surmised with data that were verbalized by the team. The developed themes were discussed as a group to ensure consistency of major findings.

Ethical and Regulatory Oversight

This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).

Four teaching teams (4 attendings, 4 senior residents, 9 interns, and 14 medical students) were observed over 33 distinct shifts and 168 hours. Observations included morning rounds (96 h), postround call days (52 h), and postround non-call days (20 h). Morning rounds lasted an average of 127 min (range: 48-232 min) and included an average of 9 patients (range: 4-16 patients).

Themes Regarding the Diagnostic Process

We identified the following 4 primary themes related to the diagnostic process in teaching hospitals: (1) diagnosis is a social phenomenon; (2) data necessary to make diagnoses are fragmented; (3) distractions undermine the diagnostic process; and (4) time pressures interfere with diagnostic decision making (Appendix Table 1).

(1) Diagnosis is a Social Phenomenon.

Team members viewed the process of diagnosis as a social exchange of facts, findings, and strategies within a defined structure. The opportunity to discuss impressions with others was valued as a means to share, test, and process assumptions.

“Rounds are the most important part of the process. That is where we make most decisions in a collective, collaborative way with the attending present. We bounce ideas off each other.” (Intern)

Typical of social processes, variations based on time of day and schedule were observed. For instance, during call days, learners gathered data and formed working diagnosis and treatment plans with minimal attending interaction. This separation of roles and responsibilities introduced a hierarchy within diagnosis as follows:

“The interns would not call me first; they would talk to the senior resident and then if the senior thought he should chat with me, then they would call. But for the most part, they gather information and come up with the plan.” (Attending).

The work system was suited to facilitate social interactions. For instance, designated rooms (with team members informally assigned to a computer) provided physical proximity of the resident to interns and medical students. In this space, numerous informal discussions between team members (eg, “What do you think about this test?” “I’m not sure what to do about this finding.” “Should I call a [consult] on this patient?”) were observed. Although proximity to each other was viewed as beneficial, dangers to the social nature of diagnosis in the form of anchoring (ie, a cognitive bias where emphasis is placed on the first piece of data)¹⁶ were also mentioned. Similarly, the paradox associated with social proof (ie, the pressure to assume conformity within a group) was also observed as disagreement between team members and attendings rarely occurred during observations.

“I mean, they’re the attending, right? It’s hard to argue with them when they want a test or something done. When I do push back, it’s rare that others will support me–so it’s usually me and the attending.” (Resident)

“I would push back if I think it’s really bad for the patient or could cause harm–but the truth is, it doesn’t happen much.” (Intern)

(2) Data Necessary to Make Diagnoses are Fragmented

Team members universally cited fragmentation in data delivery, retrieval, and processing as a barrier to diagnosis. Team members indicated that test results might not be looked at or acted upon in a timely manner, and participants pointed to the electronic medical record as a source of this challenge.

“Before I knew about [the app for Epic], I would literally sit on the computer to get all the information we would need on rounds. Its key to making decisions. We often say we will do something, only to find the test result doesn’t support it–and then we’re back to square 1.” (Intern)

Information used by teams came from myriad sources (eg, patients, family members, electronic records) and from various settings (eg, emergency department, patient rooms, discussions with consultants). Additionally, test results often appeared without warning. Thus, availability of information was poorly aligned with clinical duties.

“They (the lab) will call us when a blood culture is positive or something is off. That is very helpful but it often comes later in the day, when we’re done with rounds.” (Resident)

The work system was highlighted as a key contributor to data fragmentation. Peculiarities of our electronic medical record (EMR) and how data were collected, stored, or presented were described as “frustrating,” and “unsafe,” by team members. Correspondingly, we frequently observed interns asking for assistance for tasks such as ordering tests or finding information despite being “trained” to use the EMR.

“People have to learn how to filter, how to recognize the most important points and link data streams together in terms of causality. But we assume they know where to find that information. It’s actually a very hard thing to do, for both the house staff and me.” (Attending)

(3) Distractions Undermine the Diagnostic Process

Distractions often created cognitive difficulties. For example, ambient noise and interruptions from neighbors working on other teams were cited as barriers to diagnosis. In addition, we observed several team members using headphones to drown out ambient noise while working on the computer.

“I know I shouldn’t do it (wear headphones), but I have no other way of turning down the noise so I can concentrate.” (Intern)

Similarly, the unpredictable nature and the volume of pages often interrupted thinking about diagnosis.

“Sometimes the pager just goes off all the time and (after making sure its not an urgent issue), I will just ignore it for a bit, especially if I am in the middle of something. It would be great if I could finish my thought process knowing I would not be interrupted.” (Resident)

To mitigate this problem, 1 attending described how he would proactively seek out nurses caring for his patients to “head off” questions (eg, “I will renew the restraints and medications this morning,” and “Is there anything you need in terms of orders for this patient that I can take care of now?”) that might lead to pages. Another resident described his approach as follows:

“I make it a point to tell the nurses where I will be hanging out and where they can find me if they have any questions. I tell them to come talk to me rather than page me since that will be less distracting.” (Resident).

Most of the interns described documentation work such as writing admission and progress notes in negative terms (“an academic exercise,” “part of the billing activity”). However, in the context of interruptions, some described this as helpful.

“The most valuable part of the thinking process was writing the assessment and plan because that’s actually my schema for all problems. It literally is the only time where I can sit and collect my thoughts to formulate a diagnosis and plan.” (Intern)

(4) Time Pressures Interfere With Diagnostic Decision Making

All team members spoke about the challenge of finding time for diagnosis during the workday. Often, they had to skip learning sessions for this purpose.

“They tell us we should go to morning report or noon conference but when I’m running around trying to get things done. I hate having to choose between my education and doing what’s best for the patient–but that’s often what it comes down to.” (Intern)

When specifically asked whether setting aside dedicated time to specifically review and formulate diagnoses would be valuable, respondents were uniformly enthusiastic. Team members described attentional conflicts as being the worst when “cross covering” other teams on call days, as their patient load effectively doubled during this time. Of note, cross-covering occurred when teams were also on call—and thus took them away from important diagnostic activities such as data gathering or synthesis for patients they were admitting.

“If you were to ever design a system where errors were likely–this is how you would design it: take a team with little supervision, double their patient load, keep them busy with new challenging cases and then ask questions about patients they know little about.” (Resident)

Although diagnostic errors have been called “the next frontier for patient safety,”¹⁷ little is known about the process, barriers, and facilitators to diagnosis in teaching hospitals. In this focused ethnography conducted at 2 academic medical centers, we identified multiple cognitive and system-level challenges and potential strategies to improve diagnosis from trainees engaged in this activity. Key themes identified by those we observed included the social nature of diagnosis, fragmented information delivery, constant distractions and interruptions, and time pressures. In turn, these insights allow us to generate strategies that can be applied to improve the diagnostic process in teaching hospitals.

Our study underscores the importance of social interactions in diagnosis. In contrast, most of the interventions to prevent diagnostic errors target individual providers through practices such as metacognition and “thinking about thinking.”^18-20 These interventions are based on Daniel Kahnemann’s work on dual thought process. Type 1 thought processes are fast, subconscious, reflexive, largely intuitive, and more vulnerable to error. In contrast, Type 2 processes are slower, deliberate, analytic, and less prone to error.²¹ Although an individual’s Type 2 thought capacity is limited, a major goal of cognitive interventions is to encourage Type 2 over Type 1 thinking, an approach termed “de-biasing.”^22-24 Unfortunately, cognitive interventions testing such approaches have suffered mixed results–perhaps because of lack of focus on collective wisdom or group thinking, which may be key to diagnosis from our findings.^9,25 In this sense, morning rounds were a social gathering used to strategize and develop care plans, but with limited time to think about diagnosis.²⁶ Introduction of defined periods for individuals to engage in diagnostic activities such as de-biasing (ie, asking “what else could this be)²⁷ before or after rounds may provide an opportunity for reflection and improving diagnosis. In addition, embedding tools such as diagnosis expanders and checklists within these defined time slots^28,29 may prove to be useful in reflecting on diagnosis and preventing diagnostic errors.

An unexpected yet important finding from this study were the challenges posed by distractions and the physical environment. Potentially maladaptive workarounds to these interruptions included use of headphones; more productive strategies included updating nurses with plans to avert pages and creating a list of activities to ensure that key tasks were not forgotten.^30,31 Applying lessons from aviation, a focused effort to limit distractions during key portions of the day, might be worth considering for diagnostic safety.³² Similarly, improving the environment in which diagnosis occurs—including creating spaces that are quiet, orderly, and optimized for thinking—may be valuable.³³Our study has limitations. First, our findings are limited to direct observations; we are thus unable to comment on how unobserved aspects of care (eg, cognitive processes) might have influenced our findings. Our observations of clinical care might also have introduced a Hawthorne effect. However, because we were closely integrated with teams and conducted focus groups to corroborate our assessments, we believe that this was not the case. Second, we did not identify diagnostic errors or link processes we observed to errors. Third, our approach is limited to 2 teaching centers, thereby limiting the generalizability of findings. Relatedly, we were only able to conduct observations during weekdays; differences in weekend and night resources might affect our insights.

The cognitive and system-based barriers faced by clinicians in teaching hospitals suggest that new methods to improve diagnosis are needed. Future interventions such as defined “time-outs” for diagnosis, strategies focused on limiting distractions, and methods to improve communication between team members are novel and have parallels in other industries. As challenges to quantify diagnostic errors abound,³⁴ improving cognitive- and system-based factors via reflection through communication, concentration, and organization is necessary to improve medical decision making in academic medical centers.

Disclosures

None declared for all coauthors.

Funding

This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11-222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.

Diagnostic error—defined as a failure to establish an accurate and timely explanation of the patient’s health problem—is an important source of patient harm.¹ Data suggest that all patients will experience at least 1 diagnostic error in their lifetime.^2-4 Not surprisingly, diagnostic errors are among the leading categories of paid malpractice claims in the United States.⁵

Despite diagnostic errors being morbid and sometimes deadly in the hospital,^6,7 little is known about how residents and learners approach diagnostic decision making. Errors in diagnosis are believed to stem from cognitive or system failures,⁸ with errors in cognition believed to occur due to rapid, reflexive thinking operating in the absence of a more analytical, deliberate process. System-based problems (eg, lack of expert availability, technology barriers, and access to data) have also been cited as contributors.⁹ However, whether and how these apply to trainees is not known.

Therefore, we conducted a focused ethnography of inpatient medicine teams (ie, attendings, residents, interns, and medical students) in 2 affiliated teaching hospitals, aiming to (a) observe the process of diagnosis by trainees and (b) identify methods to improve the diagnostic process and prevent errors.

We designed a multimethod, focused ethnographic study to examine diagnostic decision making in hospital settings.^10,11 In contrast to anthropologic ethnographies that study entire fields using open-ended questions, our study was designed to examine the process of diagnosis from the perspective of clinicians engaged in this activity.¹¹ This approach allowed us to capture diagnostic decisions and cognitive and system-based factors in a manner currently lacking in the literature.¹²

Setting and Participants

Between January 2016 and May 2016, we observed the members of four inpatient internal medicine teaching teams at 2 affiliated teaching hospitals. We purposefully selected teaching teams for observation because they are the primary model of care in academic settings and we have expertise in carrying out similar studies.^13,14 Teaching teams typically consisted of a medical attending (senior-level physician), 1 senior resident (a second- or third-year postgraduate trainee), two interns (a trainee in their first postgraduate year), and two to four  medical students. Teams were selected at random using existing schedules and followed Monday to Friday so as to permit observation of work on call and noncall days. Owing to manpower limitations, weekend and night shifts were not observed. However, overnight events were captured during morning rounds.

Most of the teams began rounds at 8:30 AM. Typically, rounds lasted for 90–120 min and concluded with a recap (ie, “running the list”) with a review of explicit plans for patients after they had been evaluated by the attending. This discussion often occurred in the team rooms, with the attending leading the discussion with the trainees.

Data Collection

A multidisciplinary team, including clinicians (eg, physicians, nurses), nonclinicians (eg, qualitative researchers, social scientists), and healthcare engineers, conducted the observations. We observed preround activities of interns and residents before arrival of the attending (7:00 AM - 8:30 AM), followed by morning rounds with the entire team, and afternoon work that included senior residents, interns, and students.

To capture multiple aspects of the diagnostic process, we collected data using field notes modeled on components of the National Academy of Science model for diagnosis (Appendix).^1,15 This model encompasses phases of the diagnostic process (eg, data gathering, integration, formulation of a working diagnosis, treatment delivery, and outcomes) and the work system (team members, organization, technology and tools, physical environment, tasks).

Focus Groups and Interviews

At the end of weekly observations, we conducted focus groups with the residents and one-on- one interviews with the attendings. Focus groups with the residents were conducted to encourage a group discussion about the diagnostic process. Separate interviews with the attendings were performed to ensure that power differentials did not influence discussions. During focus groups, we specifically asked about challenges and possible solutions to improve diagnosis. Experienced qualitative methodologists (J.F., M.H., M.Q.) used semistructured interview guides for discussions (Appendix).

Data Analysis

After aggregating and reading the data, three reviewers (V.C., S.K., S.S.) began inductive analysis by handwriting notes and initial reflective thoughts to create preliminary codes. Multiple team members then reread the original field notes and the focus group/interview data to refine the preliminary codes and develop additional codes. Next, relationships between codes were identified and used to develop key themes. Triangulation of data collected from observations and interview/focus group sessions was carried out to compare data that we surmised with data that were verbalized by the team. The developed themes were discussed as a group to ensure consistency of major findings.

Ethical and Regulatory Oversight

This study was reviewed and approved by the Institutional Review Boards at the University of Michigan Health System (HUM-00106657) and the VA Ann Arbor Healthcare System (1-2016-010040).

Four teaching teams (4 attendings, 4 senior residents, 9 interns, and 14 medical students) were observed over 33 distinct shifts and 168 hours. Observations included morning rounds (96 h), postround call days (52 h), and postround non-call days (20 h). Morning rounds lasted an average of 127 min (range: 48-232 min) and included an average of 9 patients (range: 4-16 patients).

Themes Regarding the Diagnostic Process

We identified the following 4 primary themes related to the diagnostic process in teaching hospitals: (1) diagnosis is a social phenomenon; (2) data necessary to make diagnoses are fragmented; (3) distractions undermine the diagnostic process; and (4) time pressures interfere with diagnostic decision making (Appendix Table 1).

(1) Diagnosis is a Social Phenomenon.

Team members viewed the process of diagnosis as a social exchange of facts, findings, and strategies within a defined structure. The opportunity to discuss impressions with others was valued as a means to share, test, and process assumptions.

“Rounds are the most important part of the process. That is where we make most decisions in a collective, collaborative way with the attending present. We bounce ideas off each other.” (Intern)

Typical of social processes, variations based on time of day and schedule were observed. For instance, during call days, learners gathered data and formed working diagnosis and treatment plans with minimal attending interaction. This separation of roles and responsibilities introduced a hierarchy within diagnosis as follows:

“The interns would not call me first; they would talk to the senior resident and then if the senior thought he should chat with me, then they would call. But for the most part, they gather information and come up with the plan.” (Attending).

The work system was suited to facilitate social interactions. For instance, designated rooms (with team members informally assigned to a computer) provided physical proximity of the resident to interns and medical students. In this space, numerous informal discussions between team members (eg, “What do you think about this test?” “I’m not sure what to do about this finding.” “Should I call a [consult] on this patient?”) were observed. Although proximity to each other was viewed as beneficial, dangers to the social nature of diagnosis in the form of anchoring (ie, a cognitive bias where emphasis is placed on the first piece of data)¹⁶ were also mentioned. Similarly, the paradox associated with social proof (ie, the pressure to assume conformity within a group) was also observed as disagreement between team members and attendings rarely occurred during observations.

“I mean, they’re the attending, right? It’s hard to argue with them when they want a test or something done. When I do push back, it’s rare that others will support me–so it’s usually me and the attending.” (Resident)

“I would push back if I think it’s really bad for the patient or could cause harm–but the truth is, it doesn’t happen much.” (Intern)

(2) Data Necessary to Make Diagnoses are Fragmented

Team members universally cited fragmentation in data delivery, retrieval, and processing as a barrier to diagnosis. Team members indicated that test results might not be looked at or acted upon in a timely manner, and participants pointed to the electronic medical record as a source of this challenge.

“Before I knew about [the app for Epic], I would literally sit on the computer to get all the information we would need on rounds. Its key to making decisions. We often say we will do something, only to find the test result doesn’t support it–and then we’re back to square 1.” (Intern)

Information used by teams came from myriad sources (eg, patients, family members, electronic records) and from various settings (eg, emergency department, patient rooms, discussions with consultants). Additionally, test results often appeared without warning. Thus, availability of information was poorly aligned with clinical duties.

“They (the lab) will call us when a blood culture is positive or something is off. That is very helpful but it often comes later in the day, when we’re done with rounds.” (Resident)

The work system was highlighted as a key contributor to data fragmentation. Peculiarities of our electronic medical record (EMR) and how data were collected, stored, or presented were described as “frustrating,” and “unsafe,” by team members. Correspondingly, we frequently observed interns asking for assistance for tasks such as ordering tests or finding information despite being “trained” to use the EMR.

“People have to learn how to filter, how to recognize the most important points and link data streams together in terms of causality. But we assume they know where to find that information. It’s actually a very hard thing to do, for both the house staff and me.” (Attending)

(3) Distractions Undermine the Diagnostic Process

Distractions often created cognitive difficulties. For example, ambient noise and interruptions from neighbors working on other teams were cited as barriers to diagnosis. In addition, we observed several team members using headphones to drown out ambient noise while working on the computer.

“I know I shouldn’t do it (wear headphones), but I have no other way of turning down the noise so I can concentrate.” (Intern)

Similarly, the unpredictable nature and the volume of pages often interrupted thinking about diagnosis.

“Sometimes the pager just goes off all the time and (after making sure its not an urgent issue), I will just ignore it for a bit, especially if I am in the middle of something. It would be great if I could finish my thought process knowing I would not be interrupted.” (Resident)

To mitigate this problem, 1 attending described how he would proactively seek out nurses caring for his patients to “head off” questions (eg, “I will renew the restraints and medications this morning,” and “Is there anything you need in terms of orders for this patient that I can take care of now?”) that might lead to pages. Another resident described his approach as follows:

“I make it a point to tell the nurses where I will be hanging out and where they can find me if they have any questions. I tell them to come talk to me rather than page me since that will be less distracting.” (Resident).

Most of the interns described documentation work such as writing admission and progress notes in negative terms (“an academic exercise,” “part of the billing activity”). However, in the context of interruptions, some described this as helpful.

“The most valuable part of the thinking process was writing the assessment and plan because that’s actually my schema for all problems. It literally is the only time where I can sit and collect my thoughts to formulate a diagnosis and plan.” (Intern)

(4) Time Pressures Interfere With Diagnostic Decision Making

All team members spoke about the challenge of finding time for diagnosis during the workday. Often, they had to skip learning sessions for this purpose.

“They tell us we should go to morning report or noon conference but when I’m running around trying to get things done. I hate having to choose between my education and doing what’s best for the patient–but that’s often what it comes down to.” (Intern)

When specifically asked whether setting aside dedicated time to specifically review and formulate diagnoses would be valuable, respondents were uniformly enthusiastic. Team members described attentional conflicts as being the worst when “cross covering” other teams on call days, as their patient load effectively doubled during this time. Of note, cross-covering occurred when teams were also on call—and thus took them away from important diagnostic activities such as data gathering or synthesis for patients they were admitting.

“If you were to ever design a system where errors were likely–this is how you would design it: take a team with little supervision, double their patient load, keep them busy with new challenging cases and then ask questions about patients they know little about.” (Resident)

Although diagnostic errors have been called “the next frontier for patient safety,”¹⁷ little is known about the process, barriers, and facilitators to diagnosis in teaching hospitals. In this focused ethnography conducted at 2 academic medical centers, we identified multiple cognitive and system-level challenges and potential strategies to improve diagnosis from trainees engaged in this activity. Key themes identified by those we observed included the social nature of diagnosis, fragmented information delivery, constant distractions and interruptions, and time pressures. In turn, these insights allow us to generate strategies that can be applied to improve the diagnostic process in teaching hospitals.

Our study underscores the importance of social interactions in diagnosis. In contrast, most of the interventions to prevent diagnostic errors target individual providers through practices such as metacognition and “thinking about thinking.”^18-20 These interventions are based on Daniel Kahnemann’s work on dual thought process. Type 1 thought processes are fast, subconscious, reflexive, largely intuitive, and more vulnerable to error. In contrast, Type 2 processes are slower, deliberate, analytic, and less prone to error.²¹ Although an individual’s Type 2 thought capacity is limited, a major goal of cognitive interventions is to encourage Type 2 over Type 1 thinking, an approach termed “de-biasing.”^22-24 Unfortunately, cognitive interventions testing such approaches have suffered mixed results–perhaps because of lack of focus on collective wisdom or group thinking, which may be key to diagnosis from our findings.^9,25 In this sense, morning rounds were a social gathering used to strategize and develop care plans, but with limited time to think about diagnosis.²⁶ Introduction of defined periods for individuals to engage in diagnostic activities such as de-biasing (ie, asking “what else could this be)²⁷ before or after rounds may provide an opportunity for reflection and improving diagnosis. In addition, embedding tools such as diagnosis expanders and checklists within these defined time slots^28,29 may prove to be useful in reflecting on diagnosis and preventing diagnostic errors.

An unexpected yet important finding from this study were the challenges posed by distractions and the physical environment. Potentially maladaptive workarounds to these interruptions included use of headphones; more productive strategies included updating nurses with plans to avert pages and creating a list of activities to ensure that key tasks were not forgotten.^30,31 Applying lessons from aviation, a focused effort to limit distractions during key portions of the day, might be worth considering for diagnostic safety.³² Similarly, improving the environment in which diagnosis occurs—including creating spaces that are quiet, orderly, and optimized for thinking—may be valuable.³³Our study has limitations. First, our findings are limited to direct observations; we are thus unable to comment on how unobserved aspects of care (eg, cognitive processes) might have influenced our findings. Our observations of clinical care might also have introduced a Hawthorne effect. However, because we were closely integrated with teams and conducted focus groups to corroborate our assessments, we believe that this was not the case. Second, we did not identify diagnostic errors or link processes we observed to errors. Third, our approach is limited to 2 teaching centers, thereby limiting the generalizability of findings. Relatedly, we were only able to conduct observations during weekdays; differences in weekend and night resources might affect our insights.

The cognitive and system-based barriers faced by clinicians in teaching hospitals suggest that new methods to improve diagnosis are needed. Future interventions such as defined “time-outs” for diagnosis, strategies focused on limiting distractions, and methods to improve communication between team members are novel and have parallels in other industries. As challenges to quantify diagnostic errors abound,³⁴ improving cognitive- and system-based factors via reflection through communication, concentration, and organization is necessary to improve medical decision making in academic medical centers.

Disclosures

None declared for all coauthors.

Funding

This project was supported by grant number P30HS024385 from the Agency for Healthcare Research and Quality. The funding source played no role in study design, data acquisition, analysis or decision to report these data. Dr. Chopra is supported by a career development award from the Agency of Healthcare Research and Quality (1-K08-HS022835-01). Dr. Krein is supported by a VA Health Services Research and Development Research Career Scientist Award (RCS 11-222). Dr. Singh is partially supported by Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the Department of Veterans Affairs.