Teaching Evidence-Based Dermatology Using a Web-Based Journal Club: A Pilot Study and Survey

To the Editor:

With a steady increase in dermatology publications over recent decades, there is an expanding pool of evidence to address clinical questions.¹ Residency training is the time when appraising the medical literature and practicing evidence-based medicine is most honed. Evidence-based medicine is an essential component of Practice-based Learning and Improvement, a required core competency of the Accreditation Council for Graduate Medical Education.² Assimilation of new research evidence is traditionally taught through didactics and journal club discussions in residency.

However, at a time when the demand for information overwhelms safeguards that exist to evaluate its quality, it is more important than ever to be equipped with the proper tools to critically appraise novel literature. Beyond accepting a scientific article at face value, physicians must learn to ask targeted questions of the study design, results, and clinical relevance. These questions change based on the type of study, and organizations such as the Oxford Centre for Evidence-Based Medicine provide guidance through critical appraisal worksheets.³

To investigate the utility of using guided questions to evaluate the reliability, significance, and applicability of clinical evidence, we beta tested a novel web-based application in an academic dermatology setting to design and run a journal club for residents. Six dermatology residents participated in this institutional review board–approved study comprised of 3 phases: (1) independent article appraisal through the web-based application, (2) group discussion, and (3) anonymous postsurvey.

Using this platform, we uploaded a recent article into the interactive reader, which contained an integrated tool for appraisal based on specific questions. Because the article described the results of a randomized clinical trial, we used questions from the Centre for Evidence-Based Medicine’s Randomised Controlled Trials Critical Appraisal Worksheet, which has a series of questions to evaluate internal validity, results, and external validity and applicability.³

Residents used the platform to independently read the article, highlight areas of the text that corresponded to 8 critical appraisal questions, and answer yes or no to these questions. Based on residents’ answers, a final appraisal score (on a scale of 1% to 100%) was generated. Simultaneously, the attending dermatologist leading the journal club (C.W.) also completed the assignment to establish an expert score.

Scores from the residents’ independent appraisal ranged from 75% to 100% (mean, 85.4%). Upon discussing the article in a group setting, the residents established a consensus score of 75%. This consensus score matched the expert score, which suggested to us that both independently reviewing the article using guided questions and conducting a group debriefing were necessary to match the expert level of critical appraisal.

Of note, the residents’ average independent appraisal score was higher than both the consensus and expert scores, indicating that the residents evaluated the article less critically on their own. With more practice using this method, it is possible that the precision and accuracy of the residents’ critical appraisal of scientific articles will improve.

In the postsurvey, we asked residents about the critical appraisal of the medical literature. All residents agreed that evaluating the quality of evidence when reading a scientific article was somewhat important or very important to them; however, only 2 of 6 evaluated the quality of evidence all the time, and the other 4 did so half of the time or less than half of the time.

When critically appraising articles, 2 of 6 residents used specific rubrics half of the time; 4 of 6 less than half of the time. Most important, 5 of 6 residents agreed that the quality of evidence affected their management decisions more than half of the time or all of the time. Although it is clear that residents value evidence-based medicine and understand the importance of evaluating the quality of evidence, doing so currently might not be simple or practical.

An organized framework for appraising articles would streamline the process. Five of 6 residents agreed that the use of specific questions as a guide made it easier to appraise an article for the quality of its evidence. Four of 6 residents found that juxtaposing specific questions with the interactive reader was helpful; 5 of 6 agreed that they would use a web-based journal club platform if given the option.

Lastly, 5 of 6 residents agreed that if such a tool were available, a platform containing all major dermatology publications in an interactive reader format, along with relevant appraisal questions on the side, would be useful.

This pilot study augmented the typical journal club experience by emphasizing goal-directed reading and the importance of analyzing the quality of evidence. The combination of independent appraisal of an article using targeted questions and a group debrief led to better understanding of the evidence and its clinical applicability. The COVID-19 pandemic may be a better time than ever to explore innovative ways to teach evidence-based medicine in residency training.

To the Editor:

With a steady increase in dermatology publications over recent decades, there is an expanding pool of evidence to address clinical questions.¹ Residency training is the time when appraising the medical literature and practicing evidence-based medicine is most honed. Evidence-based medicine is an essential component of Practice-based Learning and Improvement, a required core competency of the Accreditation Council for Graduate Medical Education.² Assimilation of new research evidence is traditionally taught through didactics and journal club discussions in residency.

However, at a time when the demand for information overwhelms safeguards that exist to evaluate its quality, it is more important than ever to be equipped with the proper tools to critically appraise novel literature. Beyond accepting a scientific article at face value, physicians must learn to ask targeted questions of the study design, results, and clinical relevance. These questions change based on the type of study, and organizations such as the Oxford Centre for Evidence-Based Medicine provide guidance through critical appraisal worksheets.³

To investigate the utility of using guided questions to evaluate the reliability, significance, and applicability of clinical evidence, we beta tested a novel web-based application in an academic dermatology setting to design and run a journal club for residents. Six dermatology residents participated in this institutional review board–approved study comprised of 3 phases: (1) independent article appraisal through the web-based application, (2) group discussion, and (3) anonymous postsurvey.

Using this platform, we uploaded a recent article into the interactive reader, which contained an integrated tool for appraisal based on specific questions. Because the article described the results of a randomized clinical trial, we used questions from the Centre for Evidence-Based Medicine’s Randomised Controlled Trials Critical Appraisal Worksheet, which has a series of questions to evaluate internal validity, results, and external validity and applicability.³

Residents used the platform to independently read the article, highlight areas of the text that corresponded to 8 critical appraisal questions, and answer yes or no to these questions. Based on residents’ answers, a final appraisal score (on a scale of 1% to 100%) was generated. Simultaneously, the attending dermatologist leading the journal club (C.W.) also completed the assignment to establish an expert score.

Scores from the residents’ independent appraisal ranged from 75% to 100% (mean, 85.4%). Upon discussing the article in a group setting, the residents established a consensus score of 75%. This consensus score matched the expert score, which suggested to us that both independently reviewing the article using guided questions and conducting a group debriefing were necessary to match the expert level of critical appraisal.

Of note, the residents’ average independent appraisal score was higher than both the consensus and expert scores, indicating that the residents evaluated the article less critically on their own. With more practice using this method, it is possible that the precision and accuracy of the residents’ critical appraisal of scientific articles will improve.

In the postsurvey, we asked residents about the critical appraisal of the medical literature. All residents agreed that evaluating the quality of evidence when reading a scientific article was somewhat important or very important to them; however, only 2 of 6 evaluated the quality of evidence all the time, and the other 4 did so half of the time or less than half of the time.

When critically appraising articles, 2 of 6 residents used specific rubrics half of the time; 4 of 6 less than half of the time. Most important, 5 of 6 residents agreed that the quality of evidence affected their management decisions more than half of the time or all of the time. Although it is clear that residents value evidence-based medicine and understand the importance of evaluating the quality of evidence, doing so currently might not be simple or practical.

An organized framework for appraising articles would streamline the process. Five of 6 residents agreed that the use of specific questions as a guide made it easier to appraise an article for the quality of its evidence. Four of 6 residents found that juxtaposing specific questions with the interactive reader was helpful; 5 of 6 agreed that they would use a web-based journal club platform if given the option.

Lastly, 5 of 6 residents agreed that if such a tool were available, a platform containing all major dermatology publications in an interactive reader format, along with relevant appraisal questions on the side, would be useful.

This pilot study augmented the typical journal club experience by emphasizing goal-directed reading and the importance of analyzing the quality of evidence. The combination of independent appraisal of an article using targeted questions and a group debrief led to better understanding of the evidence and its clinical applicability. The COVID-19 pandemic may be a better time than ever to explore innovative ways to teach evidence-based medicine in residency training.

To the Editor:

With a steady increase in dermatology publications over recent decades, there is an expanding pool of evidence to address clinical questions.¹ Residency training is the time when appraising the medical literature and practicing evidence-based medicine is most honed. Evidence-based medicine is an essential component of Practice-based Learning and Improvement, a required core competency of the Accreditation Council for Graduate Medical Education.² Assimilation of new research evidence is traditionally taught through didactics and journal club discussions in residency.

However, at a time when the demand for information overwhelms safeguards that exist to evaluate its quality, it is more important than ever to be equipped with the proper tools to critically appraise novel literature. Beyond accepting a scientific article at face value, physicians must learn to ask targeted questions of the study design, results, and clinical relevance. These questions change based on the type of study, and organizations such as the Oxford Centre for Evidence-Based Medicine provide guidance through critical appraisal worksheets.³

To investigate the utility of using guided questions to evaluate the reliability, significance, and applicability of clinical evidence, we beta tested a novel web-based application in an academic dermatology setting to design and run a journal club for residents. Six dermatology residents participated in this institutional review board–approved study comprised of 3 phases: (1) independent article appraisal through the web-based application, (2) group discussion, and (3) anonymous postsurvey.

Using this platform, we uploaded a recent article into the interactive reader, which contained an integrated tool for appraisal based on specific questions. Because the article described the results of a randomized clinical trial, we used questions from the Centre for Evidence-Based Medicine’s Randomised Controlled Trials Critical Appraisal Worksheet, which has a series of questions to evaluate internal validity, results, and external validity and applicability.³

Residents used the platform to independently read the article, highlight areas of the text that corresponded to 8 critical appraisal questions, and answer yes or no to these questions. Based on residents’ answers, a final appraisal score (on a scale of 1% to 100%) was generated. Simultaneously, the attending dermatologist leading the journal club (C.W.) also completed the assignment to establish an expert score.

Scores from the residents’ independent appraisal ranged from 75% to 100% (mean, 85.4%). Upon discussing the article in a group setting, the residents established a consensus score of 75%. This consensus score matched the expert score, which suggested to us that both independently reviewing the article using guided questions and conducting a group debriefing were necessary to match the expert level of critical appraisal.

Of note, the residents’ average independent appraisal score was higher than both the consensus and expert scores, indicating that the residents evaluated the article less critically on their own. With more practice using this method, it is possible that the precision and accuracy of the residents’ critical appraisal of scientific articles will improve.

In the postsurvey, we asked residents about the critical appraisal of the medical literature. All residents agreed that evaluating the quality of evidence when reading a scientific article was somewhat important or very important to them; however, only 2 of 6 evaluated the quality of evidence all the time, and the other 4 did so half of the time or less than half of the time.

When critically appraising articles, 2 of 6 residents used specific rubrics half of the time; 4 of 6 less than half of the time. Most important, 5 of 6 residents agreed that the quality of evidence affected their management decisions more than half of the time or all of the time. Although it is clear that residents value evidence-based medicine and understand the importance of evaluating the quality of evidence, doing so currently might not be simple or practical.

An organized framework for appraising articles would streamline the process. Five of 6 residents agreed that the use of specific questions as a guide made it easier to appraise an article for the quality of its evidence. Four of 6 residents found that juxtaposing specific questions with the interactive reader was helpful; 5 of 6 agreed that they would use a web-based journal club platform if given the option.

Lastly, 5 of 6 residents agreed that if such a tool were available, a platform containing all major dermatology publications in an interactive reader format, along with relevant appraisal questions on the side, would be useful.

This pilot study augmented the typical journal club experience by emphasizing goal-directed reading and the importance of analyzing the quality of evidence. The combination of independent appraisal of an article using targeted questions and a group debrief led to better understanding of the evidence and its clinical applicability. The COVID-19 pandemic may be a better time than ever to explore innovative ways to teach evidence-based medicine in residency training.