Charlie M. Wray, DO, MS

Social determinants of health (SDoH) are social, economic, environmental, and occupational factors that are known to influence an individual’s health care utilization and clinical outcomes.^1,2 Because the Veterans Health Administration (VHA) is charged to address both the medical and nonmedical needs of the veteran population, it is increasingly interested in the impact SDoH have on veteran care.^3,4 To combat the adverse impact of such factors, the VHA has implemented several large-scale programs across the US that focus on prevalent SDoH, such as homelessness, substance abuse, and alcohol use disorders.^5,6 While such risk factors are generally universal in their distribution, variation across regions, between urban and rural spaces, and even within cities has been shown to exist in private settings.⁷ Understanding such variability potentially could be helpful to US Department of Veterans Affairs (VA) policymakers and leaders to better allocate funding and resources to address such issues.

Although previous work has highlighted regional and neighborhood-level variability of SDoH, no study has examined the facility-level variability of commonly encountered social risk factors within the VHA.^4,8 The aim of this study was to describe the interfacility variation of 5 common SDoH known to influence health and health outcomes among a national cohort of veterans hospitalized for common medical issues by using administrative data.

Methods

We used a national cohort of veterans aged ≥ 65 years who were hospitalized at a VHA acute care facility with a primary discharge diagnosis of acute myocardial infarction (AMI), heart failure (HF), or pneumonia in 2012. These conditions were chosen because they are publicly reported and frequently used for interfacility comparison.

Using the International Classification of Diseases–9th Revision (ICD-9) and VHA clinical stop codes, we calculated the median documented proportion of patients with any of the following 5 SDoH: lived alone, marginal housing, alcohol use disorder, substance use disorder, and use of substance use services for patients presenting with HF, MI, and pneumonia (Table). These SDoH were chosen because they are intervenable risk factors for which the VHA has several programs (eg, homeless outreach, substance abuse, and tobacco cessation). To examine the variability of these SDoH across VHA facilities, we determined the number of hospitals that had a sufficient number of admissions (≥ 50) to be included in the analyses. We then examined the administratively documented, facility-level variation in the proportion of individuals with any of the 5 SDoH administrative codes and examined the distribution of their use across all qualifying facilities.

Results

In total, 21,991 patients with either HF (9,853), pneumonia (9,362), or AMI (2,776) were identified across 91 VHA facilities. The majority were male (98%) and had a median (SD) age of 77.0 (9.0) years. The median facility-level proportion of veterans who had any of the SDoH risk factors extracted through administrative codes was low across all conditions, ranging from 0.5 to 2.2%. The most prevalent factors among patients admitted for HF, AMI, and pneumonia were lives alone (2.0% [Interquartile range (IQR), 1.0-5.2], 1.4% [IQR, 0-3.4], and 1.9% [IQR, 0.7-5.4]), substance use disorder (1.2% [IQR, 0-2.2], 1.6% [IQR: 0-3.0], and 1.3% [IQR, 0-2.2] and use of substance use services (0.9% [IQR, 0-1.6%], 1.0% [IQR, 0-1.7%], and 1.6% [IQR, 0-2.2%], respectively [Table]).

When utilizing the NLP algorithm, the documented prevalence of lives alone in the free text of the medical record was higher than administrative coding across all conditions (12.3% vs. 2.2%; P < .01). Among each of the 3 assessed conditions, HF (14.4% vs 2.0%, P < .01) had higher levels of lives alone compared with pneumonia (11% vs 1.9%, P < .01), and AMI (10.2% vs 1.4%, P < .01) when using the NLP algorithm. When we examined the documented facility-level variation in the proportion of individuals with any of the 5 SDoH administrative codes or NLP, we found large variability across all facilities—regardless of extraction method (Figure).

Discussion

While SDoH are known to impact health outcomes, the presence of these risk factors in administrative data among individuals hospitalized for common medical issues is low and variable across VHA facilities. Understanding the documented, facility-level variability of these measures may assist the VHA in determining how it invests time and resources—as different facilities may disproportionately serve a higher number of vulnerable individuals. Beyond the VHA, these findings have generalizable lessons for the US health care system, which has come to recognize how these risk factors impact patients’ health.¹⁰

Although the proportion of individuals with any of the assessed SDoH identified by administrative data was low, our findings are in line with recent studies that showed other risk factors such as social isolation (0.65%), housing issues (0.19%), and financial strain (0.07%) had similarly low prevalence.^8,11 Although the exact prevalence of such factors remains unclear, these findings highlight that SDoH do not appear to be well documented in administrative data. Low coding rates are likely due to the fact that SDoH administrative codes are not tied to financial reimbursement—thus not incentivizing their use by clinicians or hospital systems.

In 2014, an Institute of Medicine report suggested that collection of SDoH in electronic health data as a means to better empower clinicians and health care systems to address social disparities and further support research in SDoH.¹² Since then, data collection using SDoH screening tools has become more common across settings, but is not consistently translated to standardized data due to lack of industry consensus and technical barriers.¹³ To improve this process, the Centers for Medicare and Medicaid Services created “z-codes” for the ICD-10 classification system—a subset of codes that are meant to better capture patients’ underlying social risk.¹⁴ It remains to be seen if such administrative codes have improved the documentation of SDoH.

As health care systems have grown to understand the impact of SDoH on health outcomes,other means of collecting these data have evolved.^1,10 For example, NLP-based extraction methods and electronic screening tools have been proposed and utilized as alternative for obtaining this information. Our findings suggest that some of these measures (eg, lives alone) often may be documented as part of routine care in the electronic health record, thus highlighting NLP as a tool to obtain such data. However, other studies using NLP technology to extract SDoH have shown this technology is often complicated by quality issues (ie, missing data), complex methods, and poor integration with current information technology infrastructures—thus limiting its use in health care delivery.^15-18

While variance among SDoH across a national health care system is natural, it remains an important systems-level characteristic that health care leaders and policymakers should appreciate. As health care systems disperse financial resources and initiate quality improvement initiatives to address SDoH, knowing that not all facilities are equally affected by SDoH should impact allocation of such resources and energies. Although previous work has highlighted regional and neighborhood levels of variation within the VHA and other health care systems, to our knowledge, this is the first study to examine variability at the facility-level within the VHA.^2,4,13,19

Limitations

There are several limitations to this study. First, though our findings are in line with previous data in other health care systems, generalizability beyond the VA, which primarily cares for older, male patients, may be limited.⁸ Though, as the nation’s largest health care system, lessons from the VHA can still be useful for other health care systems as they consider SDoH variation. Second, among the many SDoH previously identified to impact health, our analysis only focused on 5 such variables. Administrative and medical record documentation of other SDoH may be more common and less variable across institutions. Third, while our data suggests facility-level variation in these measures, this may be in part related to variation in coding across facilities. However, the single SDoH variable extracted using NLP also varied at the facility-level, suggesting that coding may not entirely drive the variation observed.

Conclusions

As US health care systems continue to address SDoH, our findings highlight the various challenges in obtaining accurate data on a patient’s social risk. Moreover, these findings highlight the large variability that exists among institutions in a national integrated health care system. Future work should explore the prevalence and variance of other SDoH as a means to help guide resource allocation and prioritize spending to better address SDoH where it is most needed.

Acknowledgments

This work was supported by NHLBI R01 RO1 HL116522-01A1. Support for VA/CMS data is provided by the US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, VA Information Resource Center (Project Numbers SDR 02-237 and 98-004).

Social determinants of health (SDoH) are social, economic, environmental, and occupational factors that are known to influence an individual’s health care utilization and clinical outcomes.^1,2 Because the Veterans Health Administration (VHA) is charged to address both the medical and nonmedical needs of the veteran population, it is increasingly interested in the impact SDoH have on veteran care.^3,4 To combat the adverse impact of such factors, the VHA has implemented several large-scale programs across the US that focus on prevalent SDoH, such as homelessness, substance abuse, and alcohol use disorders.^5,6 While such risk factors are generally universal in their distribution, variation across regions, between urban and rural spaces, and even within cities has been shown to exist in private settings.⁷ Understanding such variability potentially could be helpful to US Department of Veterans Affairs (VA) policymakers and leaders to better allocate funding and resources to address such issues.

Although previous work has highlighted regional and neighborhood-level variability of SDoH, no study has examined the facility-level variability of commonly encountered social risk factors within the VHA.^4,8 The aim of this study was to describe the interfacility variation of 5 common SDoH known to influence health and health outcomes among a national cohort of veterans hospitalized for common medical issues by using administrative data.

Methods

We used a national cohort of veterans aged ≥ 65 years who were hospitalized at a VHA acute care facility with a primary discharge diagnosis of acute myocardial infarction (AMI), heart failure (HF), or pneumonia in 2012. These conditions were chosen because they are publicly reported and frequently used for interfacility comparison.

Using the International Classification of Diseases–9th Revision (ICD-9) and VHA clinical stop codes, we calculated the median documented proportion of patients with any of the following 5 SDoH: lived alone, marginal housing, alcohol use disorder, substance use disorder, and use of substance use services for patients presenting with HF, MI, and pneumonia (Table). These SDoH were chosen because they are intervenable risk factors for which the VHA has several programs (eg, homeless outreach, substance abuse, and tobacco cessation). To examine the variability of these SDoH across VHA facilities, we determined the number of hospitals that had a sufficient number of admissions (≥ 50) to be included in the analyses. We then examined the administratively documented, facility-level variation in the proportion of individuals with any of the 5 SDoH administrative codes and examined the distribution of their use across all qualifying facilities.

Results

In total, 21,991 patients with either HF (9,853), pneumonia (9,362), or AMI (2,776) were identified across 91 VHA facilities. The majority were male (98%) and had a median (SD) age of 77.0 (9.0) years. The median facility-level proportion of veterans who had any of the SDoH risk factors extracted through administrative codes was low across all conditions, ranging from 0.5 to 2.2%. The most prevalent factors among patients admitted for HF, AMI, and pneumonia were lives alone (2.0% [Interquartile range (IQR), 1.0-5.2], 1.4% [IQR, 0-3.4], and 1.9% [IQR, 0.7-5.4]), substance use disorder (1.2% [IQR, 0-2.2], 1.6% [IQR: 0-3.0], and 1.3% [IQR, 0-2.2] and use of substance use services (0.9% [IQR, 0-1.6%], 1.0% [IQR, 0-1.7%], and 1.6% [IQR, 0-2.2%], respectively [Table]).

When utilizing the NLP algorithm, the documented prevalence of lives alone in the free text of the medical record was higher than administrative coding across all conditions (12.3% vs. 2.2%; P < .01). Among each of the 3 assessed conditions, HF (14.4% vs 2.0%, P < .01) had higher levels of lives alone compared with pneumonia (11% vs 1.9%, P < .01), and AMI (10.2% vs 1.4%, P < .01) when using the NLP algorithm. When we examined the documented facility-level variation in the proportion of individuals with any of the 5 SDoH administrative codes or NLP, we found large variability across all facilities—regardless of extraction method (Figure).

Discussion

While SDoH are known to impact health outcomes, the presence of these risk factors in administrative data among individuals hospitalized for common medical issues is low and variable across VHA facilities. Understanding the documented, facility-level variability of these measures may assist the VHA in determining how it invests time and resources—as different facilities may disproportionately serve a higher number of vulnerable individuals. Beyond the VHA, these findings have generalizable lessons for the US health care system, which has come to recognize how these risk factors impact patients’ health.¹⁰

Although the proportion of individuals with any of the assessed SDoH identified by administrative data was low, our findings are in line with recent studies that showed other risk factors such as social isolation (0.65%), housing issues (0.19%), and financial strain (0.07%) had similarly low prevalence.^8,11 Although the exact prevalence of such factors remains unclear, these findings highlight that SDoH do not appear to be well documented in administrative data. Low coding rates are likely due to the fact that SDoH administrative codes are not tied to financial reimbursement—thus not incentivizing their use by clinicians or hospital systems.

In 2014, an Institute of Medicine report suggested that collection of SDoH in electronic health data as a means to better empower clinicians and health care systems to address social disparities and further support research in SDoH.¹² Since then, data collection using SDoH screening tools has become more common across settings, but is not consistently translated to standardized data due to lack of industry consensus and technical barriers.¹³ To improve this process, the Centers for Medicare and Medicaid Services created “z-codes” for the ICD-10 classification system—a subset of codes that are meant to better capture patients’ underlying social risk.¹⁴ It remains to be seen if such administrative codes have improved the documentation of SDoH.

As health care systems have grown to understand the impact of SDoH on health outcomes,other means of collecting these data have evolved.^1,10 For example, NLP-based extraction methods and electronic screening tools have been proposed and utilized as alternative for obtaining this information. Our findings suggest that some of these measures (eg, lives alone) often may be documented as part of routine care in the electronic health record, thus highlighting NLP as a tool to obtain such data. However, other studies using NLP technology to extract SDoH have shown this technology is often complicated by quality issues (ie, missing data), complex methods, and poor integration with current information technology infrastructures—thus limiting its use in health care delivery.^15-18

While variance among SDoH across a national health care system is natural, it remains an important systems-level characteristic that health care leaders and policymakers should appreciate. As health care systems disperse financial resources and initiate quality improvement initiatives to address SDoH, knowing that not all facilities are equally affected by SDoH should impact allocation of such resources and energies. Although previous work has highlighted regional and neighborhood levels of variation within the VHA and other health care systems, to our knowledge, this is the first study to examine variability at the facility-level within the VHA.^2,4,13,19

Limitations

There are several limitations to this study. First, though our findings are in line with previous data in other health care systems, generalizability beyond the VA, which primarily cares for older, male patients, may be limited.⁸ Though, as the nation’s largest health care system, lessons from the VHA can still be useful for other health care systems as they consider SDoH variation. Second, among the many SDoH previously identified to impact health, our analysis only focused on 5 such variables. Administrative and medical record documentation of other SDoH may be more common and less variable across institutions. Third, while our data suggests facility-level variation in these measures, this may be in part related to variation in coding across facilities. However, the single SDoH variable extracted using NLP also varied at the facility-level, suggesting that coding may not entirely drive the variation observed.

Conclusions

As US health care systems continue to address SDoH, our findings highlight the various challenges in obtaining accurate data on a patient’s social risk. Moreover, these findings highlight the large variability that exists among institutions in a national integrated health care system. Future work should explore the prevalence and variance of other SDoH as a means to help guide resource allocation and prioritize spending to better address SDoH where it is most needed.

Acknowledgments

This work was supported by NHLBI R01 RO1 HL116522-01A1. Support for VA/CMS data is provided by the US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, VA Information Resource Center (Project Numbers SDR 02-237 and 98-004).

Social determinants of health (SDoH) are social, economic, environmental, and occupational factors that are known to influence an individual’s health care utilization and clinical outcomes.^1,2 Because the Veterans Health Administration (VHA) is charged to address both the medical and nonmedical needs of the veteran population, it is increasingly interested in the impact SDoH have on veteran care.^3,4 To combat the adverse impact of such factors, the VHA has implemented several large-scale programs across the US that focus on prevalent SDoH, such as homelessness, substance abuse, and alcohol use disorders.^5,6 While such risk factors are generally universal in their distribution, variation across regions, between urban and rural spaces, and even within cities has been shown to exist in private settings.⁷ Understanding such variability potentially could be helpful to US Department of Veterans Affairs (VA) policymakers and leaders to better allocate funding and resources to address such issues.

Although previous work has highlighted regional and neighborhood-level variability of SDoH, no study has examined the facility-level variability of commonly encountered social risk factors within the VHA.^4,8 The aim of this study was to describe the interfacility variation of 5 common SDoH known to influence health and health outcomes among a national cohort of veterans hospitalized for common medical issues by using administrative data.

Methods

We used a national cohort of veterans aged ≥ 65 years who were hospitalized at a VHA acute care facility with a primary discharge diagnosis of acute myocardial infarction (AMI), heart failure (HF), or pneumonia in 2012. These conditions were chosen because they are publicly reported and frequently used for interfacility comparison.

Using the International Classification of Diseases–9th Revision (ICD-9) and VHA clinical stop codes, we calculated the median documented proportion of patients with any of the following 5 SDoH: lived alone, marginal housing, alcohol use disorder, substance use disorder, and use of substance use services for patients presenting with HF, MI, and pneumonia (Table). These SDoH were chosen because they are intervenable risk factors for which the VHA has several programs (eg, homeless outreach, substance abuse, and tobacco cessation). To examine the variability of these SDoH across VHA facilities, we determined the number of hospitals that had a sufficient number of admissions (≥ 50) to be included in the analyses. We then examined the administratively documented, facility-level variation in the proportion of individuals with any of the 5 SDoH administrative codes and examined the distribution of their use across all qualifying facilities.

Results

In total, 21,991 patients with either HF (9,853), pneumonia (9,362), or AMI (2,776) were identified across 91 VHA facilities. The majority were male (98%) and had a median (SD) age of 77.0 (9.0) years. The median facility-level proportion of veterans who had any of the SDoH risk factors extracted through administrative codes was low across all conditions, ranging from 0.5 to 2.2%. The most prevalent factors among patients admitted for HF, AMI, and pneumonia were lives alone (2.0% [Interquartile range (IQR), 1.0-5.2], 1.4% [IQR, 0-3.4], and 1.9% [IQR, 0.7-5.4]), substance use disorder (1.2% [IQR, 0-2.2], 1.6% [IQR: 0-3.0], and 1.3% [IQR, 0-2.2] and use of substance use services (0.9% [IQR, 0-1.6%], 1.0% [IQR, 0-1.7%], and 1.6% [IQR, 0-2.2%], respectively [Table]).

When utilizing the NLP algorithm, the documented prevalence of lives alone in the free text of the medical record was higher than administrative coding across all conditions (12.3% vs. 2.2%; P < .01). Among each of the 3 assessed conditions, HF (14.4% vs 2.0%, P < .01) had higher levels of lives alone compared with pneumonia (11% vs 1.9%, P < .01), and AMI (10.2% vs 1.4%, P < .01) when using the NLP algorithm. When we examined the documented facility-level variation in the proportion of individuals with any of the 5 SDoH administrative codes or NLP, we found large variability across all facilities—regardless of extraction method (Figure).

Discussion

While SDoH are known to impact health outcomes, the presence of these risk factors in administrative data among individuals hospitalized for common medical issues is low and variable across VHA facilities. Understanding the documented, facility-level variability of these measures may assist the VHA in determining how it invests time and resources—as different facilities may disproportionately serve a higher number of vulnerable individuals. Beyond the VHA, these findings have generalizable lessons for the US health care system, which has come to recognize how these risk factors impact patients’ health.¹⁰

Although the proportion of individuals with any of the assessed SDoH identified by administrative data was low, our findings are in line with recent studies that showed other risk factors such as social isolation (0.65%), housing issues (0.19%), and financial strain (0.07%) had similarly low prevalence.^8,11 Although the exact prevalence of such factors remains unclear, these findings highlight that SDoH do not appear to be well documented in administrative data. Low coding rates are likely due to the fact that SDoH administrative codes are not tied to financial reimbursement—thus not incentivizing their use by clinicians or hospital systems.

In 2014, an Institute of Medicine report suggested that collection of SDoH in electronic health data as a means to better empower clinicians and health care systems to address social disparities and further support research in SDoH.¹² Since then, data collection using SDoH screening tools has become more common across settings, but is not consistently translated to standardized data due to lack of industry consensus and technical barriers.¹³ To improve this process, the Centers for Medicare and Medicaid Services created “z-codes” for the ICD-10 classification system—a subset of codes that are meant to better capture patients’ underlying social risk.¹⁴ It remains to be seen if such administrative codes have improved the documentation of SDoH.

As health care systems have grown to understand the impact of SDoH on health outcomes,other means of collecting these data have evolved.^1,10 For example, NLP-based extraction methods and electronic screening tools have been proposed and utilized as alternative for obtaining this information. Our findings suggest that some of these measures (eg, lives alone) often may be documented as part of routine care in the electronic health record, thus highlighting NLP as a tool to obtain such data. However, other studies using NLP technology to extract SDoH have shown this technology is often complicated by quality issues (ie, missing data), complex methods, and poor integration with current information technology infrastructures—thus limiting its use in health care delivery.^15-18

While variance among SDoH across a national health care system is natural, it remains an important systems-level characteristic that health care leaders and policymakers should appreciate. As health care systems disperse financial resources and initiate quality improvement initiatives to address SDoH, knowing that not all facilities are equally affected by SDoH should impact allocation of such resources and energies. Although previous work has highlighted regional and neighborhood levels of variation within the VHA and other health care systems, to our knowledge, this is the first study to examine variability at the facility-level within the VHA.^2,4,13,19

Limitations

There are several limitations to this study. First, though our findings are in line with previous data in other health care systems, generalizability beyond the VA, which primarily cares for older, male patients, may be limited.⁸ Though, as the nation’s largest health care system, lessons from the VHA can still be useful for other health care systems as they consider SDoH variation. Second, among the many SDoH previously identified to impact health, our analysis only focused on 5 such variables. Administrative and medical record documentation of other SDoH may be more common and less variable across institutions. Third, while our data suggests facility-level variation in these measures, this may be in part related to variation in coding across facilities. However, the single SDoH variable extracted using NLP also varied at the facility-level, suggesting that coding may not entirely drive the variation observed.

Conclusions

As US health care systems continue to address SDoH, our findings highlight the various challenges in obtaining accurate data on a patient’s social risk. Moreover, these findings highlight the large variability that exists among institutions in a national integrated health care system. Future work should explore the prevalence and variance of other SDoH as a means to help guide resource allocation and prioritize spending to better address SDoH where it is most needed.

Acknowledgments

This work was supported by NHLBI R01 RO1 HL116522-01A1. Support for VA/CMS data is provided by the US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, VA Information Resource Center (Project Numbers SDR 02-237 and 98-004).

In recent years, social media platforms like Twitter, Facebook, and Instagram have become popular gathering spots for clinicians to connect, engage, and share medical content. Medical journals, which often act as purveyors of this content, have recognized social media’s growing power and influence and have begun looking for ways to better engage their audiences.

In 2016, the Annals of Surgery was looking to better disseminate the work being published in its pages and looked to Twitter as one way of accomplishing this. At the time, most journals were only posting the title or a brief description of the published manuscript and hoping their Twitter followers would click on the article link. As journal editors were finding, if the audience was not immediately familiar with the topic or able to quickly capture the nuances of the study, there was a good chance the reader would continue to scroll past the post and never view the article.

Recognizing that social media heavily relies on visual material to garner attention, Annals turned to Andrew Ibrahim, MD, an architect turned surgeon, to help them rethink their social media strategy. Using the design training he had previously received in his career as an architect, Dr. Ibrahim created a simple visual tool that could be used to capture the often complicated and nuanced aspects of a research study. He called his creation a “visual abstract.”

But what is a visual abstract? Simply, they are visual representations of the key findings of a published manuscript; or put another way, a “movie trailer” to the full manuscript. While they can take many different forms and designs, they often consist of three key components: (1) a simple, easy to understand title, (2) a primary focus on outcomes, and (3) the use of visual cues or images to help the reader absorb and remember the take home message. This simplified delivery of complex information allows the producer to efficiently share complex findings in a format that allows for rapid visualization and interpretation.

Andrew M. Ibrahim

Since its inception, several studies have examined the influence visual abstracts have on disseminating research. One study conducted by Dr. Ibrahim and his colleagues found that articles tweeted with a visual abstract had an almost eightfold increase in the number of Twitter impressions (a measure of social media dissemination) and a threefold increase in article visits, compared with those manuscripts tweeted with the article title only.¹ These results reflect what behavioral scientists have long understood: Humans process visual data better than any other type of data.² For instance, according to research compiled by 3M, the company behind popular sticky notes, visual data is processed 60,000 times faster than text and has been shown to improve learning by 400%.³ Likewise, digital marketers have found that pages with videos and images draw on average 94% more views than their text-only counterparts.⁴

This knowledge, along with the substantial difference in engagement and dissemination characteristics from Dr. Ibrahim’s study, was far beyond what anyone might have expected and started a trend in medicine that continues to grow today. Medical journals across all practices and disciplines, including several leading journals, such as the New England Journal of Medicine, the Journal of the American Medical Association, and the Journal of Hospital Medicine (JHM), are utilizing this new tool to help disseminate their work in social media.

Visual abstracts have expanded beyond the social media sphere and are now frequently used in Grand Rounds presentations and as teaching tools among medical educators. JHM was one of the first journals to adopt the use of visual abstracts and has since published more than 150 in total. Given the growing popularity and expanded use of visual abstracts, JHM recently began archiving them on the journal’s website to allow clinicians to use the material in their own creative ways.

Visual abstracts are just one piece of the growing enterprise in social media for JHM. Recognizing the growing utilization of social media among physicians, JHM has taken a leading role in the use of online journal clubs. Since 2014, JHM has run a monthly Twitter-based journal club that discusses recently published articles and hospital medicine–based topics, called #JHMChat.⁵ This forum has allowed hospitalists from across the country, and around the world, to connect, network, and engage around topics important to the field of hospital medicine. The journal frequently reaches beyond hospital medicine borders and partners with other specialties and interest groups to gain perspective and insights into shared topic areas. To date, #JHMChat has one of the most robust online communities and continues to attract new followers each month.

As social media use continues to expand among clinicians, engagement tools like visual abstracts and Twitter chats will certainly continue to grow. Given that more clinicians are scrolling through websites than flipping through journal pages, medical journals like JHM will continually look for novel ways to engage their audiences and create communities among their followers. While a former architect who now practices as a surgeon led the way with visual abstracts, it remains to be seen who will create the next tool used to capture our attention on the ever-evolving sphere of social media.
 

Dr. Wray is a hospitalist at the University of California, San Francisco, and the San Francisco Veterans Affairs Medical Center. He also serves as a digital media and associate editor for the Journal of Hospital Medicine.

References

1. Ibrahim AM et al. Visual abstracts to disseminate research on social media: A prospective, case-control crossover study. Ann Surg. 2017;266(6):e46.

2. Tufte ER. The Visual Display of Quantitative Information. Second edition. Cheshire, Conn. Graphics Press, 2001. https://search.library.wisc.edu/catalog/999913808702121.

3. Polishing Your Presentation. http://web.archive.org/web/20001014041642/http://www.3m.com:80/meetingnetwork/files/meetingguide_pres.pdf. Accessed May 28, 2017.

4. 7 reasons you need visual content in your marketing strategy. https://medium.com/@nikos_iliopoulos/7-reasons-you-need-visual-content-in-your-marketing-strategy-bc77ca5521ac. Accessed May 28, 2017.

5. Wray CM et al. The adoption of an online journal club to improve research dissemination and social media engagement among hospitalists. J Hosp Med. 2018. doi: 10.12788/jhm.2987.

In recent years, social media platforms like Twitter, Facebook, and Instagram have become popular gathering spots for clinicians to connect, engage, and share medical content. Medical journals, which often act as purveyors of this content, have recognized social media’s growing power and influence and have begun looking for ways to better engage their audiences.

In 2016, the Annals of Surgery was looking to better disseminate the work being published in its pages and looked to Twitter as one way of accomplishing this. At the time, most journals were only posting the title or a brief description of the published manuscript and hoping their Twitter followers would click on the article link. As journal editors were finding, if the audience was not immediately familiar with the topic or able to quickly capture the nuances of the study, there was a good chance the reader would continue to scroll past the post and never view the article.

Recognizing that social media heavily relies on visual material to garner attention, Annals turned to Andrew Ibrahim, MD, an architect turned surgeon, to help them rethink their social media strategy. Using the design training he had previously received in his career as an architect, Dr. Ibrahim created a simple visual tool that could be used to capture the often complicated and nuanced aspects of a research study. He called his creation a “visual abstract.”

But what is a visual abstract? Simply, they are visual representations of the key findings of a published manuscript; or put another way, a “movie trailer” to the full manuscript. While they can take many different forms and designs, they often consist of three key components: (1) a simple, easy to understand title, (2) a primary focus on outcomes, and (3) the use of visual cues or images to help the reader absorb and remember the take home message. This simplified delivery of complex information allows the producer to efficiently share complex findings in a format that allows for rapid visualization and interpretation.

Andrew M. Ibrahim

Since its inception, several studies have examined the influence visual abstracts have on disseminating research. One study conducted by Dr. Ibrahim and his colleagues found that articles tweeted with a visual abstract had an almost eightfold increase in the number of Twitter impressions (a measure of social media dissemination) and a threefold increase in article visits, compared with those manuscripts tweeted with the article title only.¹ These results reflect what behavioral scientists have long understood: Humans process visual data better than any other type of data.² For instance, according to research compiled by 3M, the company behind popular sticky notes, visual data is processed 60,000 times faster than text and has been shown to improve learning by 400%.³ Likewise, digital marketers have found that pages with videos and images draw on average 94% more views than their text-only counterparts.⁴

This knowledge, along with the substantial difference in engagement and dissemination characteristics from Dr. Ibrahim’s study, was far beyond what anyone might have expected and started a trend in medicine that continues to grow today. Medical journals across all practices and disciplines, including several leading journals, such as the New England Journal of Medicine, the Journal of the American Medical Association, and the Journal of Hospital Medicine (JHM), are utilizing this new tool to help disseminate their work in social media.

Visual abstracts have expanded beyond the social media sphere and are now frequently used in Grand Rounds presentations and as teaching tools among medical educators. JHM was one of the first journals to adopt the use of visual abstracts and has since published more than 150 in total. Given the growing popularity and expanded use of visual abstracts, JHM recently began archiving them on the journal’s website to allow clinicians to use the material in their own creative ways.

Visual abstracts are just one piece of the growing enterprise in social media for JHM. Recognizing the growing utilization of social media among physicians, JHM has taken a leading role in the use of online journal clubs. Since 2014, JHM has run a monthly Twitter-based journal club that discusses recently published articles and hospital medicine–based topics, called #JHMChat.⁵ This forum has allowed hospitalists from across the country, and around the world, to connect, network, and engage around topics important to the field of hospital medicine. The journal frequently reaches beyond hospital medicine borders and partners with other specialties and interest groups to gain perspective and insights into shared topic areas. To date, #JHMChat has one of the most robust online communities and continues to attract new followers each month.

As social media use continues to expand among clinicians, engagement tools like visual abstracts and Twitter chats will certainly continue to grow. Given that more clinicians are scrolling through websites than flipping through journal pages, medical journals like JHM will continually look for novel ways to engage their audiences and create communities among their followers. While a former architect who now practices as a surgeon led the way with visual abstracts, it remains to be seen who will create the next tool used to capture our attention on the ever-evolving sphere of social media.
 

Dr. Wray is a hospitalist at the University of California, San Francisco, and the San Francisco Veterans Affairs Medical Center. He also serves as a digital media and associate editor for the Journal of Hospital Medicine.

References

1. Ibrahim AM et al. Visual abstracts to disseminate research on social media: A prospective, case-control crossover study. Ann Surg. 2017;266(6):e46.

2. Tufte ER. The Visual Display of Quantitative Information. Second edition. Cheshire, Conn. Graphics Press, 2001. https://search.library.wisc.edu/catalog/999913808702121.

3. Polishing Your Presentation. http://web.archive.org/web/20001014041642/http://www.3m.com:80/meetingnetwork/files/meetingguide_pres.pdf. Accessed May 28, 2017.

4. 7 reasons you need visual content in your marketing strategy. https://medium.com/@nikos_iliopoulos/7-reasons-you-need-visual-content-in-your-marketing-strategy-bc77ca5521ac. Accessed May 28, 2017.

5. Wray CM et al. The adoption of an online journal club to improve research dissemination and social media engagement among hospitalists. J Hosp Med. 2018. doi: 10.12788/jhm.2987.

In recent years, social media platforms like Twitter, Facebook, and Instagram have become popular gathering spots for clinicians to connect, engage, and share medical content. Medical journals, which often act as purveyors of this content, have recognized social media’s growing power and influence and have begun looking for ways to better engage their audiences.

In 2016, the Annals of Surgery was looking to better disseminate the work being published in its pages and looked to Twitter as one way of accomplishing this. At the time, most journals were only posting the title or a brief description of the published manuscript and hoping their Twitter followers would click on the article link. As journal editors were finding, if the audience was not immediately familiar with the topic or able to quickly capture the nuances of the study, there was a good chance the reader would continue to scroll past the post and never view the article.

Recognizing that social media heavily relies on visual material to garner attention, Annals turned to Andrew Ibrahim, MD, an architect turned surgeon, to help them rethink their social media strategy. Using the design training he had previously received in his career as an architect, Dr. Ibrahim created a simple visual tool that could be used to capture the often complicated and nuanced aspects of a research study. He called his creation a “visual abstract.”

But what is a visual abstract? Simply, they are visual representations of the key findings of a published manuscript; or put another way, a “movie trailer” to the full manuscript. While they can take many different forms and designs, they often consist of three key components: (1) a simple, easy to understand title, (2) a primary focus on outcomes, and (3) the use of visual cues or images to help the reader absorb and remember the take home message. This simplified delivery of complex information allows the producer to efficiently share complex findings in a format that allows for rapid visualization and interpretation.

Andrew M. Ibrahim

Since its inception, several studies have examined the influence visual abstracts have on disseminating research. One study conducted by Dr. Ibrahim and his colleagues found that articles tweeted with a visual abstract had an almost eightfold increase in the number of Twitter impressions (a measure of social media dissemination) and a threefold increase in article visits, compared with those manuscripts tweeted with the article title only.¹ These results reflect what behavioral scientists have long understood: Humans process visual data better than any other type of data.² For instance, according to research compiled by 3M, the company behind popular sticky notes, visual data is processed 60,000 times faster than text and has been shown to improve learning by 400%.³ Likewise, digital marketers have found that pages with videos and images draw on average 94% more views than their text-only counterparts.⁴

This knowledge, along with the substantial difference in engagement and dissemination characteristics from Dr. Ibrahim’s study, was far beyond what anyone might have expected and started a trend in medicine that continues to grow today. Medical journals across all practices and disciplines, including several leading journals, such as the New England Journal of Medicine, the Journal of the American Medical Association, and the Journal of Hospital Medicine (JHM), are utilizing this new tool to help disseminate their work in social media.

Visual abstracts have expanded beyond the social media sphere and are now frequently used in Grand Rounds presentations and as teaching tools among medical educators. JHM was one of the first journals to adopt the use of visual abstracts and has since published more than 150 in total. Given the growing popularity and expanded use of visual abstracts, JHM recently began archiving them on the journal’s website to allow clinicians to use the material in their own creative ways.

Visual abstracts are just one piece of the growing enterprise in social media for JHM. Recognizing the growing utilization of social media among physicians, JHM has taken a leading role in the use of online journal clubs. Since 2014, JHM has run a monthly Twitter-based journal club that discusses recently published articles and hospital medicine–based topics, called #JHMChat.⁵ This forum has allowed hospitalists from across the country, and around the world, to connect, network, and engage around topics important to the field of hospital medicine. The journal frequently reaches beyond hospital medicine borders and partners with other specialties and interest groups to gain perspective and insights into shared topic areas. To date, #JHMChat has one of the most robust online communities and continues to attract new followers each month.

As social media use continues to expand among clinicians, engagement tools like visual abstracts and Twitter chats will certainly continue to grow. Given that more clinicians are scrolling through websites than flipping through journal pages, medical journals like JHM will continually look for novel ways to engage their audiences and create communities among their followers. While a former architect who now practices as a surgeon led the way with visual abstracts, it remains to be seen who will create the next tool used to capture our attention on the ever-evolving sphere of social media.
 

Dr. Wray is a hospitalist at the University of California, San Francisco, and the San Francisco Veterans Affairs Medical Center. He also serves as a digital media and associate editor for the Journal of Hospital Medicine.

References

1. Ibrahim AM et al. Visual abstracts to disseminate research on social media: A prospective, case-control crossover study. Ann Surg. 2017;266(6):e46.

2. Tufte ER. The Visual Display of Quantitative Information. Second edition. Cheshire, Conn. Graphics Press, 2001. https://search.library.wisc.edu/catalog/999913808702121.

3. Polishing Your Presentation. http://web.archive.org/web/20001014041642/http://www.3m.com:80/meetingnetwork/files/meetingguide_pres.pdf. Accessed May 28, 2017.

4. 7 reasons you need visual content in your marketing strategy. https://medium.com/@nikos_iliopoulos/7-reasons-you-need-visual-content-in-your-marketing-strategy-bc77ca5521ac. Accessed May 28, 2017.

5. Wray CM et al. The adoption of an online journal club to improve research dissemination and social media engagement among hospitalists. J Hosp Med. 2018. doi: 10.12788/jhm.2987.

The peer review and editorial processes are integral activities in academic medicine that provide ethical, independent, and unbiased critical assessment of submitted manuscripts to academic journals. Recognizing that few trainees or junior faculty are formally exposed to these processes,¹ the Journal of Hospital Medicine aims to fill this opportunity gap through the launch of a one-year Editorial Fellowship.

The Fellowship is open to chief residents, hospital medicine fellows, and junior faculty (eg, Assistant Professor or Clinical Instructor). Starting in July of each year, a group of four to six applicants are paired with editorial mentors who are current JHM Deputy or Associate Editors. Structured as a distance-learning program, this program aims to allow Fellows the ability to continue in their full time professional roles while also allowing the opportunity to engage with national leaders in hospital medicine. Regular communication and interactions take place through both synchronous and asynchronous means. Fellows’ responsibilities during the 12-month experience include: completion of six guided peer reviews, preparation of one or two editorials, participation in monthly editorial meetings, and quarterly educational videoconferences. Interested Fellows may also have an opportunity to co-lead the journal’s online journal club, #JHMChat.² Fellows are expected to attend the editorial staff meeting at the annual Society of Hospital Medicine Conference.

With this program, JHM aims to accomplish several tasks. First, we hope to offer a unique educational experience that allows for further growth, development, inspiration, and experience in academic medicine—specifically around the manuscript review and editorial processes. Second, recognizing that a journal’s quality is frequently a product of its reviewers, JHM hopes to build a cadre of well-trained and experienced reviewers and, hopefully, future members of the JHM editorial leadership team. Third, the program hopes to act as a networking experience, allowing editorial Fellows to learn from, collaborate with, and become academic leaders in the field. Finally, we hope to provide an opportunity for Fellows to be academically productive in their composition of editorial content—an output that will help catalyze their professional development.

We believe that in working closely with the JHM editorial staff, this program will help develop the next generation of leaders in academic hospital medicine. We strongly encourage applications from physicians who have been historically under-represented in leadership in academic medicine. Further details and the application can be found in the appendix and on the JHM website (www.journalofhospitalmedicine.com). It will be announced annually through the @JHospMedicine twitter handle.

Disclosures

The authors have nothing to disclose.

The peer review and editorial processes are integral activities in academic medicine that provide ethical, independent, and unbiased critical assessment of submitted manuscripts to academic journals. Recognizing that few trainees or junior faculty are formally exposed to these processes,¹ the Journal of Hospital Medicine aims to fill this opportunity gap through the launch of a one-year Editorial Fellowship.

The Fellowship is open to chief residents, hospital medicine fellows, and junior faculty (eg, Assistant Professor or Clinical Instructor). Starting in July of each year, a group of four to six applicants are paired with editorial mentors who are current JHM Deputy or Associate Editors. Structured as a distance-learning program, this program aims to allow Fellows the ability to continue in their full time professional roles while also allowing the opportunity to engage with national leaders in hospital medicine. Regular communication and interactions take place through both synchronous and asynchronous means. Fellows’ responsibilities during the 12-month experience include: completion of six guided peer reviews, preparation of one or two editorials, participation in monthly editorial meetings, and quarterly educational videoconferences. Interested Fellows may also have an opportunity to co-lead the journal’s online journal club, #JHMChat.² Fellows are expected to attend the editorial staff meeting at the annual Society of Hospital Medicine Conference.

With this program, JHM aims to accomplish several tasks. First, we hope to offer a unique educational experience that allows for further growth, development, inspiration, and experience in academic medicine—specifically around the manuscript review and editorial processes. Second, recognizing that a journal’s quality is frequently a product of its reviewers, JHM hopes to build a cadre of well-trained and experienced reviewers and, hopefully, future members of the JHM editorial leadership team. Third, the program hopes to act as a networking experience, allowing editorial Fellows to learn from, collaborate with, and become academic leaders in the field. Finally, we hope to provide an opportunity for Fellows to be academically productive in their composition of editorial content—an output that will help catalyze their professional development.

We believe that in working closely with the JHM editorial staff, this program will help develop the next generation of leaders in academic hospital medicine. We strongly encourage applications from physicians who have been historically under-represented in leadership in academic medicine. Further details and the application can be found in the appendix and on the JHM website (www.journalofhospitalmedicine.com). It will be announced annually through the @JHospMedicine twitter handle.

Disclosures

The authors have nothing to disclose.

The peer review and editorial processes are integral activities in academic medicine that provide ethical, independent, and unbiased critical assessment of submitted manuscripts to academic journals. Recognizing that few trainees or junior faculty are formally exposed to these processes,¹ the Journal of Hospital Medicine aims to fill this opportunity gap through the launch of a one-year Editorial Fellowship.

The Fellowship is open to chief residents, hospital medicine fellows, and junior faculty (eg, Assistant Professor or Clinical Instructor). Starting in July of each year, a group of four to six applicants are paired with editorial mentors who are current JHM Deputy or Associate Editors. Structured as a distance-learning program, this program aims to allow Fellows the ability to continue in their full time professional roles while also allowing the opportunity to engage with national leaders in hospital medicine. Regular communication and interactions take place through both synchronous and asynchronous means. Fellows’ responsibilities during the 12-month experience include: completion of six guided peer reviews, preparation of one or two editorials, participation in monthly editorial meetings, and quarterly educational videoconferences. Interested Fellows may also have an opportunity to co-lead the journal’s online journal club, #JHMChat.² Fellows are expected to attend the editorial staff meeting at the annual Society of Hospital Medicine Conference.

With this program, JHM aims to accomplish several tasks. First, we hope to offer a unique educational experience that allows for further growth, development, inspiration, and experience in academic medicine—specifically around the manuscript review and editorial processes. Second, recognizing that a journal’s quality is frequently a product of its reviewers, JHM hopes to build a cadre of well-trained and experienced reviewers and, hopefully, future members of the JHM editorial leadership team. Third, the program hopes to act as a networking experience, allowing editorial Fellows to learn from, collaborate with, and become academic leaders in the field. Finally, we hope to provide an opportunity for Fellows to be academically productive in their composition of editorial content—an output that will help catalyze their professional development.

We believe that in working closely with the JHM editorial staff, this program will help develop the next generation of leaders in academic hospital medicine. We strongly encourage applications from physicians who have been historically under-represented in leadership in academic medicine. Further details and the application can be found in the appendix and on the JHM website (www.journalofhospitalmedicine.com). It will be announced annually through the @JHospMedicine twitter handle.

Disclosures

The authors have nothing to disclose.

Clinicians, educators, and medical journals are increasingly using the social media outlet, Twitter, as a medium to connect and engage with their colleagues. In particular, online journal clubs have created a space for the timely discussion of research, creation of online communities, and dissemination of research.

Social media-based journal clubs are thought to be one way in which journals can leverage the power of social networks so that researchers can engage with a diverse range of end users⁴ (including bedside clinicians, administrators, and patients). Several examples of these models exist. For example, #GeriMedJC acts as a complimentary, synchronous chat that takes place at the same time as a live, in-person journal club. #NephJC offers multiple 1-hour chats per month and provides an in-depth summary and analysis of each article, while #UroJC is an asynchronous discussion that takes place over 48 hours. Few data exist to describe whether any of these programs produce measurable improvements in indicators of engagement or dissemination of results.

In 2015, the Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club as a means to connect and engage the Hospital Medicine community and allow for discussion and rapid exchange of information and opinions around a specific clinical topic. This study aims to describe the implementation of the first Journal-sponsored, Twitter-based online journal club and ascertain its impact on both Twitter and journal metrics.

#JHMChat was launched in October 2015, and was initially held every 2-3 months until January 2017, when chats began to take place monthly. Each 1-hour chat focused on a recently published article in JHM, was moderated by a JHM social media editor (C.M.W., V.M.A.), and included at least 1 study author or guest expert. Articles were chosen by the social media editors based on the following criteria: (1) attractiveness to possible participants, (2) providing topic variety within the journal club series, and (3) sustainability and topic conduciveness to the online chat model. Chats were held at 9 PM EST in order to engage hospitalists across all US time zones and on different days to accommodate authors’ availability. All sessions were framed by 3-4 questions intended to encourage discussion and presented to chat participants at spaced intervals so as to stimulate a current of responses.

Chats were promoted by way of the JHM (@JHospMedicine, 3400 followers) and Society of Hospital Medicine (SHM; @SHMLive, 5800 followers) Twitter feeds beginning 1 month prior to each session. Visual Abstracts^5,6 were used to publicize the sessions, also via Twitter, starting in February 2017.

Continuing Medical Education (CME) credits were offered through the SHM to registered participants, starting in July 2016.⁷ All sessions were cosponsored by the American Board of Internal Medicine (ABIM) Foundation and the Costs of Care Organization, a non-profit organization aimed at improving healthcare value.

Twitter Metrics

After each session, the following Twitter-based engagement metrics were obtained using the Symplur^© Healthcare Hashtag project;⁸ total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.

Altmetrics and Page Views

As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score⁹, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.

Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.

Thematic Analysis

In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).

Participant and Author Experience

To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey^®) meant to assess the authors’ experience with the journal club (Appendix).

Between October 2015 and November 2017, a total of 15 sessions were held with a mean of 2.17 (±0.583) million impressions/session, 499 (±129) total tweets/session, and 73 (±24) participants/session (compared to a range of 21-58 participants/session from other online journal clubs, where reported) with 7.2 (±2.0) tweets/participant (Table 1). The total number of participants for all sessions was 1096. Participating authors had on average 1389 (±2714) followers, ranging from a low of 37 to a high of 10,376 (Appendix). No correlation between author following and number of participants (r = 0.19), impressions (r = 0.05), or change in Altmetric score (r = 0.17) was seen.

Thematic analysis revealed 3 predominant themes among the chosen articles: Value-based care (VBC), Quality and Patient Safety (QPS), and Medical Education (ME). Articles focused on VBC had the greatest number of impressions (mean ±SD: 2.61 ± 0.55 million) and participants (mean ±SD: 90 ± 12), while QPS articles had the fewest impressions (mean ±SD: 1.71 ± 0.59 million) and number of participants (mean ±SD: 47 ± 16). The mean increase in the Altmetric score among all discussed articles was 14 (±12), from an average baseline of 30 (±37). Medical Education-themed articles appeared to garner the greatest increase in Altmetric scores, averaging an increase of 32 points, compared with an average baseline score of 31 (±32). In contrast, VBC and QPS articles averaged an increase of 8.6 and 8.4 points, from average baselines of 55 (±53) and 17 (±13), respectively. A 2-month analysis of JHM articles not included in these discussions, in which Altmetric scores were measured in the same way as those from the discussion, revealed a baseline Altmetric score of 27 (±24) with an average increase of 8 (±6) 2 weeks following the chat.

Four articles met the inclusion criteria for page view analysis and suggested that article page views increased to similar levels as the eTOC release (mean: 2668 vs. 2998, respectively; P = .35) (Figure). These increases equate to a 33% and 50% increase in average daily page views (2002) for the chat and eTOC release, respectively.

The Twitter-based journal club #JHMChat appears to increase social media awareness and dissemination of journal articles and was considered a useful engagement platform by both authors and participants.

Articles with a focus on VBC and ME had the greatest impact on dissemination metrics, particularly, total impressions and Altmetric scores, respectively. Given the strong presence and interest in these topics within Twitter and social media, these findings are not surprising.^10,11 For example, over the past several years, the VBC movement has taken shape and grown alongside the expansion of social media, thus giving a space for this community to grow and engage. Of note, the cosponsorship relationship with the ABIM Foundation (which works closely with the Choosing Wisely™ campaign) and the Costs of Care Organization could have influenced the participation and dissemination rates of VBC articles. Medical education articles were also popular and appeared to have increased uptake after chats, based on their Altmetric scores. This may be explained by the fact that medical educators have long utilized social media as a means to connect and engage within their community.^12–14 It is also possible that the use of Twitter by trainees (residents, students) may have driven some of the dissemination of ME articles, as this group may not be regular subscribers to JHM.

Online journal clubs offer distinct advantages over traditional in-person journal clubs. First, online journal clubs allow for increased connectivity among online communities, bringing together participants from different geographic areas with diverse training and clinical experiences. Subsequently, this allows for the rapid exchange of both personal and organizational approaches to the topic of discussion.^15–17 Second, online journal clubs allow for continual access to the discussion material. For example, while the metrics used in this study only assessed active, synchronous participation, anecdotal evidence and feedback to the authors suggests that many individuals passively engaged by following along or reviewed the chat feed post hoc at their convenience. This asynchronous access is a quality not found in more traditional journal club formats. Finally, because online journal clubs commonly operate with a flattened hierarchy,¹⁸ they can break down access barriers to both the researchers who performed the study and thought leaders who commonly participate.¹⁷

Several insightful lessons were gleaned in the production and management of this online journal club. On the implementation side, promotion, preparation, and continued organization of an online journal club requires a fair amount of work. In this case, the required time and resources were provided by 2 social media editors in addition to administrative assistance from the SHM. The high attrition rate of online journal clubs over the years attests to these difficulties.²⁴ Additionally, finding incentives to attract and sustain participation can be difficult, as we noted that neither CME nor author popularity (based on their Twitter following) appeared to influence engagement metrics (number of participants, total tweets, and tweets/participant). We also found that partnering with other journal club communities, in particular #NephJC, lead to greater participation rates and impressions. Thus, leveraging connections and topics that span clinical domains may be one way to improve and broaden engagement within these forums. Finally, feedback from participants revealed that the timing of the journal club and the inability to have in-depth discussions, a characteristic commonly associated with traditional journal clubs, were problematic.

This study has several limitations. First, the metrics used to assess social media engagement and dissemination can be easily skewed. For instance, the activity of 1 or 2 individuals with large followings can dramatically influence the number of impressions, giving a falsely elevated sense of broad dissemination. Conversely, there may have been some participants who did not use the #JHMChat hashtag, thus leading to an underestimation in these metrics. Second, while we report total impressions as a measure of dissemination, this metric represents possible interactions and does not guarantee interaction or visualization of that tweet. Additionally, we were unable to characterize our participants and their participation rates over time, as this information is not made available through Symplur© analytics. Third, our page view assessment was limited to 2016 sessions only; therefore, these data may not be an accurate reflection of the impact of #JHMChat on this metric. Fourth, given the marginal response rate to our CME questionnaire, a selection bias could have occurred. Finally, whether social media discussions such as online journal clubs act as leading indicators for future citations remains unclear, as some research has shown an association between increased Altmetric scores and increased citation rates,^19-21 while others have not.^22,23 Our study was not equipped to assess this correlation.

Online journal clubs create new opportunities to connect, engage, and disseminate medical research. These developing forums provide journal editors, researchers, patients, and clinicians with a means to connect and discuss research in ways that were not previously possible. In order to continue to evolve and grow, future research in online journal clubs should explore the downstream effects on citation rates, clinical uptake, and participant knowledge after the sessions.

Acknowledgments

The authors would like to thank Felicia Steele for her assistance in organizing and promoting the chats. Additionally, the authors would like to thank all the authors, guests and participants who took time from their families, work, and daily lives to participate in these activities. Your time and presence were truly appreciated.

Disclosures

The authors of this article operate as the Social Media Editors (C.M.W., V.M.A.) and the Editor-in-Chief (A.A.) for the Journal of Hospital Medicine. Dr. Wray had full access to all the data in the project, takes responsibility for the integrity of the data, and the accuracy of the data analysis.

Clinicians, educators, and medical journals are increasingly using the social media outlet, Twitter, as a medium to connect and engage with their colleagues. In particular, online journal clubs have created a space for the timely discussion of research, creation of online communities, and dissemination of research.

Social media-based journal clubs are thought to be one way in which journals can leverage the power of social networks so that researchers can engage with a diverse range of end users⁴ (including bedside clinicians, administrators, and patients). Several examples of these models exist. For example, #GeriMedJC acts as a complimentary, synchronous chat that takes place at the same time as a live, in-person journal club. #NephJC offers multiple 1-hour chats per month and provides an in-depth summary and analysis of each article, while #UroJC is an asynchronous discussion that takes place over 48 hours. Few data exist to describe whether any of these programs produce measurable improvements in indicators of engagement or dissemination of results.

In 2015, the Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club as a means to connect and engage the Hospital Medicine community and allow for discussion and rapid exchange of information and opinions around a specific clinical topic. This study aims to describe the implementation of the first Journal-sponsored, Twitter-based online journal club and ascertain its impact on both Twitter and journal metrics.

#JHMChat was launched in October 2015, and was initially held every 2-3 months until January 2017, when chats began to take place monthly. Each 1-hour chat focused on a recently published article in JHM, was moderated by a JHM social media editor (C.M.W., V.M.A.), and included at least 1 study author or guest expert. Articles were chosen by the social media editors based on the following criteria: (1) attractiveness to possible participants, (2) providing topic variety within the journal club series, and (3) sustainability and topic conduciveness to the online chat model. Chats were held at 9 PM EST in order to engage hospitalists across all US time zones and on different days to accommodate authors’ availability. All sessions were framed by 3-4 questions intended to encourage discussion and presented to chat participants at spaced intervals so as to stimulate a current of responses.

Chats were promoted by way of the JHM (@JHospMedicine, 3400 followers) and Society of Hospital Medicine (SHM; @SHMLive, 5800 followers) Twitter feeds beginning 1 month prior to each session. Visual Abstracts^5,6 were used to publicize the sessions, also via Twitter, starting in February 2017.

Continuing Medical Education (CME) credits were offered through the SHM to registered participants, starting in July 2016.⁷ All sessions were cosponsored by the American Board of Internal Medicine (ABIM) Foundation and the Costs of Care Organization, a non-profit organization aimed at improving healthcare value.

Twitter Metrics

After each session, the following Twitter-based engagement metrics were obtained using the Symplur^© Healthcare Hashtag project;⁸ total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.

Altmetrics and Page Views

As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score⁹, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.

Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.

Thematic Analysis

In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).

Participant and Author Experience

To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey^®) meant to assess the authors’ experience with the journal club (Appendix).

Between October 2015 and November 2017, a total of 15 sessions were held with a mean of 2.17 (±0.583) million impressions/session, 499 (±129) total tweets/session, and 73 (±24) participants/session (compared to a range of 21-58 participants/session from other online journal clubs, where reported) with 7.2 (±2.0) tweets/participant (Table 1). The total number of participants for all sessions was 1096. Participating authors had on average 1389 (±2714) followers, ranging from a low of 37 to a high of 10,376 (Appendix). No correlation between author following and number of participants (r = 0.19), impressions (r = 0.05), or change in Altmetric score (r = 0.17) was seen.

Thematic analysis revealed 3 predominant themes among the chosen articles: Value-based care (VBC), Quality and Patient Safety (QPS), and Medical Education (ME). Articles focused on VBC had the greatest number of impressions (mean ±SD: 2.61 ± 0.55 million) and participants (mean ±SD: 90 ± 12), while QPS articles had the fewest impressions (mean ±SD: 1.71 ± 0.59 million) and number of participants (mean ±SD: 47 ± 16). The mean increase in the Altmetric score among all discussed articles was 14 (±12), from an average baseline of 30 (±37). Medical Education-themed articles appeared to garner the greatest increase in Altmetric scores, averaging an increase of 32 points, compared with an average baseline score of 31 (±32). In contrast, VBC and QPS articles averaged an increase of 8.6 and 8.4 points, from average baselines of 55 (±53) and 17 (±13), respectively. A 2-month analysis of JHM articles not included in these discussions, in which Altmetric scores were measured in the same way as those from the discussion, revealed a baseline Altmetric score of 27 (±24) with an average increase of 8 (±6) 2 weeks following the chat.

Four articles met the inclusion criteria for page view analysis and suggested that article page views increased to similar levels as the eTOC release (mean: 2668 vs. 2998, respectively; P = .35) (Figure). These increases equate to a 33% and 50% increase in average daily page views (2002) for the chat and eTOC release, respectively.

The Twitter-based journal club #JHMChat appears to increase social media awareness and dissemination of journal articles and was considered a useful engagement platform by both authors and participants.

Articles with a focus on VBC and ME had the greatest impact on dissemination metrics, particularly, total impressions and Altmetric scores, respectively. Given the strong presence and interest in these topics within Twitter and social media, these findings are not surprising.^10,11 For example, over the past several years, the VBC movement has taken shape and grown alongside the expansion of social media, thus giving a space for this community to grow and engage. Of note, the cosponsorship relationship with the ABIM Foundation (which works closely with the Choosing Wisely™ campaign) and the Costs of Care Organization could have influenced the participation and dissemination rates of VBC articles. Medical education articles were also popular and appeared to have increased uptake after chats, based on their Altmetric scores. This may be explained by the fact that medical educators have long utilized social media as a means to connect and engage within their community.^12–14 It is also possible that the use of Twitter by trainees (residents, students) may have driven some of the dissemination of ME articles, as this group may not be regular subscribers to JHM.

Online journal clubs offer distinct advantages over traditional in-person journal clubs. First, online journal clubs allow for increased connectivity among online communities, bringing together participants from different geographic areas with diverse training and clinical experiences. Subsequently, this allows for the rapid exchange of both personal and organizational approaches to the topic of discussion.^15–17 Second, online journal clubs allow for continual access to the discussion material. For example, while the metrics used in this study only assessed active, synchronous participation, anecdotal evidence and feedback to the authors suggests that many individuals passively engaged by following along or reviewed the chat feed post hoc at their convenience. This asynchronous access is a quality not found in more traditional journal club formats. Finally, because online journal clubs commonly operate with a flattened hierarchy,¹⁸ they can break down access barriers to both the researchers who performed the study and thought leaders who commonly participate.¹⁷

Several insightful lessons were gleaned in the production and management of this online journal club. On the implementation side, promotion, preparation, and continued organization of an online journal club requires a fair amount of work. In this case, the required time and resources were provided by 2 social media editors in addition to administrative assistance from the SHM. The high attrition rate of online journal clubs over the years attests to these difficulties.²⁴ Additionally, finding incentives to attract and sustain participation can be difficult, as we noted that neither CME nor author popularity (based on their Twitter following) appeared to influence engagement metrics (number of participants, total tweets, and tweets/participant). We also found that partnering with other journal club communities, in particular #NephJC, lead to greater participation rates and impressions. Thus, leveraging connections and topics that span clinical domains may be one way to improve and broaden engagement within these forums. Finally, feedback from participants revealed that the timing of the journal club and the inability to have in-depth discussions, a characteristic commonly associated with traditional journal clubs, were problematic.

This study has several limitations. First, the metrics used to assess social media engagement and dissemination can be easily skewed. For instance, the activity of 1 or 2 individuals with large followings can dramatically influence the number of impressions, giving a falsely elevated sense of broad dissemination. Conversely, there may have been some participants who did not use the #JHMChat hashtag, thus leading to an underestimation in these metrics. Second, while we report total impressions as a measure of dissemination, this metric represents possible interactions and does not guarantee interaction or visualization of that tweet. Additionally, we were unable to characterize our participants and their participation rates over time, as this information is not made available through Symplur© analytics. Third, our page view assessment was limited to 2016 sessions only; therefore, these data may not be an accurate reflection of the impact of #JHMChat on this metric. Fourth, given the marginal response rate to our CME questionnaire, a selection bias could have occurred. Finally, whether social media discussions such as online journal clubs act as leading indicators for future citations remains unclear, as some research has shown an association between increased Altmetric scores and increased citation rates,^19-21 while others have not.^22,23 Our study was not equipped to assess this correlation.

Online journal clubs create new opportunities to connect, engage, and disseminate medical research. These developing forums provide journal editors, researchers, patients, and clinicians with a means to connect and discuss research in ways that were not previously possible. In order to continue to evolve and grow, future research in online journal clubs should explore the downstream effects on citation rates, clinical uptake, and participant knowledge after the sessions.

Acknowledgments

The authors would like to thank Felicia Steele for her assistance in organizing and promoting the chats. Additionally, the authors would like to thank all the authors, guests and participants who took time from their families, work, and daily lives to participate in these activities. Your time and presence were truly appreciated.

Disclosures

The authors of this article operate as the Social Media Editors (C.M.W., V.M.A.) and the Editor-in-Chief (A.A.) for the Journal of Hospital Medicine. Dr. Wray had full access to all the data in the project, takes responsibility for the integrity of the data, and the accuracy of the data analysis.

Clinicians, educators, and medical journals are increasingly using the social media outlet, Twitter, as a medium to connect and engage with their colleagues. In particular, online journal clubs have created a space for the timely discussion of research, creation of online communities, and dissemination of research.

Social media-based journal clubs are thought to be one way in which journals can leverage the power of social networks so that researchers can engage with a diverse range of end users⁴ (including bedside clinicians, administrators, and patients). Several examples of these models exist. For example, #GeriMedJC acts as a complimentary, synchronous chat that takes place at the same time as a live, in-person journal club. #NephJC offers multiple 1-hour chats per month and provides an in-depth summary and analysis of each article, while #UroJC is an asynchronous discussion that takes place over 48 hours. Few data exist to describe whether any of these programs produce measurable improvements in indicators of engagement or dissemination of results.

In 2015, the Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club as a means to connect and engage the Hospital Medicine community and allow for discussion and rapid exchange of information and opinions around a specific clinical topic. This study aims to describe the implementation of the first Journal-sponsored, Twitter-based online journal club and ascertain its impact on both Twitter and journal metrics.

#JHMChat was launched in October 2015, and was initially held every 2-3 months until January 2017, when chats began to take place monthly. Each 1-hour chat focused on a recently published article in JHM, was moderated by a JHM social media editor (C.M.W., V.M.A.), and included at least 1 study author or guest expert. Articles were chosen by the social media editors based on the following criteria: (1) attractiveness to possible participants, (2) providing topic variety within the journal club series, and (3) sustainability and topic conduciveness to the online chat model. Chats were held at 9 PM EST in order to engage hospitalists across all US time zones and on different days to accommodate authors’ availability. All sessions were framed by 3-4 questions intended to encourage discussion and presented to chat participants at spaced intervals so as to stimulate a current of responses.

Chats were promoted by way of the JHM (@JHospMedicine, 3400 followers) and Society of Hospital Medicine (SHM; @SHMLive, 5800 followers) Twitter feeds beginning 1 month prior to each session. Visual Abstracts^5,6 were used to publicize the sessions, also via Twitter, starting in February 2017.

Continuing Medical Education (CME) credits were offered through the SHM to registered participants, starting in July 2016.⁷ All sessions were cosponsored by the American Board of Internal Medicine (ABIM) Foundation and the Costs of Care Organization, a non-profit organization aimed at improving healthcare value.

Twitter Metrics

After each session, the following Twitter-based engagement metrics were obtained using the Symplur^© Healthcare Hashtag project;⁸ total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.

Altmetrics and Page Views

As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score⁹, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.

Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.

Thematic Analysis

In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).

Participant and Author Experience

To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey^®) meant to assess the authors’ experience with the journal club (Appendix).

Between October 2015 and November 2017, a total of 15 sessions were held with a mean of 2.17 (±0.583) million impressions/session, 499 (±129) total tweets/session, and 73 (±24) participants/session (compared to a range of 21-58 participants/session from other online journal clubs, where reported) with 7.2 (±2.0) tweets/participant (Table 1). The total number of participants for all sessions was 1096. Participating authors had on average 1389 (±2714) followers, ranging from a low of 37 to a high of 10,376 (Appendix). No correlation between author following and number of participants (r = 0.19), impressions (r = 0.05), or change in Altmetric score (r = 0.17) was seen.

Thematic analysis revealed 3 predominant themes among the chosen articles: Value-based care (VBC), Quality and Patient Safety (QPS), and Medical Education (ME). Articles focused on VBC had the greatest number of impressions (mean ±SD: 2.61 ± 0.55 million) and participants (mean ±SD: 90 ± 12), while QPS articles had the fewest impressions (mean ±SD: 1.71 ± 0.59 million) and number of participants (mean ±SD: 47 ± 16). The mean increase in the Altmetric score among all discussed articles was 14 (±12), from an average baseline of 30 (±37). Medical Education-themed articles appeared to garner the greatest increase in Altmetric scores, averaging an increase of 32 points, compared with an average baseline score of 31 (±32). In contrast, VBC and QPS articles averaged an increase of 8.6 and 8.4 points, from average baselines of 55 (±53) and 17 (±13), respectively. A 2-month analysis of JHM articles not included in these discussions, in which Altmetric scores were measured in the same way as those from the discussion, revealed a baseline Altmetric score of 27 (±24) with an average increase of 8 (±6) 2 weeks following the chat.

Four articles met the inclusion criteria for page view analysis and suggested that article page views increased to similar levels as the eTOC release (mean: 2668 vs. 2998, respectively; P = .35) (Figure). These increases equate to a 33% and 50% increase in average daily page views (2002) for the chat and eTOC release, respectively.

The Twitter-based journal club #JHMChat appears to increase social media awareness and dissemination of journal articles and was considered a useful engagement platform by both authors and participants.

Articles with a focus on VBC and ME had the greatest impact on dissemination metrics, particularly, total impressions and Altmetric scores, respectively. Given the strong presence and interest in these topics within Twitter and social media, these findings are not surprising.^10,11 For example, over the past several years, the VBC movement has taken shape and grown alongside the expansion of social media, thus giving a space for this community to grow and engage. Of note, the cosponsorship relationship with the ABIM Foundation (which works closely with the Choosing Wisely™ campaign) and the Costs of Care Organization could have influenced the participation and dissemination rates of VBC articles. Medical education articles were also popular and appeared to have increased uptake after chats, based on their Altmetric scores. This may be explained by the fact that medical educators have long utilized social media as a means to connect and engage within their community.^12–14 It is also possible that the use of Twitter by trainees (residents, students) may have driven some of the dissemination of ME articles, as this group may not be regular subscribers to JHM.

Online journal clubs offer distinct advantages over traditional in-person journal clubs. First, online journal clubs allow for increased connectivity among online communities, bringing together participants from different geographic areas with diverse training and clinical experiences. Subsequently, this allows for the rapid exchange of both personal and organizational approaches to the topic of discussion.^15–17 Second, online journal clubs allow for continual access to the discussion material. For example, while the metrics used in this study only assessed active, synchronous participation, anecdotal evidence and feedback to the authors suggests that many individuals passively engaged by following along or reviewed the chat feed post hoc at their convenience. This asynchronous access is a quality not found in more traditional journal club formats. Finally, because online journal clubs commonly operate with a flattened hierarchy,¹⁸ they can break down access barriers to both the researchers who performed the study and thought leaders who commonly participate.¹⁷

Several insightful lessons were gleaned in the production and management of this online journal club. On the implementation side, promotion, preparation, and continued organization of an online journal club requires a fair amount of work. In this case, the required time and resources were provided by 2 social media editors in addition to administrative assistance from the SHM. The high attrition rate of online journal clubs over the years attests to these difficulties.²⁴ Additionally, finding incentives to attract and sustain participation can be difficult, as we noted that neither CME nor author popularity (based on their Twitter following) appeared to influence engagement metrics (number of participants, total tweets, and tweets/participant). We also found that partnering with other journal club communities, in particular #NephJC, lead to greater participation rates and impressions. Thus, leveraging connections and topics that span clinical domains may be one way to improve and broaden engagement within these forums. Finally, feedback from participants revealed that the timing of the journal club and the inability to have in-depth discussions, a characteristic commonly associated with traditional journal clubs, were problematic.

This study has several limitations. First, the metrics used to assess social media engagement and dissemination can be easily skewed. For instance, the activity of 1 or 2 individuals with large followings can dramatically influence the number of impressions, giving a falsely elevated sense of broad dissemination. Conversely, there may have been some participants who did not use the #JHMChat hashtag, thus leading to an underestimation in these metrics. Second, while we report total impressions as a measure of dissemination, this metric represents possible interactions and does not guarantee interaction or visualization of that tweet. Additionally, we were unable to characterize our participants and their participation rates over time, as this information is not made available through Symplur© analytics. Third, our page view assessment was limited to 2016 sessions only; therefore, these data may not be an accurate reflection of the impact of #JHMChat on this metric. Fourth, given the marginal response rate to our CME questionnaire, a selection bias could have occurred. Finally, whether social media discussions such as online journal clubs act as leading indicators for future citations remains unclear, as some research has shown an association between increased Altmetric scores and increased citation rates,^19-21 while others have not.^22,23 Our study was not equipped to assess this correlation.

Online journal clubs create new opportunities to connect, engage, and disseminate medical research. These developing forums provide journal editors, researchers, patients, and clinicians with a means to connect and discuss research in ways that were not previously possible. In order to continue to evolve and grow, future research in online journal clubs should explore the downstream effects on citation rates, clinical uptake, and participant knowledge after the sessions.

Acknowledgments

The authors would like to thank Felicia Steele for her assistance in organizing and promoting the chats. Additionally, the authors would like to thank all the authors, guests and participants who took time from their families, work, and daily lives to participate in these activities. Your time and presence were truly appreciated.

Disclosures

The authors of this article operate as the Social Media Editors (C.M.W., V.M.A.) and the Editor-in-Chief (A.A.) for the Journal of Hospital Medicine. Dr. Wray had full access to all the data in the project, takes responsibility for the integrity of the data, and the accuracy of the data analysis.