Take-Home Points
- The Internet is a popular resource for orthopedic fellowship applicants.
- 86% of OTF websites are accessible from Google and FREIDA.
- Accessible websites feature only 40% of fellowship applicant content.
- Accessibility and content of OTF websites are highly variable and largely deficient.
- Improvement of the accessibility and content of website information should be a future focus of OTF programs.
The Orthopaedic Trauma Fellowship Match facilitates the matching process for orthopedic residency graduates pursuing a career as orthopedic traumatologists. This match is supported by the Orthopaedic Trauma Association (OTA) and the San Francisco Matching Program (SFMP). Orthopedic trauma fellowship (OTF) programs are accredited by the OTA and may receive oversight by the American Council for Graduate Medical Education (ACGME), which defines uniform standards for fellowship training. 1
Studies have found that the internet is an important and popular resource for applicants researching residency and fellowship programs. 2-5 For many applicants, the internet is their initial and main source of information. 5 Unfortunately, training programs do not have standardized website accessibility and content.
Few studies have addressed online content on orthopedic fellowship programs, 4,6,7 and to our knowledge no one has studied online content on OTF programs. We conducted a study to assess the accessibility and ease of navigation of OTF websites and to evaluate the content on these sites. We wanted to identify content that applicants may reliably expect on OTF sites. Any deficits identified may be useful to fellowship programs and program directors interested in improving website quality. We hypothesized that the accessibility and content of online OTF content would be highly variable and largely deficient.
Methods
This study was conducted at New York University Hospital for Joint Diseases. On February 5, 2015, both the OTA database 8 and the Fellowship and Residency Electronic Interactive Database (FREIDA) 9 were accessed in order to create a comprehensive list of OTF programs. FREIDA, a catalog of all ACGME-accredited graduate medical education programs in the United States, is supported by the American Medical Association and provides cursory program information, including training program duration and number of positions per year.
The databases were reviewed for links to OTF program websites. An independent Google search for program websites was also initiated on February 5, 2015. The Google search was performed in the format “program name + orthopaedic trauma fellowship” to assess how accessible the program sites are from outside the 2 databases (OTA, FREIDA). Google was used because it is the most commonly used search engine. 10 The first 25 search results were reviewed for links to OTF websites. Programs without accessible links to OTF websites—from the OTA database, from FREIDA, or from the Google search—were excluded from content assessment.
Accessible websites were electronically captured to ensure consistency of content during assessment. OTF site content was evaluated using methods described in similar investigations. 4,5,11,12 In our dichotomous assessment of fellow education content, we awarded 1 point per content item on the website. The 10 education content items evaluated were call responsibilities, didactic instruction, journal club, research requirements, evaluation criteria, rotation schedule, operative experience, office/clinic experience, meetings attended, and courses attended. We also performed a dichotomous assessment of fellow recruitment content. The 10 recruitment content items evaluated were program description, application requirements, selection criteria, OTA link, SFMP link, location description, program contact information, fellow listing, faculty listing, and salary. Content items were chosen for evaluation on the basis of published OTF applicant experience. 13 Percentages of education content, recruitment content, and total content were compared by program location, number of fellows, ACGME accreditation status, 14 affiliation with a top 20 orthopedic hospital, 15 and affiliation with a top 20 medical school, 16 as in similar studies. 7,17
Chi-square tests were used to compare content by fellowship location, number of fellows, ACGME accreditation status, affiliation with a top 20 orthopedic hospital, and affiliation with a top 20 medical school. For all tests, the significance level was set at P < .05.
Results
Of the 49 OTF programs identified with database queries, 9 appeared in both the OTA database and FREIDA, 39 appeared only in the OTA database, and 1 appeared only in FREIDA. There were 48 programs total in the OTA database and 10 total in FREIDA.
The OTA database had no OTF website links. Of the 10 OTF links in FREIDA, 3 (6%) were nonfunctioning, 6 (12%) had multiple steps for accessing program information, and 1 (2%) connected directly to program information. Therefore, FREIDA had a total of 7 accessible OTF links (14%). The independent Google search yielded website links for 42 (86%) of the 49 OTF programs. Five links (10%, 5/49) had multiple steps for accessing program information, and 37 links (76%, 37/49) connected directly to program information. The 7 OTF links accessible through FREIDA were accessible through Google as well. Table 1 summarizes the accessibility data.
Table 1.
All 42 accessible OTF websites were assessed for content. On average, these sites had 40% (range, 0%-75%) of the total assessed content. Mean (SD) education content score was 3.6 (2.2) out of 10. Operative experience (88%) and research requirements (81%) were the most consistently presented education items. Didactic learning (45%) and description of common office/clinic cases (43%) were next. Less than 5% of the sites had content on the training courses (eg, sponsored fracture courses) attended by fellows. Figure 1 summarizes the education items on the OTF websites.
Figure 1.
Mean (SD) recruitment content score was 4.4 (2.2) out of 10. Program description (93%) and program contact information (88%) were the most consistently presented recruitment items. Clinical faculty (52%) and current and/or prior fellows (36%) were next. Fellow selection criteria appeared least often (12%). Figure 2 summarizes the recruitment items on the OTF websites.
Figure 2.
Thirty-six percent of OTF programs with accessible websites were in the southern United States. However, there were no significant differences in online content between OTF program locations. Websites of programs with >1 fellow had significantly more education content (48% vs 33%; P = .043) and total content (46% vs 37%; P = .01) than websites of programs with 1 fellow. ACGME accreditation status, affiliation with a top 20 orthopedic hospital, and affiliation with a top 20 medical school did not have a significant effect on OTF website content. Table 2 summarizes OTF website content by location, number of fellows, top 20 orthopedic hospital affiliation, and top 20 medical school affiliation.
Table 2.
Discussion
We conducted this study to assess the accessibility of OTF program websites and to evaluate the content of the sites. Our hypothesis, that the accessibility and content of online OTF content would be highly variable and largely deficient, was supported by our findings. We found that the OTA database had no OTF website links and that FREIDA links connected directly to only 2% of OTF sites. The majority of OTF sites were accessed from the Google search, which had direct links to 76% of the OTF programs.
Other studies have had similar findings regarding the accessibility of fellowship websites. Mulcahey and colleagues 6 evaluated sports medicine fellowship websites for accessibility and content, and found that the website of the American Orthopaedic Society for Sports Medicine directly linked to fellowship information for only 3% of programs; a Google search yielded direct links to 71% of program websites. Davidson and colleagues 4 examined the quality and accessibility of online information on pediatric orthopedic fellowships and found no program links on the website of the Pediatric Orthopaedic Society of North America; a Google search yielded direct links to 68% of programs. Silvestre and colleagues 7 assessed spine fellowship information on the Internet. The North American Spine Society website had working links to only 3% of fellowship sites, and FREIDA connected to only 6% of sites.
Content scores in our study were highly variable. Mean education and recruitment content scores were 3.6 (range, 0-9) and 4.4 (range, 0-10), respectively. Operative experience (88%) and program description (93%) were the most frequently presented education and recruitment items, respectively. Consistency in presenting program descriptions on OTF websites was slightly poorer than that in other orthopedic specialties. Sports medicine, pediatric orthopedic, and spine fellowship websites provided program descriptions for fellowship recruitment. 4,6,7 Nevertheless, overall content scores in our study and in the aforementioned studies were similarly poor.
In our study, OTF websites showed no significant differences in content scores for program location, ACGME accreditation status, affiliation with a top 20 orthopedic hospital, or affiliation with a top 20 medical school. Lack of a significant effect of medical school or orthopedic hospital affiliation suggests academic prestige does not play a large role in attempts by OTF websites to attract applicants. However, programs with >1 fellow had significantly more education and total content than programs with 1 fellow. Results from a comparable study support this finding. Silvestre and colleagues 18 assessed the accessibility of online plastic surgery residency content. Programs with 3 or 4 residents had significantly more online education content than programs with 1 resident. This finding may relate to the cost efficiency of developing low-cost websites to attract applicants to multiple positions. 7
Despite lacking links to OTF websites, the OTA database had a large amount of content on 98% (48/49) of OTFs. In addition to presenting the content that we assessed in this study, the OTA database provided the number of inpatient beds at the primary teaching hospital, the annual number of emergency department visits, the annual number of trauma admissions, and the annual number of orthopedic trauma procedures. This standardized information may be very helpful to fellowship applicants and may be an important adjunct to fellowship websites.
FREIDA provided similar content, but accessible links were found for only 14% of the assessed programs. Although the deficiency in accessible OTF links in the OTA database and FREIDA is not well understood, it is important. The results of our study and of similar studies suggest that the listing of active fellowship program links on society websites would benefit orthopedic fellowship applicants, likely fostering a better understanding and a more efficient review of available programs. In addition, links on society websites afford fellowship directors the means to efficiently publicize their programs to large numbers of potential applicants, who likely use society websites as an initial informational resource.
Our study had limitations. First, its findings are subject to the dynamism of the internet, and OTF information may have been updated after this investigation was conducted. Second, our study did not rank-order accessible links, which may have provided more information on the efficiency of using Internet search engines in a review of OTF programs. In addition, our study involved dichotomous assessment of OTF content. Multichotomous evaluation may have further elucidated the quality of website information. Last, our study evaluated websites only for US-based OTF programs. Inclusion of international OTF programs, though outside the scope of this study, may have yielded different findings.
Conclusion
Our results highlight the difficulties that OTF applicants may experience in gathering fellowship information online. OTF website accessibility and content were found to be highly variable and largely deficient. Comparing our findings with those of similar studies revealed that fellowship websites generally provided little information that orthopedic specialty applicants could use. OTF programs should focus on improving their website accessibility and content.