Readability of Orthopedic Trauma Patient Education Materials on the Internet

Take-Home Points

• The Flesch-Kincaid Readability Scale is a useful tool in evaluating the readability of PEMs.
• Only 1 article analyzed in our study was below a sixth-grade readability level.
• Coauthorship of PEMs with other subspecialty groups had no effect on readability.
• Poor health literacy has been associated with poor health outcomes.
• Efforts must be undertaken to make PEMs more readable across medical subspecialties.

Patients increasingly turn to the Internet to self-educate about orthopedic conditions.^1,2 Accordingly, the Internet has become a valuable tool in maintaining effective physician-patient communication.^3-5 Given the Internet’s importance as a medium for conveying patient information, it is important that orthopedic patient education materials (PEMs) on the Internet provide high-quality information that is easily read by the target patient population. Unfortunately, studies have found that many of the Internet’s orthopedic PEMs have been neither of high quality^6-8 nor presented such that they are easy for patients to read and comprehend.^1,9-12

Readability, which is the reading comprehension level (school grade level) a person must have to understand written materials, is determined by systematic formulae¹²; readability levels correlate with the ability to comprehend written information.² Studies have consistently found that orthopedic PEMs are written at readability levels too high for the average patient to understand.^1,9,13 The readability of PEMs in orthopedics as a whole⁹ and within the orthopedic subspecialties of arthroplasty,¹ foot and ankle surgery,² sports medicine,¹² and spine surgery¹³ has been evaluated, but so far there has been no evaluation of PEMs in orthopedic trauma (OT).

We conducted a study to assess the readability of OT-PEMs available online from the American Academy of Orthopaedic Surgeons (AAOS) in conjunction with the Orthopaedic Trauma Association (OTA) and other orthopedic subspecialty societies. We hypothesized the readability levels of these OT-PEMs would be above the level (sixth to eighth grade) recommended by several healthcare organizations, including the Centers for Disease Control and Prevention.^9,11,14 We also assessed the effect that orthopedic subspecialty coauthorship has on PEM readability.

Methods

In July 2014, we searched the AAOS online patient education library (Broken Bones & Injuries section, http://orthoinfo.aaos.org/menus/injury.cfm) and the AAOS OrthoPortal website (Trauma section, http://pubsearch.aaos.org/search?q=trauma&client=OrthoInfo&site=PATIENT&output=xml_no_dtd&proxystylesheet=OrthoInfo&filter=0) for all relevant OT-PEMs. Although OTA does not publish its own PEMs on its website, it coauthored several of the articles in the AAOS patient education library. Other subspecialty organizations, including the American Orthopaedic Society for Sports Medicine (AOSSM), the American Society for Surgery of the Hand (ASSH), the Pediatric Orthopaedic Society of North America (POSNA), the American Shoulder and Elbow Surgeons (ASES), the American Association of Hip and Knee Surgeons (AAHKS), and the American Orthopaedic Foot and Ankle Society (AOFAS), coauthored several of these online OT-PEMs as well.

Using the technique described by Badarudeen and Sabharwal,¹⁰ we saved all articles to be included in the study as separate Microsoft Word 2011 files. We saved them in plain-text format to remove any HTML tags and any other hidden formatting that might affect readability results. Then we edited them to remove elements that might affect readability result accuracy—deleted article topic–unrelated information (eg, copyright notice, disclaimers, author information) and all numerals, decimal points, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes.¹⁰Mr. Mohan used the Flesch-Kincaid (FK) Readability Scale to calculate grade level for each article. Microsoft Word 2011 was used as described in other investigations of orthopedic PEM readability^2,10,12,13: Its readability function is enabled by going to the Tools tab and then to the Spelling & Grammar tool, where the “Show readability statistics” option is selected.¹⁰ Readability scores are calculated with the Spelling & Grammar tool; the readability score is displayed after completion of the spelling-and-grammar check. The formula used to calculate FK grade level is¹⁵: (0.39 × average number of words per sentence) + (11.8 × average number of syllables per word) – 15.59.

Statistical Analysis

Descriptive statistics, including means and 95% confidence intervals (CIs), were calculated for the FK grade levels. Student t tests were used to compare average FK grade levels of articles written exclusively by AAOS with those of articles coauthored by AAOS and other orthopedic subspecialty societies. A 2-sample unequal-variance t test was used, and significance was set at P < .05. Total number of articles written at or below the sixth- and eighth-grade levels, the reading levels recommended for PEMs, were tabulated.^1,9-12 Intraobserver and interobserver reliabilities were calculated with intraclass correlation coefficients (ICCs): Mr. Mohan, who calculated the FK scores earlier, now 1 week later calculated the readability levels of 15 randomly selected articles^10,11; in addition, Mr. Mohan and Dr. Yi independently calculated the readability levels of 30 randomly selected articles.^10,11 The same method described earlier—edit plain-text files, then use Microsoft Word to obtain FK scores—was again used. ICCs of 0 to 0.24 correspond to poor correlation; 0.25 to 0.49, low correlation; 0.5 to 0.69, fair correlation; 0.7 to 0.89, good correlation; and 0.9 to 1.0, excellent correlation.^10,11 All statistical analyses were performed with Microsoft Excel 2011 and VassarStats (http://vassarstats.net/tu.html).