Summary

Publicly available online shoulder arthroscopy patient education materials scored poorly with respect to readability, understandability, and actionability.

Abstract

Introduction

The National Institutes of Health (NIH) and American Medical Association (AMA) strongly recommend that online health information be written at a 6th grade or lower reading level in order for the average adult in the United States to fully comprehend the material. Syntax reading grade-level has been previously utilized to evaluate online patient education material. The Patient Education Materials Assessment Tool (PEMAT-P) is a valid and reliable method to measure whether text is written such that readers can process key information (understandability) or identify actions to take (actionability). The goal of the study was to evaluate online resources regarding shoulder arthroscopy utilizing measures of readability, understandability, and actionability. Current online resources were hypothesized to score poorly on these objective measures. Additionally, it was anticipated that these measures would not correlate with the order of the listed search results (rank).

Methods

An online search utilizing the term “shoulder arthroscopy” was performed (Google.com) by two independent authors. From the top 50 search results, websites directed at educating patients regarding shoulder arthroscopy were included. News articles, non-text material (audiovisual), journal articles, industry websites, and unrelated websites were excluded. The reading grade level (readability) of included resources was calculated using objective algorithms: Flesch-Kincaid Grade-Level (FKGL), Simple Measure of Gobbledygook (SMOG) grade, Coleman-Liau Index (CLI), and Gunning-Fog Index (GFI). The PEMAT-P form was used to assess understandability and actionability. A 70% PEMAT score indicates acceptable understandability or actionability. Scores were compared across institution types (academic, private practice, and commercial health publishers). The relationship between a website’s average Google search rank and its readability, understandability, and actionability were also calculated.

Results

Two independent searches yielded 53 unique websites. A total of 44 websites (83.02%) met inclusion criteria. The mean FKGL was 11.68±2.78, corresponding to upper high school reading level. No mean readability score performed below a high school sophomore (10th grade) reading level. Only one website scored at or below 6th grade reading level. Mean understandability and actionability scores were 63.02%±12.09 and 29.77%±20.63, neither of which met the PEMAT threshold of 70%. A total of 12 (27.27%) websites met or exceeded the PEMAT score threshold for understandability, while no (0%) website met the minimum threshold for actionability. The three institution categories scored similarly in understandability including 61.71%, 62.68%, and 63.67% for academics, private practice, and commercial health publishers respectively (p=0.9536). Although commercial health publishers scored highest in actionability (43.75%), it was well below the acceptable PEMAT threshold. No readability (p= -0.1628 – 0.0499), understandability (p= -0.0475), nor actionability (p= -0.0268) score was significantly associated with search result rank.

Conclusion

Overall, publicly available online shoulder arthroscopy patient education materials scored poorly with respect to readability, understandability, and actionability. Only one website scored at the NIH and AMA recommended reading level. Only 27.27% of websites scored above the 70% PEMAT score for understandability and none met the threshold for acceptable actionability. Understandability did not differ between institution types. If online resources are misunderstood due to poor readability, understandability, or actionability, these resources may lead to poor decision making by patients. Future efforts should be made to improve online resources in order to optimize patient knowledge and facilitate informed decision-making.