Summary
This study demonstrates that an AI-powered framework can effectively cross-check anterior cruciate ligament (ACL) tear reports from public football databases with high specificity, reducing manual verification efforts and improving the reliability of sports medicine research.
Abstract
Introduction
There is an exciting opportunity in using public databases, such as Transfermarkt.com, which contains rich football players’ and game statistics, including injuries, for sports medicine research. However, the validity of research based on data retrieved from such public databases has been questioned. Although Transfermarkt.com is 89% accurate in injury denomination and location, concerns about misdiagnosed or mislabeled cases persist. Injury verification, involving manual review of third-party sources such as news reports, is a critical yet time-consuming task to ensure data accuracy. Notwithstanding, manually reviewing thousands of websites for frequent, high-profile injuries like anterior cruciate ligament (ACL) tears can lead to errors and inconsistencies. We hypothesized that an AI-powered framework could cross-check ACL tear-related information from large, publicly available datasets with high specificity and efficiency.
Materials And Methods
A database of ACL tears in male professional footballers from first- and second-tier leagues worldwide (1999-2024) was collected from Transfermarkt.com. The AI-powered framework uses Google Programmable Search Engine to search a curated, multilingual list of websites, including official club/federation pages and reputable news outlets. It employs OpenAI’s GPT to translate search queries, appraise search results, and analyze injury-related information in search result items (SRIs). Injury-related information includes mentions of an ACL rupture, whether it was total or partial, associated lesions (e.g., meniscal or collateral ligament injuries), and whether the athlete had undergone or was scheduled for surgery. Specificity was the target performance metric, and search result item was the evaluation unit. After preliminary testing, 1,522 SRIs were estimated to confirm a 99% specificity for a power of 99.5% and at a 5% significance level. A researcher extracted injury-related information from each SRI’s text, serving as ground truth. Player age at injury, time until return-to-play, and GPT processing time and cost were also recorded.
Results
From Transfermarkt.com, 6,232 athletes with cruciate ligament injuries were identified. After excluding incomplete or incorrect data, 2,907 athletes remained. Verification of 231 athletes yielded 1,546 SRIs. Human analysis of the SRIs showed that 335 mentioned an ACL rupture, 6 mentioned partial injuries, 51 noted associated lesions, and 107 indicated the athlete had or was awaiting surgery. These correspond to 83 athletes with ACL tears, 3 with partial injuries, 20 with associated lesions, and 52 who had or were awaiting surgery. The GPT’s specificity/sensitivity in identifying mentions of ACL tears in a player was 99.3%/88.4%. For identifying mentions of partial injuries, associated lesions, and surgery, the specificity/sensitivity was, respectively, 99.9/66.7%, 98.9/84.3%, and 98.4/76.6%. The mean age at rupture was 26.6 years (SD: 4.6, 95% CI: 25.6-27.6), and the median return-to-play time was 225 days (IQR: 96, 95% CI: 209-251), which is comparable to official data in the literature. GPT processing took 41 minutes and cost around 5 USD.
Conclusion
This study shows that an AI-powered framework can achieve high specificity in cross-checking ACL tear reports from public databases, significantly reducing manual workload and enhancing the reliability of sports medicine research using large public datasets. Future research will explore its application to other injuries and sports and expand its capabilities.