A COMMERCIALLY available AI algorithm has been shown to improve the performance of junior radiologists when grading knee osteoarthritis (KOA), in a recently published study. Previously, conflicting findings in the literature on the topic have led to concerns about a lack of objectivity in grading KOA, which researchers aimed to address through the use of AI.
The team performed a retrospective observer performance study, in which consecutive standing knee radiographs were collected from patients with suspected KOA across three European centres between April 2019 and May 2022. Each centre recruited four readers across radiology and orthopaedic surgery at in-training and board-certified experience levels. Kellgren-Lawrence (KL) grading (KL-0 = no KOA, KL-4 = severe KOA) on the frontal view was assessed by readers with and without assistance from a commercial AI tool. The majority vote of three musculoskeletal radiology consultants established the reference standard. The ordinal receiver operating characteristic method was used to estimate grading performance. Light kappa was used to estimate interrater agreement, and bootstrapped t statistics were used to compare groups.
In total, 75 studies were included from each centre, making for a total 225 studies (mean patient age, 55 years ± 15 [SD]; 113 female patients). The KL grades were KL-0, 24.0% (n = 54); KL-1, 28.0% (n = 63); KL-2, 21.8% (n = 49); KL-3, 18.7% (n = 42); and KL-4, 7.6% (n = 17). Eleven readers completed their readings. The research team found that three of the six junior readers showed higher KL grading performance with versus without AI assistance (area under the receiver operating characteristic curve, 0.81 ± 0.017 [SEM] vs 0.88 ± 0.011 [P < 0.001]; 0.76 ± 0.018 vs 0.86 ± 0.013 [P < 0.001]; and 0.89 ± 0.011 vs 0.91 ± 0.009 [P = 0.008]). Interobserver agreement for KL grading among all readers was higher with versus without AI assistance (κ = 0.77 ± 0.018 [SEM] vs 0.85 ± 0.013; P < 0.001). Board-certified radiologists achieved almost perfect agreement for KL grading when assisted by AI (κ = 0.90 ± 0.01), which was higher than that achieved by the reference readers independently (κ = 0.84 ± 0.017; P = 0.01).
Researchers concluded that AI assistance increased junior readers’ radiographic KOA grading performance and increased interobserver agreement for osteoarthritis grading across all readers and experience levels. “AI-assisted grading could enhance patient inclusion consistency in pragmatic randomized clinical trials and will be important as the Kellgren-Lawrence grading system is increasingly used in selecting patient candidacy for knee arthroplasty,” the researchers commented.
Victoria Antoniou, EMJ
Reference
Brejnebøl MW et al. Interobserver agreement and performance of concurrent AI assistance for radiographic evaluation of knee osteoarthritis. Radiology. 2024;312(1):e233341.
