Summary

Evaluation of the interrater reliability among surgeons of varying experience levels and specialty training backgrounds when evaluating skeletal maturity in the distal femur and proximal tibia of children and adolescents with patellofemoral instability.

Abstract

Purpose

Surgical decision-making and pre-operative planning for children and adolescents with patellofemoral instability relies heavily on skeletal maturity status. In order to be clinically useful, radiologic assessments of skeletal maturity must demonstrate acceptable interrater reliability and accuracy. The purpose of this study was to evaluate the interrater reliability among surgeons of varying experience levels and specialty training backgrounds when evaluating skeletal maturity in the distal femur and proximal tibia of children and adolescents with patellofemoral instability.

Methods

Cross-sectional study design was used. Six fellowship-trained orthopedic surgeons (4 pediatric orthopedic, 2 sports medicine) who perform a high volume of patellofemoral instability surgery examined 20 blinded and randomized knee radiographs and MR images. They assessed these images for clinically relevant growth (open physis) or clinically insignificant growth (closing/closed physis) remaining in the distal femoral and proximal tibial physes. Fleiss’s kappa was calculated for each measurement. After initial ratings, raters discussed consensus methods to improve reliability and assessed the images again in order to determine if training and new criteria improved interrater reliability.

Results

Reliability for initial assessments of distal femoral and proximal tibial physeal status was poor (Kappa range: 0.01 – 0.58). After consensus building, all assessments demonstrated almost perfect interrater reliability (Kappa = 0.99 for all measurements).

Conclusion

Surgical decision-making and pre-operative planning for children and adolescents with patellofemoral instability relies heavily on radiologic assessment of skeletal maturity. This study found that initial interrater reliability of physeal status and clinical decision making was unacceptably low. However, with the addition of new criteria, a consensus-building process, and training, these variables became highly reliable.

Significance

Treatment of patellofemoral instability heavily relies on skeletal maturity status. Physeal assessments of the knee must be reliable and clinically relevant to ensure optimal patient care. Lack of assessment reliability can jeopardize patient care, both when determining surgical indications as well as choosing an appropriate procedure based upon skeletal maturity. Physeal assessment of the knee without training can be unreliable among fellowship trained orthopedic surgeons. Surgeons should focus on using reliable imaging metrics in children and adolescents with patellofemoral instability, and measurements that remain unreliable after consensus building and training should be removed from clinical decision-making algorithms. With consensus building and training in the use of the methods described in this manuscript, physeal assessments can demonstrate almost perfect reliability.