Introduction
In ulcerative colitis (UC) clinical trials, endoscopic disease activity is commonly assessed via central reading of the overall Mayo Endoscopic Score (MES). Segmental MES can provide a more detailed evaluation of mucosal response, but it is limited by high inter-rater variability in both anatomical landmark localization and disease activity grading. Automatic MES grading can assist reducing reader discrepancies and detect local mucosal changes that may not be estimated by global scoring. In this study, we developed and evaluated an artificial intelligence (AI)-based system for automated segmental MES grading enabling disease extent quantification and granular endoscopic remission estimation.
Aims & Methods
In contrast to previous approaches, the proposed AI system introduces a novel hierarchical framework for automated MES grading at the frame-, segment-, and video-level. The system comprises three modules, trained on 5,897 retrospectively acquired, aggregated, and anonymized endoscopic videos. A foundation model (FM), trained on 22 million frames, was developed to extract clinically relevant frames from full-length colonoscopies. A colon segmentation module was built upon the FM to localize three anatomical segments: descending colon (DC), sigmoid colon (SI), and rectum (RM). An automated MES classification module was then trained to perform frame-level MES grading. Hierarchical aggregation of local scores, combined with temporal smoothing and non-informative filtering, enabled both segmental and global MES estimation. For independent testing, 915 colonoscopy videos were annotated for each segment for MES scoring by expert gastroenterologists using a 2+1 reading paradigm. Colon segment reference annotations were automatically extracted from endoscopic videos based on recording software annotations. System performance was evaluated using the Quadratic Weighted Kappa (QWK) for both segmental and overall MES agreement. Classification accuracy in detecting endoscopic granular remission (MES 0, 1 vs 2, 3) was reported for all colon segments.
Results
The AI system demonstrated strong agreement (QWK=0.752 [95% CI: 0.719, 0.782]) with gastroenterologist assessments for overall MES on the test dataset. Good agreement was confirmed for SI (QWK=0.647 [95% CI: 0.602, 0.687]) and RM (QWK=0.687 [95% CI: 0.646, 0.724]), moderate agreement was found for DC (QWK=0.592 [95% CI: 0.541, 0.638]). Frame-level colon segment classification accuracy was 41% for DC, 69% for SI, and 78% for RM; with lower performance in DC linked to high variability in reference annotations. Colon segmentation enabled a 37.72% reduction in mean colonoscopy video time by filtering out-of-body and insertion-phase sections. The system showed very high accuracy in detecting both overall (ACC=90.38%) and segmental (DC: 78.80%; SI: 82.84%; RM: 85.03%) endoscopic remission. Mean accuracy in detecting the number of colon segments per video with active disease was 59.13%.
Conclusion
We introduced a novel AI system for automated, granular MES scoring in ulcerative colitis patients, hierarchically grading colonoscopy videos. Our framework demonstrated strong agreement with gastroenterologists in both overall and segmental endoscopic activity assessments. By integrating continuous scoring with anatomical landmarking, the AI system can offer a more comprehensive view of total disease burden in UC patients which can lead to improved precision of disease estimation and treatment response evaluation in clinical settings.
Disclosure
All authors are employees of Clario Inc., Philadelphia, USA.
