Introduction

The use of artificial intelligence (AI) in colonoscopy has already been shown enable a higher polyp detection and characterization. New applications are also being developed at an increasingly high pace, but there are potential pitfalls that might appear when developing new AI methods.

Aims & Methods

The aim of this case study is to show potential pitfalls that can arise during the training of new AI models, most importantly on smaller datasets. We aim to illustrate the hidden pitfalls that might arise during development, aside from known problems such as dataset imbalance and overfitting.
This case study is based on the development of an AI network for differentiating between hyperplastic polyp (HP) subclasses using colonoscopy imagery. The original primary outcome was to determine the feasibility of such a setup, while secondary outcome was to determine the effectiveness of such a model. Finally, the tertiary outcome was to use explainable AI techniques on the trained model to see which elements were picked up during training to correctly identify both HP subclasses. The specific technique used is the SHAP technique [1], which highlights the areas in an image used for the classification.
We retrospectively collected images of 198 polyps with histological assessment of hyperplastic polyps from different colonoscopy procedures. These polyps in these images were manually segmented and the histological slices were re-examined for further subclassification into goblet cell-rich hyperplastic polyps (GCHP) and microvesicular hyperplastic polyps (MVHP). The dataset consists of 144 GCHP, 28 MVHP and 28 mixed MVHP/GCHP polyps, of which only the unmixed polyps were included in this study. The polyps were cropped out of the image with a margin of 50% to take the mucosa immediately surrounding the polyp into account. The total dataset was randomly split into training (70%), validation (10%) and test set (20%). Finally, the imbalanced training and validation datasets were balanced by randomly oversampling the images of MVHP.

Results

We obtained good results for this model, with sensitivity of 90% and 69,9% for MVHP and GCHP respectively and the area under the curve (AUC) was 86,6%. These results can be successfully presented and published, establishing our original primary and secondary objectives. The problem is that when checking the model with explainable AI techniques to achieve our tertiary objective, we noticed a problem. For MVHP, the features in the image that were used, were usually either be the pit pattern or the existence of mucus. The images that were classified as GCHP seemed to be decided using the surrounding mucosa. The problem is that around 40% of the images classified as GCHP seemed to be selected using the black edges of the scope recording. It seems that during training there were only black edges visible in training samples of GCHP, making the model assume that all samples with black edges are GCHP, which is obviously a wrong assumption. A simple solution such as randomly adding black edges in all samples solves this problem, correcting the result to a sensitivity of 70% and 64,2% for MVHP and GCHP respectively and with an AUC of 74,3%, which is slightly lower, but does not have the built-in problem.

Conclusion

In preliminary stages of AI development, it is always important to verify achieved results if possible. A model will try to capture features using all the available training data, but uncorrelated features might exist unintentionally. A good verification method is to open the black box-like AI model using explainable AI techniques.

References

[1] Lundberg, Scott, and Su-In Lee. “A Unified Approach to Interpreting Model Predictions.” arXiv.Org, 2017, https://doi.org/10.48550/arxiv.1705.07874.

Introduction

Recent insights in polyp molecular biology have suggested that hyperplastic polyps (HP) are closely related to sessile serrated lesions (SSL). Their hallmark serrated crypts closely resemble those found in SSL. Furthermore, similar to SSL, specific subtypes of HP can harbor BRAFV600E-mutations. These findings, amongst others, have raised the hypothesis that some HP may have an intrinsic malignant potential through the serrated neoplasia pathway. However, in everyday clinical practice, no routine HP subdifferentiation is performed. Therefore, we aimed to develop a deep learning algorithm to classify molecular subclasses of HP, namely microvesicular hyperplastic polyps (MVHP) and goblet cell-rich hyperplastic polyps (GCHP), based on their endoscopic features.

Aims & Methods

In this single-center trial 238 hyperplastic polyps from 159 patients were retrospectively selected. All polyps had high quality endoscopic video in white light endoscopy and virtual chromoendoscopy (i-scan) using Pentax EC38-i20c and EC34-i10 endoscopes. All histological slides were reviewed by an expert pathologist, updating diagnoses to MVHP, GCHP or mixed HP. We selected the polyps without mixed constitution for development of a deep learning algorithm. These polyps were randomly split into 70% for training, 10% for validation and 20% for testing. Sequences of still images of the polyps were manually annotated and subsequently used to train a deep learning algorithm, with dataset imbalance addressed by randomly oversampling the MVHP.

Results

The 238 HPs were reclassified as 152 GCHP, 59 MVHP and 27 mixed HPs after expert histological review. We subsequently trained a convolutional neural network on the 211 HPs without mixed constitution. This resulted in a model with an AUC of 0.817, corresponding to a sensitivity of 80.0% for predicting MVHP and 78.4% for predicting GCHP. However, due to the low prevalence of MVHP (28%) the positive predictive value for MVHP is only 33.3%, whilst the positive predictive value for GCHP is 96.7%.

Conclusion

We successfully developed a deep learning algorithm capable of differentiating between molecular subclasses of hyperplastic polyps (MVHP vs GCHP) based on their endoscopic features. This model demonstrated a robust AUC of 81.7% with a sensitivity of 80.0% and 78.4% for predicting MVHP and GCHP respectively. However, the low prevalence of MVHP (28%) limited its positive predictive value (33.3%) for predicting MVHP. Despite these challenges, this pilot study serves as a proof of concept that artificial intelligence is capable of predicting hyperplastic polyp subclasses based on endoscopic appearance. Furthermore, it highlights the potential of artificial intelligence to enhance the diagnostic precision, which could contribute to more tailored surveillance and management strategies in the future.

Disclosure

The authors declare no conflicts of interest.

Introduction

Recent insights in polyp molecular biology have suggested that hyperplastic polyps (HP) are closely related to sessile-serrated lesions (SSL). Their hallmark serrated crypts closely resemble those found in SSL. Furthermore, similar to SSL, specific subtypes of HP can harbor BRAFV600E-mutations. These findings, amongst others, have raised the hypothesis that some HP may have an intrinsic malignant potential through the serrated neoplasia pathway. However in everyday clinical practice, no routine HP subdifferentiation is performed.

Aims & Methods

In this single-center trial, we aimed to develop a deep learning algorithm capable of differentiating between the subclasses of hyperplastic polyps, namely microvesicular hyperplastic polyps (MVHP) and goblet cell-rich hyperplastic polyps (GCHP), based on their endoscopic appearance. We retrospectively selected 198 hyperplastic polyps in 127 patients. All polyps had high quality endoscopic video available in white light endoscopy and virtual chromoendoscopy (i-scan) using Pentax EC38-i20c and EC34-i10 endoscopes. We reviewed all histological slides with an expert pathologist and updated their pathology diagnosis using the MVHP/GCHP subdifferentiation. Of the total 198 HP, 144 were GCHP, 26 were MVHP and 28 were mixed GCHP/MVHP. We selected the polyps without mixed constitution for development of a deep learning algorithm. These 170 polyps were randomly split into 70% for training, 10% for validation and 20% for testing. Sequences of still images of the polyps were manually annotated and subsequently used to train a deep learning algorithm. The imbalance in the training dataset was adjusted by randomly oversampling the MVHP.

Results

We trained a deep learning algorithm using a convolutional neural network on 170 HP (144 GCHP and 26 MVHP). This model can differentiate between MVHP and GCHP based on still endoscopic images with an AUC of 0.743, corresponding to a sensitivity of 70% for MVHP. However due to the low prevalence of MVHP (14%) compared to GCHP (86%) the positive predictive value for MVHP is only 24.1%. The specificity (for GCHP) is 64.2%

Conclusion

Our preliminary results show that a deep learning algorithm is capable of detecting MVHP with a good sensitivity (70%) based on their endoscopic appearance. However, due to the low prevalence of MVHP compared to GCHP, the positive predictive value for MVHP is only 24.1%. Moreover, the current algorithm has some difficulties correctly identifying the GCHP, as is shown by the rather low specificity of 64.2%. We plan to further expand the dataset to mitigate these issues, however this pilot study serves as a proof of concept that artificial intelligence is capable of predicting hyperplastic polyp subclasses based on endoscopic appearance.

Disclosure

AJ is supported by a grant of Research Foundation Flanders (1SHD924N).
PR is supported by a Clinical Mandate from Belgian Foundation against Cancer (Stichting tegen Kanker).
RB is supported by a grant of Research Foundation Flanders (G072621N) and an FKO grant of KU Leuven (BOF-FKO 2022). RB received speaker and consultancy fees and research support from Pentax Medical.
The authors of this manuscript have no other conflict of interest to declare.

Introduction

Artificial Intelligence (AI) assisted reading in Small Bowel Capsule Endoscopy (SBCE) has recently been shown to achieve comparable and potentially superior accuracy compared to standard clinician reading. In Colon Capsule Endoscopy (CCE), AI algorithms have also demonstrated promising results. However, the extent of AI-assisted reading's advantage remains unclear, particularly regarding its performance across different polyp sizes, morphologies, locations, and other factors. A deeper understanding of these variables is crucial for optimising AI performance and facilitating its clinical integration.

Aims & Methods

This CESCAIL sub-analysis evaluates the per-polyp diagnostic accuracy of AI-assisted versus standard clinician assessment (readings) and identifies key factors influencing AI-assisted accuracy using AiSPEED^TM, the Computer-Aided Detection (CADe) system.
The CESCAIL study evaluated the diagnostic accuracy of AI-assisted CCE readings using AiSPEED™ compared to the standard readings. Standard readings were manually performed by clinicians, while AI-assisted readings incorporated an AI pre-review followed by clinician validation. A total of 1,803 polyps from 673 patients were analysed at the per-polyp level, focusing on sensitivity and positive predictive value. Factors influencing the improved accuracy of AI-assisted readings over standard clinician readings were also assessed, including polyp size, morphology, location, patient demographics (age and sex), bowel preparation quality, capsule excretion rates, comorbidities, medications, reading time, and video duration.

Results

AI-assisted reading demonstrated significantly higher sensitivity with clear superiority for smaller polyps (<10 mm) compared to larger ones (≥10 mm) (OR 2.27 vs 0.88, p<0.001) (see Table 1). While there was no observed difference in diagnostic accuracy between AI-assisted and standard clinician assessment for polyps ≥10 mm, non-inferiority was established. AI accuracy remained consistent between 6–9 mm and ≤5 mm polyps (p=0.64). The most notable improvement was observed with hyperplastic polyps (HPs) (OR 5.4, p<0.001). No significant differences were identified for pedunculated, sub-pedunculated, Lateral Spreading Tumours, or Sessile Serrated Lesions. Furthermore, AI-assisted readings were significantly more accurate for left-sided polyps than right-sided ones (OR 2.36 vs 1.66, p<0.0001), although AI-assisted reads outperformed standard reads in both locations.

Conclusion

This study highlights the performance and consistency of AI-assisted reading, particularly for detecting smaller adenomas and HPs, with notably improved accuracy in the left colon. Next-generation AI should focus on distinguishing significant from diminutive polyps and enhancing polyp characterisation, especially for right-sided lesions.

References

1. Spada C, Piccirelli S, Hassan C, Ferrari C, Toth E, Gonzalez-Suarez B, et al. AI-assisted capsule endoscopy reading in suspected small bowel bleeding: a multicentre prospective study. Lancet Digit Health. 2024;6(5):e345-e53.

Disclosure

Hagen Wenzek serves as the CEO, Elizabeth White is the Project Manager, and Pablo Laiz is the AI software engineer at Corporate Health International, which owns the intellectual properties of AiSPEED^TM.