Introduction

Automatic optical characterization (CADx) systems aim to support gastroenterologists in identifying the type of polyps found during a colonoscopy procedure (e.g. “adenomatous” or “non adenomatous”). It has the potential to decrease the risks, workloads, and costs of polypectomy and pathology tests. The majority of reported CADx systems provide a classification score for each polyp frame, followed by an aggregation of the scores to determine a final prediction of the polyp diagnosis. A common aggregation mechanism is a simple voting procedure. Other aggregation methods include the average or maximum score of the entire sequence. In all of the above, longer sequences should provide more information necessary to get an accurate prediction. However, the aggregation approach does not leverage complementary information (e.g. polyp geometry) that can be learned by jointly processing multiple views. In addition, views diversity and quality are not considered as part of the inference process. At the aggregation stage, only the per-frame score is taken into account, while the appearance of the polyp in each view is not considered. Joint processing of multiple frames, on the other hand, allows relative weighing of each frame with respect to the other available frames in the sequence, while considering their quality and diversity.

Aims & Methods

We propose a multiview learning model, in which multiple frames of a polyp are processed jointly as part of the training process, resulting in a single score for the entire sequence. We apply a convolutional neural network (CNN) to embed each frame into a vector representation. We then use an attention model (Transformer) to jointly process these representations and predict a single score for the entire sequence, rather than for each frame separately. The system is trained in an end-to-end manner, allowing the model to learn to predict a score based on multiple views with varying quality and diversity.
Here we test the performance of the proposed approach in predicting the endoscopic appearance-based polyp classification done by GIs. Predicting the histology results is left for future work. The polyps are annotated by expert GIs into adenomatous or non adenomatous. The test set consists of 565 polyps taken from 317 videos that were unanimously classified by 2 expert GIs, independently. We compare our suggested model to a baseline model which was trained to predict a score for each frame separately, followed by aggregation of the scores for each polyp.

Results

We compare the AUROC of the joint learning model ("Joint Learning") and 3 baseline aggregation schemes - averaging the per-frame scores ("Mean Score"), maximum score ("Maximum Score"), and majority voting ("Voting"). We use polyp frames sequences of varying lengths and evaluate the performance repeatedly on 10 random selections of frames from each sequence. We report the mean and STD of the AUROC for each sequence length. As seen in the table, the joint learning model outperforms all baseline aggregation schemes.

Aggregation Method \ Sequence Length (# Frames)	1	10	20	30	40
Joint Learning	0.913 ± 0.007	0.959 ± 0.003	0.962 ± 0.003	0.962 ± 0.002	0.961 ± 0.001
Mean Score	0.907 ± 0.013	0.952 ± 0.003	0.955 ± 0.003	0.955 ± 0.002	0.956 ± 0.002
Maximum Score	0.907 ± 0.015	0.954 ± 0.004	0.957 ± 0.001	0.956 ± 0.003	0.956 ± 0.002
Voting	0.831 ± 0.022	0.937 ± 0.007	0.945 ± 0.044	0.946 ± 0.002	0.948 ± 0.002

Conclusion

Joint learning of polyp appearance in multiple frames results in better performance for polyp classification in colonoscopy. The joint learning process allows the model to consider diversity, quality and complementary information from multiple views of a single polyp, rather than process each frames separately.

References

Hassan, Cesare, et al. "Artificial intelligence allows leaving-in-situ colorectal polyps." Clinical Gastroenterology and Hepatology 20.11 (2022): 2505-2513.
Mori, Yuichi, et al. "Benefits and challenges in implementation of artificial intelligence in colonoscopy: World Endoscopy Organization position statement." Digestive Endoscopy (2023).
Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
Dosovitskiy, Alexey, et al. "An image is worth 16x16 words: Transformers for image recognition at scale." arXiv preprint arXiv:2010.11929 (2020).

Disclosure

The authors are affiliated with Verily (@verily.com) and Google (@google.com).

Introduction

Colorectal neuroendocrine tumors (NETs) and polyps have similar endoscopic manifestations and are easily misidentified. Due to the low incidence of NETs, it is difficult to obtain adequate images to fit deep-learning models. This study introduced a few-shot learning (FSL) model for the ternary classification of NETs and polyps on endoscopic images.

Aims & Methods

Three classes of endoscopic images (56 serrated lesions and polyps [SLPs], 86 traditional adenomas and 53 NETs) were collected from three datasets. First, three images from each class were randomly selected for Support Set (SS), while others for Query Set (QS). Second, ResNet50 V2 was pretrained based on ImageNet and then on esophageal endoscopic images Third, imaging eigenvectors were extracted based on a dual-pretrained model. Fourth, the Euclidean distances to the nine SS images were calculated for each QS image. Finally, a k-nearest neighbor classifier was used for classification based on Euclidean distances. The proposed FSL model was tested in three rounds.

Results

The performance of the FSL model (macro-area under the curve [AUC], 0.731; macro F1-score, 0.674; Matthews Correlation Coefficient [MCC], 0.526; Kappa, 0.523) was better than that of the traditional ResNet50 V2, junior endoscopist, and senior endoscopist. When identifying NETs, the proposed FSL model achieved the highest accuracy 0.823, sensitivity 0.653, precision 0.673 and F1-score 0.659, whereas the senior endoscopist achieved the highest specificity 0.904.

Conclusion

This study presented a FSL method based on transfer learning and metric learning to distinguish between SLPs, adenoma, and NETs on endoscopic images. This suggests that FSL might be an effective method for computer-aided diagnosis of rare diseases.

Introduction

Colonoscopy is a promising diagnostic procedure for removal colorectal neoplasia, which can help prevent colorectal cancer. With an increasing trend towards colorectal cancer prevention, the number of colonoscopies and polypectomies is increasing, with approximately 11 million colonoscopy procedures performed annually in the United States alone. This has resulted in a significant burden on physicians and healthcare resources. Previous studies have demonstrated the utility of computer-aided diagnosis (CADx) systems in improving the characterization of colorectal polyps. However, the time-saving aspect of implementing AI-based CADx systems has not been thoroughly investigated. In this study, we aim to evaluate the potential of CADx systems to reduce the time required for polyp characterization.

Aims & Methods

This was a single center, retrospective study, was conducted at a university hospital between July 2021 and January 2022. We randomly extracted pathologically confirmed 100 polyps and one representative NBI image was prepared for each lesion. Three non-experts (a resident with no endoscopic experience, a gastroenterologist with one year of experience, and a gastroenterologist with two years of experience) were included. Randomly selected NBI images were shown separately to these doctors separately using laptop computer. They diagnosed each lesion diagnosed each lesion as neoplastic or non-neoplastic. For this study, we considered sessile serrated lesions (SSLs) as neoplasms. In the first examination, non-experts diagnosed without any assistance. After a four-week interval, a second examination was conducted with the assistance of a CADx system that provided a pathological prediction (neoplastic or non-neoplastic). We recorded the time required for diagnosing each polyp and compared it between the first and second examinations. The outcomes were, difference in diagnostic time and diagnostic accuracy of the endoscopists with or without CADx assistance. Wilcoxon signed rank test and McNemar test were used.

Results

Out of the 100 lesions, there were 72 adenomas/intramucosal carcinomas, 6 SSLs, 21 hyperplastic polyps, and 1 juvenile polyp. With the use of CADx, the mean (±SD) diagnostic time was significantly shorter than without CADx assistance (10.7±8.2 sec vs 5.3±3.6 sec, P value < 0.001). Regarding the diagnostic performance for differentiating neoplasms from non-neoplasms, diagnostic accuracy was significantly higher in with CADx assistance (75%　[95%CI 69.7-79.8] vs 92.3% [95%CI 88.7-95.1], P value < 0.001). Assuming the mean number of polyps per patient as 1.87 (Taku Sakamoto, et al. United European Gastroenterol J, 2019), time saving effect was estimated as 17 min / 100 colonoscopies.

Conclusion

Our study demonstrates that the utilization of the CADx can save diagnosing time and increase accuracy for non-experts. Therefore, we suggest that the CADx has the potential to decrease examination time by expediting the diagnosis of lesions.

Introduction

Optical diagnosis of colorectal polyps is challenging. Artificial intelligence based systems allow the detection and histological characterization of polyps.

Aims & Methods

The aim of this study is to compare the diagnostic accuracy of Polydeep, a CADe/x system with expert endoscopists for the optical diagnosis of colorectal polyps.
We designed a diagnostic test study based on the evaluation of 2364 polyps still images with confirmed histological diagnosis (69.12% adenomas, 14.8% Traditional Serrated Adenoma, TSA, or Serrated Sessile Lesion, SSL, 16.7% hyperplastic). The lesions was classificated as neoplastic (adenomatous, TSA, SSL) and no neoplastic (hyperplastic lesions). The images were shown to Polydeep and to the expert endoscopists to obtain the predicted histology. The endoscopists classified the lesions as adenoma, TSA, SSL or hyperplastic lesions, while PolyDeep classified the lesions as neoplastic and non-neoplastic. We calculated sensitivity, specificity, predictive values, likelihood ratios, diagnostic odds ratio and the Area Under the Curve (AUC). We compared sensitivity and specificity using the McNemar test and the AUC using the homogeneity areas test.

Results

PolyDeep classified 77.62% of evaluated lesions as neoplastic and the endoscopists classified 87.1% of the lesions as neoplastic (adenomas: 75.76%, TSA: 3.86%, SSL: 7.55%). The diagnostic accuracy results as well as the comparison between PolyDeep and the endoscopists for the neoplastic category is shown in the attached table.

	Endoscopists	PolyDeep	P-value
Sensitivity	91.56%	90.34%	<0.001
Specificity	35.23%	35.31%	0.355
PPV	88.21%	88.72%
NPV	44.07%	39.37%
LR +	1.41	1.39
LR -	0.24	0.27
OR	5.89	5.11
AUC	0.634	0.628	0.91

Table 1: The table show the comparison of diagnostic test between endoscopists and PolyDeep, furthermore the AUC and p-values of McNemar test and the comparison between both ROC curves. Positive Predictive Value (PPV), Negative predictive Value (NPV), positive Likelihood Ratio (LR+), negative Likelihood Ratio (LR-), Odds Ratio (OR), Area Under the Curve (AUC).

Conclusion

Polydeep, a CADe/x system, is as accurate as expert endoscopists for the optical diagnosis of colorectal polyps.

Introduction

Colonoscopy is the "gold-standard" technique for early diagnosis and resection of colorectal cancer precursor lesions. However, this technique has limitations, and can result in missed adenomas and polyps.
In recent years, artificial intelligence (AI) systems have been developed for the automatic detection of polyps. This study aims to investigate the impact of an AI system applied in real time on the detection rate, compared with conventional detection by the endoscopist.

Aims & Methods

A prospective observational study was carried out by seven gastroenterologists in the Digestive Endoscopy Unit of a tertiary hospital. Colonoscopies were performed on consecutive patients for polyp screening or surveillance. High-definition colonoscopes were used for the examinations and an AI system was applied in real time.
Each procedure consisted of two phases: (1) an initial assessment was performed by the endoscopist who withdrew the endoscope after reaching the cecum, identifying all visualized lesions without resecting them, and (2) the colonoscope was reintroduced up to the cecal pole with an AI device. The automatic detection of the AI device was assessed by means of a light and sound signal.
The primary objective was to compare the detection rate of adenomas and polyps. Secondary objectives were to compare the total number of polyps and serrated lesions detected, as well as detection of polyps by size, location and morphology, with and without the device.

Results

The sixty-six patients included in the analysis were examined simultaneously by colonoscopy, and the AI device was applied. The AI system identified a greater mean number of adenomas (2.85 vs 2.41, p<0.05) and polyps (3.68 vs 2.92, p<0.05), as well as a greater number of serrated lesions (0.83 vs 0.52, p <0.05), compared with the endoscopist. No significant differences were found in the adenoma detection rate (78.8% vs 75.7%; p > 0.05).
AI significantly increased the detection of small lesions (178 vs 135, p<0.001), and those of Paris type IIa flat morphology (165 vs 123, p<0.001). There were no differences in the detection of lesions larger than 5 mm or with other morphologies.

Detection of adenomas and polyps and characteristics
	Examination by endoscopist	Examination IA assisted	p value
Adenoma detection rate (ADR)	0.7575	0.7878	0.678
Polyp detection rate (PDR)	0.7878	0.8333	0.505
Total number of adenomas, n (mean)	159 (2.41)	188 (2.85)	< 0.05
Total number of polyps, n (mean)	193 (2.92)	243 (3.68)	< 0.05
Total number of serrated lesions, n (mean)	34 (0.52)	55 (0.83)	< 0.05
Polyp location Right colon, n (%) Transverse colon, n (%) Left colon, n (%)	70 (36.3) 52 (26.9) 71 (36.8)	98 (40.3) 64 (26.4) 81 (33.3)	< 0.001 0.012 0.018
Polyp morphology Paris 0-IIa, n (%) Paris 0-IIb, n (%) Paris 0-IIc, n (%) Paris 0-Ip, n (%) Paris 0-Is, n (%) LST, n (%)	123 (63.7) 8 (4.2) 1 (0.5) 17 (8.8) 41 (21.2) 3 (1.6)	165 (67.9) 13 (5.3) 1 (0.4) 17 (7) 42 (17.3) 5 (2.1)	< 0.001 0.096 1.000 1.000 0.564 0.157
Polyp size 0-5 mm, n (%) 6-9 mm, n (%) > 10 mm, n (%)	135 (70) 30 (15.5) 28 (14.5)	178 (73.2) 33 (13.6) 32 (13.2)	< 0.001 0.450 0.102

Conclusion

In a high-risk population, the application of an AI system allowed a significantly higher number of polyps and adenomatous lesions to be detected, at the expense of small lesions and those with flat morphology, compared with conventional evaluation by the endoscopist. There were no differences in the adenoma detection rate.