Introduction

Deep learning systems in gastrointestinal endoscopy are traditionally pretrained on large, diverse datasets comprising generic, non-medical images before being trained on scarce endoscopic data. Foundation models are models pretrained based on domain-specific datasets. For endoscopy, foundation models require large datasets of general endoscopic images. Yet datasets for developing such models remain limited. In this study, we present GastroNet-5M, a dataset comprising 4,820,653 endoscopic images of approximately 500,000 procedures.

Aims & Methods

GastroNet-5M is composed of anonymized endoscopic images collected from eight Dutch hospitals between 2012 and 2020. Using a self-supervised learning approach, this dataset enabled the development of a foundation model for various endoscopic AI applications. In this study, we compared GastroNet-5M-pretrained models against current state-of-the-art models with traditional pretraining approaches and all publicly available endoscopic foundation models.
We evaluated diagnostic performance on 9 classification and 9 segmentation tasks, including Barrett’s neoplasia detection, colorectal polyp characterization, and gastric cancer invasion depth prediction. All models were trained from scratch for each task. Data efficiency was assessed by repeating all classification tasks with progressively smaller training sets (i.e. 100%, 50%, 25%, 10% and 5%). Robustness was evaluated across 12 datasets reflecting data heterogeneity such as variation in endoscope manufacturer, imaging modality (e.g., virtual chromoendoscopy), and image quality.

Results

GastroNet-5M-pretrained models outperformed all benchmarks in 8 of 9 classification tasks, with an average AUC improvement of 6.8% (Table 1). For segmentation, they achieved higher Dice scores compared to all other models across all 9 tasks, with an average gain of 15.4%. In data efficiency experiments, GastroNet-5M models achieved the highest AUC in 5 of 9 tasks when training data was limited, while no other model led in more than one. In robustness testing, they outperformed other models in 8 of 12 tasks evaluating heterogeneous data conditions.

Table 1. Area-Under-the-Curve scores of the GastroNet-5M-pretrained model and benchmark models for all classification tasks.

Conclusion

GastroNet-5M, a multicenter dataset of over 5 million unlabeled endoscopic images, offers a valuable resource for pretraining deep learning models in endoscopy. The use of GastroNet-5M enhances diagnostic accuracy, reduces the need for scarce application-specific endoscopic imagery, and improves robustness of endoscopic AI systems to the inevitable data heterogeneity in clinical practice. GastroNet-5M is publicly accessible for further research and development.

References

1. Sanderson E, Matuszewski BJ. arXiv preprint arXiv:2401.06278. 2024.
2. van der Putten J, de Groof J, Struyvenberg M, et al. Artif Intell Med. 2020;107.
3. Wang Z, Liu C, Zhang S, et al. arXiv preprint arXiv:2306.16741. 2023.
4. Liu Z, Mao H, Wu C-Y, et al. arXiv preprint arXiv:2201.03545. 2022.
5. Dosovitskiy A, Beyer L, Kolesnikov A, et al. arXiv preprint arXiv:2010.11929. 2020.
6. Oquab M, Darcet T, Moutakanni T, et al. arXiv preprint arXiv:2304.07193. 2023.
7. Jocher G, Qiu J. Ultralytics YOLOv11, v11.0.0. 2024.
8. de Groof AJ, Struyvenberg MR, van der Putten J, et al. Gastroenterology. 2020;158(4):915–29.e4.
9. Fockens KN, Jukema JB, Boers T, et al. United European Gastroenterol J. 2023;11(4):324–36.
10. Jukema J, Kusters C, Jong M, et al. Endoscopy (ESGE Days). 2023.
11. GIANA Challenge. Available from: https://giana.grand-challenge.org
12. Borgli H, Thambawita V, Smedsrud PH, et al. Sci Data. 2020;7(1):283.
13. Polat G, Kani HT, Ergenc I, et al. Inflamm Bowel Dis. 2023;29(9):1431–9.
14. van der Zander QEW, Schreuder RM, Fonollà R, et al. Endoscopy. 2021.
15. Tang D, Zhou J, Wang L, et al. Front Oncol. 2021;11:654093.
16. Houwen BB, Hazewinkel Y, Giotis I, et al. Endoscopy. 2022;54(Suppl 1):eP133.

Introduction

Endoscopic ultrasound (EUS) is the gold standard for evaluating pancreatic lesions, allowing for real-time imaging and biopsy sampling. Nevertheless, the differentiation between solid pancreatic lesions — such as pancreatic ductal adenocarcinoma (PDAC) and pancreatic neuroendocrine tumors (PNETs) — as well as cystic lesions is often troublesome. Specifically, identifying intraductal papillary mucinous neoplasms (IPMNs) with varying degrees of dysplasia is crucial for guiding clinical management. However, the diagnostic accuracy of current methods remains suboptimal. This multicenter study aimed to develop and validate convolutional neural networks (CNNs) to characterize solid and cystic pancreatic lesions and stratify IPMNs into high-grade dysplasia/carcinoma (HGD/C) and low-grade dysplasia (LGD).

Aims & Methods

This multicenter study included EUS images from 12 centers in Spain, Brazil, the United Kingdom and the United States of America. Solid lesion differentiation (PDAC vs. PNET) was achieved using a CNN trained on 49034 images from 95 patients (n=69 for PDAC and n=26 for PNET), with diagnoses confirmed via EUS-guided fine-needle biopsy or surgical specimen analysis. A second CNN was trained on 79395 images from 82 patients (n=64 for mucinous cystic lesions and n=18 for non-mucinous cystic lesions). Finally, a third model used the subset of IPMNs (n=51046 frames, 30 patients) to distinguish between HGD/C (n=5 patients) and LGD (n=25 patients). Histological confirmation was used as the gold standard. Performance metrics included sensitivity, specificity, accuracy, and area under the precision-recall curve (AUC-PR). To comprehensively determine the model's performance, we evaluated multiple thresholds for the confidence score and the IoU, between 0.45 and 0.55 with a step of 0.02 ([0.45, 0.47, 0.50, 0.52, 0.55]). The datasets were split into training (70%), validation (20%), and testing (10%) subsets.

Results

For solid lesions, the model differentiated PDAC from PNETs with an accuracy of 91.8%, with sensitivities of 96.2% and 97.8%, respectively. The AUC-PR was 0.982 for PDAC and 0.977 for PNET. Regarding cystic lesions the model detected mucinous cystic lesions with a sensitivity of 97.5%, an overall accuracy of 89.7% and an AUC-PR of 0.977. When the subset of IPMNs was specifically analyzed, the model distinguished IPMNs with HGD/C from those with LGD with a sensitivity of 97.5%, specificity of 94.3%, and an overall accuracy of 89.8%. The AUC was 0.988.

Conclusion

This is the first multicenter, transatlantic AI study aiming to provide a holistic approach to both solid and cystic pancreatic lesions, focusing on those with a significantly higher malignant potential. Our models achieved high accuracy in identifying and differentiating these conditions. Furthermore, to our knowledge this is the first integrative model to include the ability to stratify IPMNs into LGD and HGD/C, which has significant clinical implications, potentially reducing unnecessary surgeries and ensuring timely intervention for high-risk patients. The inclusion of data from diverse regions reduces demographic bias and enhances model generalizability. Real-time implementation of these solutions could significantly improve decision-making in EUS and optimize patient outcomes.

Introduction

Self-expanding metal stents (SEMS) are recommended by guidelines to relieve biliary obstruction caused by malignant strictures in a palliative setting. There are three types of SEMS, uncovered (UC), semi-covered (SC), and fully covered (FC) indicating the degree to which they are covered by a plastic film. The main concerns regarding bile duct stents are their ability to provide adequate drainage, maintain long-term patency, and overall safety. The intention with the plastic covering is to prevent tumour ingrowth and occlusion of the stent. However, this also makes the stents more liable to dislodge spontaneously (1). There are conflicting reports on UC and SC stents regarding long-term patency; some studies have shown a higher frequency of dysfunction in UC stents while others have not (2, 3). Only a few studies have examined the use of FC stents(4). The aim of this study is to examine the differences in long-term patency of UC, SC, and FC SEMS in palliative patients with jaundice caused by malignant obstruction of the distal bile duct.

Aims & Methods

In this prospective randomized multicenter study patients were recruited between March 2020 and August 2024 at 13 centers in Sweden performing ERCPs. Patients with malignant extrahepatic strictures, where curative surgery was considered unlikely, were included. Patients were randomized in a 1:1:1 ratio to receive UC, SC, or FC SEMS and followed sequentially for 12 months. The primary outcome was stent failure defined as either the patient undergoing re-ERCP or other biliary drainage intervention or clinical and laboratory signs indicating stent dysfunction. Patients were censored at stent failure, death, time point of curative surgery or when the follow-up time ended. This study was registered at ClinicalTrials.gov (NCT06453590).

Results

A total of 450 patients who had reached their respective endpoints were included in this preliminary intention-to-treat analysis: 163 patients in the UC group, 151 in the FC group and 136 in the SC group. The majority (83%) of the patients had pancreatic cancer. After 12 months of follow-up, 50 patients (11%) were alive with a functioning stent, 151 patients (34%) had received palliative oncological treatment, and 18 patients (4%) had undergone surgery with curative intent. Stent failure occurred in 17% (95% CI: 11-23%) of the UC group, 13% (95% CI: 7-18%) of the FC group, and 7% (95% CI: 3-12%) of the SC group. The failure rate was lower in the SC group (HR 0.4; 95% CI: 0.2-0.8) compared to the UC group. There was a non-significant difference in failure rate in the FC group (HR 0.7; 95% CI: 0.4-1.3) relative to the UC group.

Conclusion

To our knowledge, this is the first prospective randomized trial comparing the long-term patency of the three different types of SEMS. The results suggest that semi-covered SEMS have the lowest rate of stent failure in patients with extrahepatic malignant obstruction and may therefore be the most appropriate choice of stent for palliative patients.

References

1. Kim HS, Lee DK, Kim HG, Park JJ, Park SH, Kim JH, et al. Features of malignant biliary obstruction affecting the patency of metallic stents: a multicenter study. Gastrointest Endosc. 2002;55(3):359-65.
2. Saleem A, Leggett CL, Murad MH, Baron TH. Meta-analysis of randomized trials comparing the patency of covered and uncovered self-expandable metal stents for palliation of distal malignant bile duct obstruction. Gastrointest Endosc. 2011;74(2):321-7.e1-3.
3. Almadi MA, Barkun AN, Martel M. No benefit of covered vs uncovered self-expandable metal stents in patients with malignant distal biliary obstruction: a meta-analysis. Clin Gastroenterol Hepatol. 2013;11(1):27-37.e1.
4. Yamashita Y, Tachikawa A, Shimokawa T, Yamazaki H, Itonaga M, Sakai Y, et al. Covered versus uncovered metal stent for endoscopic drainage of a malignant distal biliary obstruction: Meta-analysis. Dig Endosc. 2022;34(5):938-51.

Disclosure

The study received partial funding from Boston Scientific Nordic AB

Introduction

Colorectal anastomotic dehiscence (CAD) is one of the most severe postoperative complications in colorectal surgery, significantly impacting patient morbidity and mortality. Current risk assessment methods rely on clinical and intraoperative factors, but no real-time predictive tool exists. This study aimed to develop an artificial intelligence model based on intraoperative recording of the anastomosis for CAD prediction.

Aims & Methods

A convolutional neural network (CNN) was trained using a dataset of annotated frames from colorectal surgery videos across three international high-volume centers: IPO Lisboa, Hospital das Clínicas de Ribeirão Preto, São Paulo, Brazil and Royal Liverpool University Hospital. The dataset included a total of 5,356 annotated frames from 25 patients, 2,007 frames classified as CAD-positive and 3,349 as CAD-negative. Four CNN architectures (EfficientNetB0, EfficientNetB7, ResNet50, and MobileNetV2) were tested. Performance was evaluated using sensitivity, specificity, accuracy, and area under the receiver operating characteristic (AUROC) curve. Heatmaps were generated to identify key regions influencing predictions.

Results

The best-performing model achieved an accuracy of 99.55%, AUROC of 99.60%, sensitivity of 99.21%, specificity of 100%, PPV of 100%, and NPV of 98.94%. The model reliably identified CAD-positive frames and provided visual explanations through heatmaps.

Conclusion

To our knowledge, this is the first AI model developed to predict CAD using intraoperative video analysis. Its accuracy suggests the potential to redefine surgical decision-making by providing real-time risk assessment. Further refinement with a larger dataset and diverse surgical techniques could enable intraoperative interventions to prevent CAD before it occurs, marking a paradigm shift in colorectal surgery.