Introduction

Traditional sole deep-learning (DL) diagnostic systems are unexplainable due to the “black box” nature of algorithms, which may impact human-machine interaction and human trust in artificial intelligence (AI). Explainable AI diagnostic systems based on machine-learning algorithms, whose diagnostic logic is interpretable and clear, have the potential to enhance human trust in AI and promote AI application in real clinics. In our previous work, we constructed an explainable AI named ENDOANGEL-ED (explainable-diagnosis) for diagnosing gastric neoplasms under white light endoscopy, it showed its diagnostic indexes on the dashboard. We also trained a traditional sole DL using the same training dataset as a comparison. The ENDOANGEL-ED performed better than the sole DL and greatly improved the endoscopists’ accuracy. However, it is unclear whether interpretability or precision will affect human performance and whether AI with different precision affects humans differently. In this study, we aimed to explore the effect of interpretability and precision on the diagnostic performance of endoscopists in a multi-reader multi-case (MRMC) study.

Aims & Methods

The ENDOANGEL-ED consists of feature-extraction models and quantitative analysis methods to determine the characteristics of a lesion and a fitting diagnosis model to combine the characteristics and make a final diagnosis of whether the lesion is neoplastic. The surface (smooth or rough), protrusion (present or absent), tone (reddish, whitish, or unaltered), image entropy of S-channel in HSI color space (high or low), location (lower stomach, upper-middle stomach, or undistinguishable), and texture information (high or low) were determined as feature indexes for fitting diagnosis. The ENDOANGEL-ED and sole DL were tested in 127 video clips of focal lesions, and the performance of AI was recorded. The recorded video clips and twenty-five endoscopists were included in the MRMC study. All endoscopists underwent four rounds of reading, with a washout period of at least two weeks between each round of reading: 1) reading without AI assistance; 2) with the assistance of ENDOANGEL-ED, diagnostic indexes and results of AI were shown; 3) with the assistance of a sham sole DL, whose results are the same to ENDOANGEL-ED, but the diagnostic indexes were hidden; 4) with the assistance of the real sole DL, only results of AI were shown. The performance of endoscopists in each round of reading was compared.

Results

Both ENDOANGEL and the sole DL significantly improved the endoscopists’ accuracy. Comparing the performance of endoscopists under the assistance of the ENDOANGEL-ED and the sham sole DL, whose results and precision were the same, the accuracy (79.21% vs. 78.24%, p=0.792), sensitivity (82.04% vs. 83.93%, p=0.390) and specificity (77.06% vs. 73.89%, p=0.102) showed no significant difference. Comparing the performance of endoscopists under the assistance of the sham sole DL and the real sole DL, the accuracy (78.24% vs. 76.91%, p=0.142) showed no significant difference. However, the sensitivity (83.93% vs. 88.29%, p=0.030) and specificity (73.89% vs. 68.22%, p=0.035) differed significantly.

Conclusion

The interpretability, reflected by the visible diagnostic indexes, showed little effect in improving the performance of endoscopists. In contrast, the precision of AI systems could significantly affect human diagnostic performance. Precision is the prerequisite for an AI model and is the prior property that AI health providers should consider.

Introduction

Artificial intelligence (AI) showed various performance among test sets with different diversity due to sample selection bias, which can be a stumbling block for AI application. We previously tested AI named ENDOANGEL diagnosing gastric neoplasms and early gastric cancer (EGC) on single-center videos in a man-machine competition. We aimed to retest ENDOANGEL on multi-center videos to explore challenges applying AI in multiple centers, then upgrade ENDOANGEL and explore solutions of solving the challenge.

Aims & Methods

ENDOANGEL was retested on multi-center videos retrospectively collected from 12 institutions and compared with performance in previously-reported single-center videos. We then upgraded ENDOANGEL to ENDOANGEL-2022 with more training samples and novel algorithms, and conducted competition between ENDOANGEL-2022 and endoscopists. ENDOANGEL-2022 was then tested on single-center videos and compared with performance in multi-center videos; the two AI systems were also compared with each other and endoscopists.

Results

Forty-six EGCs and 54 non-cancers were included in multi-center video cohort. On diagnosing EGCs, compared with single-center videos, ENDOANGEL showed stable sensitivity (97.83% vs. 100.00%) while sharply decreased specificity (61.11% vs. 82.54%); ENDOANGEL-2022 showed similar tendency while achieving significantly higher specificity (79.63%, p <0.01) making fewer mistakes on typical lesions than ENDOANGEL. On detecting gastric neoplasms, both AI showed stable sensitivity while sharply decreased specificity. Nevertheless, both AI outperformed endoscopists in the two competitions.

Conclusion

Great increase of false positives is prominent challenge for applying EGC diagnostic AI in multiple centers due to high heterogeneity of negative cases. Optimizing AI by adding samples and using novel algorithms is promising to overcome this challenge.

Introduction

The implementation of computer-aided detection (CAD) devices in esophagogastroduodenoscopy (EGD) could autonomously identify gastric precancerous lesions and neoplasms, and reduce the miss rate of gastric neoplasms in prospective trials. However, there is still insufficient evidence of their use in real-life clinical practice.

Aims & Methods

A real-world, two-center, large-volume, pragmatic study was conducted at Wenzhou Central Hospital (WCH) and Renmin Hospital of Wuhan University (RHWU). Random biopsy and target biopsy were adopted and CAD devices were applied in 2019 and 2021 at WCH and RHWU, respectively. We compared differences in gastric precancerous and neoplasm detection of EGD before and after the use of CAD devices in the first half of the year.

Results

A total of 33885 patients were included and 32886 patients were ultimately analysed. In WCH of which biopsy rate > 95%, with the implementation of CAD, more early gastric cancer detection rate (0.35% vs. 0.59%, p=0.028, OR(95%CI) = 1.65(1.06-2.60)) was found while gastric neoplasm detection rate (1.39% vs. 1.36%, p=0.897, OR(95%CI) = 0.98(0.76-1.26)) remained stable. In RHWU of which biopsy rate < 20%, the gastric neoplasm detection rate (1.78% vs. 3.23%, p<0.001,OR(95%CI) = 1.84(1.33-2.54)) nearly doubled after the implementation of CAD while there was no significant change in the early gastric cancer detection rate.

Conclusion

In our study of large-volume tertiary centers with varied biopsy strategies, we observed that diagnostic benefits differed depending on the application conditions, and each biopsy approach offered distinct advantages. In healthcare center with a high biopsy rate exceeding 95%, the CAD device appeared to enable endoscopists to conduct more precise, targeted biopsies, leading to an increased detection of early gastric cancer. However, the overall gastric neoplasm detection rate remained consistent. In contrast, at institutions like RHWU, which employed a targeted biopsy strategy resulting in an average biopsy rate less than 20%, the incorporation of CAD in routine EGD demonstrated a notable increase in the overall gastric neoplasm detection rate, particularly for low-grade intraepithelial neoplasia.

References

1. Sung, H., et al., Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin, 2021. 71(3): p. 209-249.
2. Correa, P., Human gastric carcinogenesis: a multistep and multifactorial process--First American Cancer Society Award Lecture on Cancer Epidemiology and Prevention. Cancer Res, 1992. 52(24): p. 6735-40.
3. Leung, W.K. and J.J. Sung, Review article: intestinal metaplasia and gastric carcinogenesis. Aliment Pharmacol Ther, 2002. 16(7): p. 1209-16.
4. Correa, P., et al., A model for gastric cancer epidemiology. Lancet, 1975. 2(7924): p. 58-60.
5. Ling, T., et al., A deep learning-based system for identifying differentiation status and delineating the margins of early gastric cancer in magnifying narrow-band imaging endoscopy. Endoscopy, 2021. 53(5): p. 469-477.
6. Gong, E.J., et al., Deep learning-based clinical decision support system for gastric neoplasms in real-time endoscopy: development and validation study. Endoscopy, 2023. 55(8): p. 701-708.
7. Wu, L., et al., Deep learning system compared with expert endoscopists in predicting early gastric cancer and its invasion depth and differentiation status (with videos). Gastrointest Endosc, 2022. 95(1): p. 92-104.e3.
8. Guimarães, P., et al., Deep-learning based detection of gastric precancerous conditions. Gut, 2020. 69(1): p. 4-6.

Introduction

Human-computer interactions (HCI) can substantially influence the performance of Artificial Intelligence (AI). Studies indicate that although endoscopists, when assisted by AI, demonstrate significant improvements in assessing Barrett’s esophagus (BE), their performance does not reach the stand-alone level achieved by AI. A critical aspect of HCI is the impact of AI on the certainty and confidence exhibited by endoscopists. Indirectly, diagnostic confidence when utilizing AI may be associated with the trust and acceptance of AI. In a study focusing on BE videos, our objective was to examine the influence of AI on endoscopists' diagnostic confidence and explore potential correlations with diagnostic performance.

Aims & Methods

Twenty-two endoscopists from twelve centers, with varying levels of BE experience, reviewed ninety-six standardized endoscopy videos. The endoscopists were categorized as experts and non-experts and were randomly assigned to evaluate the videos with and without AI. Participants were divided into two arms: Arm A assessed videos initially without AI and then with AI, while Arm B followed the opposite order. Evaluators were tasked with identifying BE-related neoplasia and providing confidence ratings both with and without AI on a scale from 0 to 9. The study was followed up by a survey.

Results

In Arm A (without AI first, with AI second), the use of AI significantly increased confidence levels for both experts and non-experts (p<0.001). Notably, only non-experts experienced a significant improvement in sensitivity and specificity (p<0.001 bzw. p<0.05) with the assistance of AI. While experts exhibited higher confidence levels without AI compared to non-experts with AI, there was no significant difference in accuracy between these two groups. In Arm B both groups demonstrated a significant reduction in confidence (p<0.001), while maintaining consistent accuracy levels. In addition, in 9% of the choice changes with AI, the wrong choice was made despite correct indications made by AI.

Conclusion

AI significantly boosted confidence levels for both expert and non-expert endoscopists - an effect that was reversible in the study model. Furthermore, experts consistently displayed higher confidence levels, with or without AI, compared to non-experts with AI, despite comparable outcomes. It was further observed that in 9% of the choice changes the investigators ignored the AI in disadvantage of the patient. These findings highlight the potential role of AI in enhancing diagnostic confidence during endoscopic assessments.

Introduction

Immunohistochemistry (IHC) is a critical tool for tumor diagnosis and treatment. However, IHC is time consuming and costly, relies on the pathologists’ subjective interpretation. Deep learning has gained ground for predicting the biomarker expression on hematoxylin and eosin (H&E) staining images. In this study, we aim to create an automatic pipeline for the construction of deep learning models to generate virtual IHC output using H&E whole slide images and validate its clinical applicability by reader study.

Aims & Methods

We obtained 134 whole-slide images (WSIs) from 73 patients at Renmin Hospital of Wuhan University and extracted 415,463 patches from H&E-stained WSIs. Our study focused on five widely used biomarkers in pathology reports, namely anti-P40, CK-pan, Desmin, P53, and Ki-67, as recommended by the NCCN guidelines. We validated these models by using independent whole-slide images (WSIs). The number of WSIs used for each model were as follows: model-P40 (6 WSIs, 38389 Patches), model-PCK (9 WSIs, 72443 Patches), model-Ki67 (7 WSIs, 53162 Patches), model-Desmin (5 WSIs, 23974 Patches), and model-P53 (5 WSIs, 36822 Patches). We aimed to predict these biomarkers expression in H&E WSIs and quantify the percentage of Ki-67. There are three main steps: Firstly, we employed HEMnet neural network to align the corresponding IHC and H&E WSIs, transfering the molecular labels from IHC to H&E images. Secondly, pathologists reviewed the annotations made by HEMnet and corrected HE-IHC mismatched tiles using VIA labelling platform. Lastly, the deep learning models were trained to predict positive tiles. The performance of models was assessed by sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC). To assess the applicability of models, we randomly selected 30 cases including H&E and IHC WSIs for three pathologists to write the reports in the reader study by reading the interpretation of real IHC and AI-generated IHC.

Results

The sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC) of the models for classifying positive pixels of each biomarker was: P40 (91.64%, 87.39%, 88.66%, AUC: 0.95), PAN-CK (92.40%, 89.35%, 90.81%, AUC: 0.96), Ki67 (89.63%, 86.04%, 86.77%, AUC: 0.95), Desmin (91.44%, 88.11%, 89.17%, AUC: 0.96), and P53 (75.44%,86.52%,83.04%, AUC: 0.90). The predicted Ki-67 percentage was comparable to the assessments made by expert pathologists, with a difference ranging from 7.00%±3.69%. In the reader study, compared with real-IHC pathology report, the accuracy of AI-IHC pathology report of each biomarker was: Desmin(100%,66/66), Pan-CK(98.3% , 59/60), P53(98.3% ,59/60), T-stage(84.8% .56/66), P40(96.7% ,29/30). and there is no significant difference between the quatification of Ki-67 score(t= 0.253 ,P= 0.800).
​

Conclusion

Our deep learning models have demonstrated promising performance in predicting gastrointestinal tumor biomarkers based on H&E slides. The utilization of these models can alleviate the burden of IHC staining and clinicopathological assessments. Furthermore, these models have the potential to aid in patient grading, facilitate informed treatment decisions, and provide valuable prognostic assessments.

Disclosure

There is no conflict of interest in this abstract.

Introduction

Recent advances in large language models, such as GPT-4.0 and GPT-4.5, have increased the accessibility of artificial intelligence (AI) in clinical settings. However, its role in decision-making remains unclear.

Aims & Methods

This study compares the appropriateness of esophagogastroduodenoscopy (EGD) prescriptions across ten clinical scenarios, as evaluated by gastroenterology specialists, residents, general practitioners, and five AI systems (ChatGPT-4.0, ChatGPT-4.5, Gemini, Claude, and OpenEvidence), using ESGE and ASGE guidelines as the reference standard. A cross-sectional survey was developed using 10 real-world clinical vignettes in gastroenterology, each with three response options: appropriate, inappropriate, or uncertain. The vignettes were developed based on real-world clinical practices by three expert gastroenterologists, and then reviewed by the authors.The final questionnaire was hosted on Google Forms and simultaneously distributed—without prior notice—to gastroenterology specialists, residents, and general practitioners across Italy to minimize external influence. The same clinical scenarios were then submitted to five AI systems (ChatGPT-4.0, ChatGPT-4.5, Gemini, Claude, and OpenEvidence) using the same multiple-choice format. AI responses were collected on April 22, 2025.

Results

A total of 122 healthcare professionals participated in the study (61% male; mean age 39.7 ± 13.6), including 41 gastroenterology specialists, 31 residents, and 50 general practitioners. Most participants were from northern Italy (76%), with additional representation from central (14%) and southern regions (10%), covering a total of 10 regions. The majority had less than 5 years of clinical experience (66%) and were employed in public hospitals (49%). Only 30% reported prior use of AI in clinical practice. All respondents considered the clinical scenarios realistic.
The average accuracy rate was highest among gastroenterologists (65.6% ± 11.6) and residents (64.8% ± 10.0), compared to general practitioners (56.8% ± 11.0), with statistically significant differences (p < 0.001 and p < 0.005, respectively).
Among AI systems, OpenEvidence and Gemini outperformed all human groups (80% and 80%, p < 0.001), while ChatGPT-4.0 (70%) and Claude (60%) showed comparable performance to clinicians. Notably, ChatGPT-4.5 (50%) underperformed relative to all human categories.

Conclusion

This study highlights significant variability in the appropriateness of upper GI endoscopy prescriptions among different medical professionals, with gastroenterologists performing better than general practitioners. Emerging AI models, particularly OpenEvidence and GEMINI, demonstrated promising performance in clinical decision-making, outperforming all human groups in certain scenarios. These AI systems could serve as valuable tools to support gastroenterologists and other healthcare professionals, enhancing adherence to evidence-based practices and facilitating more direct consultation of clinical guidelines. Further research is needed to explore the integration of AI into clinical workflows and its potential to refine decision-making in gastroenterology.

References

1) Chang PW, Amini MM, Davis RO, et al. ChatGPT4 Outperforms Endoscopists for Determination of Postcolonoscopy Rescreening and Surveillance Recommendations. Clin Gastroenterol Hepatol. 2024 Sep;22(9):1917-1925.e17. doi: 10.1016/j.cgh.2024.04.022. Epub 2024 May 9. PMID: 38729387.
2) Maida M, Celsa C, Lau LHS, et al. The Application of Large Language Models in Gastroenterology: A Review of the Literature. Cancers (Basel). 2024 Sep 28;16(19):3328. doi: 10.3390/cancers16193328. PMID: 39409948; PMCID: PMC11475222.

Introduction

AI diagnosis is used in several medical fields; however, machine learning using deep learning methods is particularly effective in diagnostic imaging and shows high diagnostic capability even in the field of endoscopy. In Japan, endoscopes manufactured by Companies A and B are generally used, and there is a possibility that differences in the image quality will affect AI diagnosis. However, there is no report of studies comparing the AI diagnostic capabilities on images taken by endoscopes of two companies.

Aims & Methods

In endoscopic AI diagnosis of early gastric cancer, the AI diagnostic capabilities were compared using images obtained by endoscopes of Companies A and B for the purpose of clarifying whether there is a need to consider the differences between the endoscopes used. For 30 lesions of 27 patients who underwent endoscopic submucosal dissection for early gastric cancer from April 2019 to March 2022 at our hospital, 356 and 338 endoscopic images of Companies A and B were the target of analysis, respectively.The images were white light (WLI), chromoendoscopy (indigo carmine) and image-enhanced endoscopy, Company A used narrow band imaging (NBI), and Company B used blue laser imaging (BLI).The images were analyzed by AI Medical Systems Inc. using an AI diagnostic system constructed by having UNET learn images taken by either Company A or B's endoscopes as training images. The diagnostic capability was assessed by comparing the specialist-marked correct range of early gastric cancer with an AI-diagnosed range of early gastric cancer, and using a DiceScore of 1.0 for a complete match and 0.0 for no match based on the area of the match.

Results

There was no significant difference in the DiceScore between the endoscopes of both companies in WLI; however, in indigo, Company B scored significantly higher than Company A. There was no significant difference between NBI and BLI. A stratification analysis by macroscopic examination of the lesions showed that in 0-IIc, Company A scored significantly higher in WLI than Company B, while in indigo, Company B scored significantly higher than Company A. In 0-IIa+IIc, Company B scored significantly higher than Company A in WLI; however, there was no difference between the two companies in indigo. In comparison by macroscopic examination of the lesions, 0-IIa+IIc lesions scored significantly higher than 0-IIc lesions in indigo for Company A, and in WLI for Company B. When the lesions were divided into a maximum diameter of <2 cm, 2–3 cm, and ≥3 cm, lesions ≥3 cm scored lower than lesions in the other groups.

Conclusion

Endoscopic images of Company B tended to have higher sharpness and stronger bluish color compared to endoscopic images of Company A. This might be the reason why the diagnostic capability improved after indigo-carmine scattering. Therefore, proactive scattering of indigo carmine was considered preferable when observing with the Company B endoscope. Presumably, the reason 0-IIa+IIc lesions had higher diagnostic performance than 0-IIc lesions was due to differences in the elevation of the lesions. The Company A endoscope was considered significantly superior for 0-IIc lesions, which are easily overlooked in clinical practice. The complexity of the morphology of the lesion increases in large lesions, which widens the gap between the AI diagnostic range and correct range. There are lesions wherein the AI diagnostic performance increased with both endoscopes of Company A and that of Company B. Therefore, it is preferable to use the endoscopes in AI diagnosis taking these facts into consideration.

References

Hirasawa T, Aoyama K, Tanimoto T, et al: Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21; 653-660: 2018
Ikenoyama Y, Hirasawa T, Ishioka M, et al: Detecting early gastric cancer; Comparison between the diagnostic ability of convolutional neural networks and endoscopists. Dig Endosc 33; 141-150: 2021
Miyaki R, Yoshida S, Tanaka S, et al: A computer system to be used with laser-based endoscopy for quantitative diagnosis of early gastric cancer. J Clin Gastroenterol 49; 108-115: 2015
Horiuchi Y, Aoyama K, Tokai Y, et al: Convolutional neural network for differentiating gastric cancer from gastritis using magnified endoscopy with narrow band imaging. Dig Dis Sci 65; 1355-1363: 2020
Ling T, Wu L, Fu Y, et al: A deep learning-based system for identifying differentiation status and delineating the margins of early gastric cancer in magnifying narrow-band imaging endoscopy. Endoscopy 53; 469-477: 2021
Hamada K, Kawahara Y, Tanimoto T, et al: Application of convolutional neural networks for evaluating the depth of invasion of early gastric cancer based on endoscopic images. J Gastroenterol Hepatol 37; 352-357: 2022