Introduction
Accurate and timely endoscopic evaluation is critical for the management of postoperative Crohn’s disease (CD). Although numerous established and novel indices exist, their relative operating properties and the optimal instrument for routine use in clinical and research settings are unknown.
Aims & Methods
We characterised the reliability and responsiveness of both historical and novel endoscopic indices using centrally assessed ileocolonoscopy videos from a randomised controlled trial comparing infliximab versus placebo for preventing postoperative CD (PREVENT; NCT01190839). Three blinded central readers independently assessed 70 videos from the PREVENT trial using the Rutgeerts and modified Rutgeerts scores, POCER index, REMIND score, Simple Endoscopic Score for CD (SES-CD), and the CD Endoscopic Index of Severity (CDEIS). Indices were scored separately for the neoterminal ileum, the anastomosis (including the blind loop of a side-to-side anastomosis), and the distal colonic segment. Intra- and inter-rater reliability was quantified using the intraclass correlation coefficient (ICC) and was interpreted using the Landis and Koch benchmarks.1 Responsiveness was quantified using the win probability (WinP), defined as the probability of a patient receiving infliximab having a better endoscopic score than a patient receiving placebo, and interpreted using Cohen’s benchmarks.2
Results
Inter-rater reliability was substantial for the Rutgeerts (ICC 0.74 [95% CI: 0.62-0.83]) and modified Rutgeerts scores (0.79 [0.68-0.86]), ileal REMIND score (0.79 [0.67-0.87]), SES‑CD (0.79 [0.70-0.86]), and CDEIS (0.80 [0.72-0.86]). Inter-rater reliability was moderate for the POCER index (ICC 0.49 [0.26-0.64]) and fair for the anastomotic REMIND score (0.30 [0.14-0.47]). Intra-rater reliability was almost perfect for all indices (ICC≥0.81) (Table 1). The Rutgeerts (WinP 0.77 [95% CI: 0.65-0.86]) and modified Rutgeerts scores (WinP 0.81 [0.69-0.89]), ileal REMIND score (WinP 0.83 [0.72-0.91]), SES-CD (WinP 0.80 [0.66-0.89]), and CDEIS (WinP 0.75 [0.61-0.86]) demonstrated a large degree of responsiveness when treatment assignment was the criterion for change. The POCER index (WinP 0.54 [0.43-0.65]) and the anastomotic REMIND score (WinP 0.53 [0.42-0.64]) had a small degree of responsiveness. The reliability and responsiveness of all indices were consistently lower when assessed at the anastomosis or distal colon compared to the neoterminal ileum.
Table 1. Reliability and responsiveness of endoscopic scores
| Index | Inter-rater ICC (95% CI)a | Intra-rater ICC (95% CI)a | WinP
(95% CI)b,c | WinP
(95% CI)b,d |
| Rutgeerts score | 0.74 (0.62, 0.83) | 0.90 (0.85, 0.94) | 0.77 (0.65, 0.86) | 0.72 (0.57, 0.83) |
| Modified Rutgeerts score | 0.79 (0.68, 0.86) | 0.92 (0.88, 0.96) | 0.81 (0.69, 0.89) | 0.68 (0.52, 0.81) |
| POCER index | 0.49 (0.26, 0.64) | 0.89 (0.78, 0.95) | 0.54 (0.43, 0.65) | 0.61 (0.46, 0.74) |
| REMIND - Ileal | 0.79 (0.67, 0.87) | 0.91 (0.85, 0.96) | 0.83 (0.72, 0.91) | 0.71 (0.55, 0.83) |
| REMIND - Anastomotic | 0.30 (0.14, 0.47) | 0.81 (0.71, 0.89) | 0.53 (0.42, 0.64) | 0.62 (0.47, 0.75) |
| SES-CD | 0.79 (0.70, 0.86) | 0.93 (0.87, 0.96) | 0.80 (0.66, 0.89) | 0.70 (0.54, 0.82) |
| CDEIS | 0.80 (0.72, 0.86) | 0.91 (0.86, 0.95) | 0.75 (0.61, 0.86) | 0.70 (0.56, 0.81) |
| Overall VAS | 0.81 (0.72, 0.87) | 0.90 (0.84, 0.94) | 0.80 (0.67, 0.88) | 0.71 (0.55, 0.83) |
a Interpreted according to the Landis and Koch benchmarks1: poor = <0; slight = 0.0–0.20; fair = 0.21–0.40; moderate = 0.41–0.60; substantial = 0.61–0.80; and almost perfect = 0.81–1.00.
b WinPs of 0.56, 0.64, and 0.71 correspond to Cohen’s effect sizes of 0.2, 0.5, and 0.8, which are the benchmarks for “small”, “moderate”, and “large” effects, respectively.2
c Using treatment assignment as the criterion for change.
d Using clinical recurrence as the criterion for change.
Abbreviations: WinP, Win probability; CDEIS, Crohn’s Disease Endoscopic Index of Severity; CI, confidence interval; ICC, intraclass correlation coefficient; POCER, Postoperative Crohn's endoscopic recurrence; SES-CD, Simple Endoscopic Score for Crohn’s Disease; VAS, visual analogue scale. |
Conclusion
Existing endoscopic indices are reliable and responsive for assessing postoperative CD activity in the neoterminal ileum but are suboptimal for assessing the anastomosis or distal colon.
References
1. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33 (1):159-174.
2. Zou G, Zou L, Qiu SF. Parametric and nonparametric methods for confidence intervals and sample size planning for win probability in parallel-group randomized trials with Likert item and Likert scale data. Pharm Stat 2023; 22 (3):418-439.
Disclosure
JH reports consulting fees from Abbvie,Alimentiv Inc,Eli Lilly,Janssen,and Takeda.
SKV,GZ,LMS report consulting fees from Alimentiv Inc.
BB reports advisory/speaker fees from: Ferring,Janssen,Abbvie,Takeda,Pfizer,BMS,Merck,Sandoz,Organon,Lifelabs,Celltrion,Alimentiv,Gilead,Iterative Health,Celgene,Merck,Amgen,Pendopharm,Eli Lilly,Fresenius Kabi,Mylan,Viatris,Bausch Health,BioJamp Pharma,Eupraxia; research support: Janssen,Abbvie,GSK,BMS,Amgen,Genentech,Merck.
MAS reports advisory fees from: Janssen,Takeda,Sandoz,Samsung Bioepis,Galapagos,AbbVie,Bristol-Myers Squibb,Tillotts,Pfizer,STADA,Sanofi,MSD; and lecture fees from: Bristol-Myers Squibb,Janssen,Takeda,MSD,Falk,AbbVie,Galapagos,Lilly.
MH is a former employee of Alimentiv Inc.
JR is an employee of Alimentiv Inc.
PDC reports advisory/speaker/consulting fees from AbbVie,Baxter,Ferring,Janssen,Celltrion,Emerge Health,Shire,and Takeda. PDC is supported by an NHMRC Emerging Leader 2 Fellowship and reports research support from AbbVie,Ferring,Shire,Janssen,Pfizer,and Takeda.
MR reports advisory/consulting fees from Abbvie,Janssen,UCB,Takeda,Pfizer,BMS,Organon,Amgen,Genentech,Gilead,Salix,Prometheus,Lilly,Celgene,Boehringer Ingelheim Pharmaceuticals Inc. (BIPI),Celltrion,Roche,Merck,Sanofi,Biocon,Abavax.
SS reports research grants from Pfizer
RWS reports advisory/consulting fees from AbbVie,Bristol Myers Squibb,CorEvitas,Eli Lilly,Exact Sciences,Gilead,Johnson and Johnson,Merck,Okura Theraeutics,Pfizer,and Takeda; received investigator initiated research funding from Abbvie,Bristol Myers Squibb,Johnson and Johnson,and Merck; holds intellectual property and equity on medical imaging and endoscopic analysis technologies licensed by the University of Michigan to PreNovo,LLC,AMI,LLC and PathwaysGI,Inc.
MD reports speaking fees from Bristol Meyers Squibb,Takeda,Galapagos,Janssen,Dr. Falk; advisory board fees from Abbvie,Bristol Meyers Squibb,Celltrion,Galapagos/Alfasigma,Janssen,Takeda; grant/research support: Pfizer,Bristol Meyers Squibb,Galapagos/Alfasigma,Janssen.
GDH reports advisory/speaker fees from Abbvie,Agomab Therapeutics,Alimentiv,AstraZeneca,Bristol Meiers Squibb,Cytoki,Celltrion,Eli Lilly,Exeliom Biosciences,Ferring,Galapagos,Glaxo Smith Kline,Pfizer,Polpharm,Immunic,Index Pharmacueticals,Johnson and Johnson,Merck,ProciseDx,Roivant,Seres Health,Takeda,Tillotts and Ventyx.
LPB reports consulting fees from Abbvie,Abivax,Adacyte,Alimentiv,Amgen,Applied Molecular Transport,Arena,Banook,Biogen,BMS,Celltrion,Connect Biopharm,Cytoki Pharma,Enthera,Ferring,Fresenius Kabi,Galapagos,Genentech,Gilead,Gossamer Bio,GSK,IAC Image Analysis,Index Pharmaceuticals,Inotrem,Janssen,Lilly,Medac,Mopac,Morphic,MSD,Nordic Pharma,Novartis,Oncodesign Precision Medicine,ONO Pharma,OSE Immunotherapeuthics,Pandion Therapeuthics,Par' Immune,Pfizer,Prometheus,Protagonist,Roche,Samsung,Sandoz,Sanofi,Satisfay,Takeda,Telavant,Theravance,Thermo Fischer,Tigenix,Tillots,Viatris,Vectivbio,Ventyx,Ysopia; grants support from Celltrion,Fresenius Kabi,Medac,MSD,Takeda; lecture fees from Abbvie,Alfasigma,Amgen,Arena,Biogen,Celltrion,Ferring,Galapagos,Genentech,Gilead,Janssen,Kern Pharma,Lilly,Medac,MSD,Nordic Pharma,Pfizer,Sandoz,Takeda,Tillots,Viatris; and support for travel from Abbvie,Alfasigma,Amgen,Celltrion,Connect Biopharm,Ferring,Galapagos,Genentech,Gilead,Gossamer Bio,Janssen,Lilly,Medac,Morphic,MSD,Pfizer,Sandoz,Takeda,Thermo Fischer,Tillots.
SD reports consulting fees from AbbVie,Alimentiv Inc,Allergan,Amgen,AstraZeneca,Athos Therapeutics,Biogen,Boehringer Ingelheim,Bristol Myers Squibb,Celgene,Celltrion,Dr Falk Pharma,Eli Lilly,Enthera,Ferring Pharmaceuticals Inc.,Gilead,Hospira,Inotrem,Janssen,Johnson & Johnson,Morphic,MSD,Mundipharma,Mylan,Pfizer,Roche,Sandoz,Sublimity Therapeutics,Takeda,Teladoc Health,TiGenix,UCB Inc.,Vial,and Vifor; lecture fees from AbbVie,Amgen,Ferring Pharmaceuticals Inc.,Gilead,Janssen,Mylan,Pfizer,and Takeda.
BES reports research grants from Janssen and Bristol Myers Squibb; consulting fees from AbbVie,Abivax,Adiso Therapeutics,Agomab,Alimentiv,Amgen,AnaptysBio,AstraZeneca,Biora Therapeutics,Boehringer Ingelheim,Bristol Myers Squibb,Celltrion,Ensho Therapeutics,Equillium,Ei Lilly and Company,Enthera,Enveda Biosciences,Evommune,Ferring,Fzata,Galapagos,Genentech,Gilead Sciences,GlaxoSmithKline,Gossamer Bio,Imhotex,Index Pharmaceuticals,Innovation Pharmaceuticals,Janssen,Johnson & Johnson,Kaleido,Kallyope,Merck & Co.,Microba,Microbiotica,Mitsubishi Tanabe Pharma,Mobius Care,Morphic Therapeutics,MRM Health,Nexus Therapeutics,Immunyx Therapeutics,Nimbus Discovery,Odyssey Therapeutics,Palisade Bio,Pfizer,Progenity,Prometheus Biosciences,Prometheus Laboratories,Protagonist Therapeutics,Q32 Bio,Rasayana Therapeutics,Recludix Therapeutics,Reistone Biotherapeutics,Sanofi,Sorriso Pharmaceuticals,Spyre Therapeutics,Surrozen,Target RWE,Takeda,Teva,Theravance Biopharma,TLL Pharmaceutical,TR1X,Union Therapeutics,Ventyx Biosciences; lecture fees from AbbVie,Abivax,Bristol Myers Squibb,Celltrion,Ei Lilly and Company,Janssen,Mitsubishi Tanabe Pharma,Pfizer,and Takeda; support for attending meetings and/or travel from Ei Lilly and Company,Janssen,and Takeda; participation on advisory board for Agomab and Amgen; stock or stock options from Ventyx Biosciences; medical writing/editorial support for AbbVie,Abivax,Bristol Myers Squibb,Celltrion,Eli Lilly and Company,Janssen,Merck & Co.,Pfizer,Ventyx Biosciences,Morphic Therapeutics,Takeda,and Alimentiv.
RP reports advisory/speaker/consulting fees from Abbott,AbbVie,Abbivax,Alimentiv (formerly Robarts),Amgen,AnaptysBio,Arena Pharmaceuticals,AstraZeneca,Biogen,Boehringer Ingelheim,Bristol-Myers Squibb,Celgene,Celltrion,Cosmos Pharmaceuticals,Eisai,Elan,Eli Lilly,Ferring,Galapagos,Fresenius Kabi,Genentech,Gilead Sciences,Glaxo-Smith Kline,JAMP Bio,Janssen,Merck,Mylan,Novartis,Oppilan Pharma,Organon,Pandion Pharma,Pendopharm,Pfizer,Progenity,Prometheus Biosciences,Protagonist Therapeutics,Roche,Sandoz,Satisfai Health,Shire,Sublimity Therapeutics,Spyre Therapeutics,Takeda Pharmaceuticals,Theravance Biopharma,Trellus,Union Biopharma,Viatris,Ventyx,UCB
BGF reports consulting fees from AbbVie,Abivax,Adiso,AgomAB Therapeutics,Allianthera,Amgen,AnaptysBio,Arena Pharma,Avoro Capital Advisors,Atomwise,,BioJamp,Biora Therapeutics,Blackbird Laboratories,Boehringer-Ingelheim,Boxer Capital,Celsius Therapeutics,Celgene/BMS,Connect BioPharma,Cytoki,Disc Medicine,Duality,EcoR1,Eli Lilly,Equillium,Ermium,First Wave,Forbion,Galapagos,Galen Atlantica,Genentech/Roche,Gilead,Gossamer Pharma,GSK,Hinge Bio,Index Pharma,Imhotex,Immunic Therapeutics,Intercept,JAKAcademy,Janssen,Japan Tobacco Inc.,Kaleido Biosciences,Klick Health,Landos Biopharma,Lenczner Slaght,LifeSci Capital,Lument AB,Mage Biologics,Mestag,Millennium,MiroBio,Monte Rose Tx,Morgan Lewis,Morphic Therapeutics,Mylan,Nexys Therapeutics,Nimbus Therapeutics,OM Pharma,OrbiMed Origo BioPharma,Orphagen,Pandion Therapeutics,Pendopharm,Pfizer,Prometheus Therapeutics and Diagnostics (Merck),Progenity,Protagonist,PTM Therapeutics,Q32 Bio,Rebiotix,REDX,Roche,Roivant/Televant,Sandoz,Sanofi,Seres Therapeutics,Silverback Therapeutics,Sobi,Spyre Therapeutics,Surrozen Inc.,Synedgen,Takeda,Teva,Thelium,Tigenix,Tillotts,Triastek,Ventyx; lecture fees from Takeda,Janssen,and AbbVie; participation on advisory board for AbbVie,AnaptysBio,Boehringer-Ingelheim,Celgene/BMS,Eli Lilly,Genentech/Roche,Janssen,Merck,MiroBio,Origo BioPharma,Pfizer,REDX Pharma,Sanofi,Takeda,Teva,Ecor1Capital,Morphic,GSK; stock or stock options from Evida,EnGene,and Connect Biopharma.
VJ reports consulting fees and participation on advisory board for AbbVie,Alimentiv,Altrubio,Amgen,AnaptysBio,Arena pharmaceuticals,Asahi Kasei Pharma,Asieris,Astra Zeneca,BiomeBank,Boehringer Ingelheim,Bristol Myers Squibb,Calluna Pharma,Celltrion,Eli Lilly,Endpoint Health,Ensho Therapeutics,Enthera,Exeliom Biosciences,Ferring,Flagship Pioneering,Fresenius Kabi,GlaxoSmithKline,Genentech,Gilead,Janssen,Merck,MRM Health,Nxera Therapeutics,Pandion,Pendopharm,Pfizer,Protagonist,Reistone Biopharma,Roche,Roivant,Sandoz,Sanofi,Second Genome,Sorriso,Spyre Therapeutics,Takeda,Teva,Tillotts,Ventyx,Vividion; speaker fees from AbbVie,Ferring,Bristol Myers Squibb,Galapagos,Janssen,Pfizer,Takeda,Tillotts,and Fresenius Kabi.
CM reports consulting fees from AbbVie,Alimentiv,Amgen,AVIR Pharma Inc,Bristol Myers Squibb,Celltrion,Domain Therapeutics,Eupraxia,Eli Lilly,Ferring,Forte Biosciences,Fresenius Kabi,Gilead,Janssen,McKesson,Mirador Therapeutics,Pendopharm,Pfizer,Prometheus Biosciences Inc.,Roche,Sanofi,Takeda,Tillotts Pharma; speaker's fees from AbbVie,Amgen,AVIR Pharma Inc,Alimentiv,Bristol Myers Squibb,Eli Lilly,Ferring,Fresenius Kabi,Janssen,Merck,Organon,Pendopharm,Pfizer,Sanofi,Takeda,Tillotts Pharma; royalties from Springer Publishing; research support from AbbVie,Eli Lilly,Ferring,Pfizer.