Next-generation AI for visually occult pancreatic cancer detection in a low-prevalence setting with longitudinal stability and multi-institutional generalisability

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Early detection of pancreatic ductal adenocarcinoma (PDA) is the most powerful approach to improve survival; however, this objective is fundamentally impeded by the morphologically normal appearance of the pancreas on conventional imaging during its curable pre-clinical phase.

• This diagnostic deficit limits the clinical utility of NICE recommendations for urgent imaging in high-risk cohorts (new onset diabetes and weight loss), thereby necessitating technologies to augment standard CT interpretation.

• Initial AI investigations showed potential but were constrained by non-scalable manual methodologies, unrealistic case–control ratios that do not accurately reflect clinical prevalence and a lack of rigorous validation regarding longitudinal stability and multi-institutional generalisability.

WHAT THIS STUDY ADDS

• This study introduces Radiomics-based Early Detection MODel (REDMOD), a next-generation AI framework featuring a robust ensemble architecture (logistic regression, random forest, XGBoost) and fully automated volumetric pancreas segmentations.

• Validated in a large low-prevalence cohort (~6:1 control–case ratio) and incorporating a clinically adaptable design with a modifiable diagnostic threshold, REDMOD detects the subvisual signature of pre-clinical PDA a median of 475 days before clinical diagnosis (AUC 0.82).

• The performance of REDMOD surpasses that of radiologists, demonstrating nearly double the sensitivity (73.0% vs 38.9%; p<0.001) for detecting visually occult PDA, with an advantage that increases to nearly threefold for cases detected more than 24 months prior to diagnosis.

• For the first time, this work establishes the longitudinal stability of a pre-clinical radiomic signal (90–92% test–retest concordance) and confirms the robust specificity of the model across independent multi-institutional (81.3%) and public (87.5%) datasets.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• REDMOD offers an automated and longitudinally stable anchor for multicentre prospective studies in risk-enriched early detection programmes, with pre-specified thresholds and an observe–retest strategy for intermediate results.

• REDMOD could operate as a triage and longitudinal monitoring tool for CT requests triggered by NICE guidelines, aligning with NHS pathways while awaiting prospective clinical evidence and cost-effectiveness.

• The release of a fully automated and externally benchmarked pipeline offers a reproducible platform for method comparisons, bias audits and cost-effectiveness modelling to guide policy.

Study participants

Fully automated AI-driven volumetric pancreas segmentation

A foundational component of the REDMOD framework is the precise and reproducible delineation of the pancreas, which is essential for subsequent radiomic feature extraction. To overcome the inherent limitations and variability of manual contouring and to ensure the scalability required for large-scale quantitative analyses, we implemented a fully automated volumetric pancreas segmentation algorithm. This segmentation was performed using a deep learning model based on the 3D nnU-Net architecture.20 Its development, rigorous validation and detailed performance characteristics are reported in our recent publication.18

Radiomic feature extraction and reduction

A comprehensive panel of 968 quantitative radiomic features was extracted from each automatically segmented pancreas following a standardised preprocessing pipeline that included multiscale image filtering (wavelet and Laplacian-of-Gaussian (LoG)). To reduce dimensionality and select for the most informative biomarkers, the Minimum Redundancy Maximum Relevance (mRMR) algorithm21 was employed, yielding a final set of 40 features for model input. A detailed description of the image preprocessing, feature computation and selection methodology is shown in online supplemental methods section S3.

Ensemble classifier construction, balanced training and feature importance analyses

To develop a classification model capable of distinguishing subtle pre-diagnostic PDA signatures from normal pancreatic tissue, several advanced ML strategies were employed, focusing on addressing class imbalance, leveraging diverse algorithmic strengths and ensuring model interpretability.

Methodological validation and component analyses

To evaluate the impact of key architectural choices and assess methodological robustness, we performed targeted component analyses which included the independent contributions of original versus multiscale filtered radiomic features, the performance of the ensemble classifier against its individual components and the stability of the model’s predictions against expert refinement of the automated pancreas segmentations. The detailed methodology for these validation studies is provided in online supplemental methods section S4.

Benchmarking against clinical practice and evaluating model reliability

To evaluate the clinical translation potential of REDMOD we conducted a series of analyses to benchmark its performance against current practice and to assess its reliability and generalisability.

Statistical analyses

Statistical analyses were performed using R and Python. The primary endpoint was the area under the curve (AUC), with 95% CIs calculated via bootstrapping. The optimal classification threshold was determined using the Youden Index on the training set. Model AUCs were compared using DeLong’s test. A rigorous Multi-Reader Multi-Case (MRMC) ROC analysis was used for the comparison between REDMOD and radiologists. Comprehensive statistical methodologies are shown in online supplemental methods section S5.

Pre-diagnostic cohort is characterised by smaller pancreatic volume

The study included 219 patients with pre-diagnostic PDA (median age 69 years, range 34–88; male to female ratio 1.3) and 1243 control subjects (median age 64 years, range 34–88; ratio 1.4), with comparable demographics across training and test subsets. The median interval from the pre-diagnostic CT scan to histopathological diagnosis of PDA was 427 days (range 90–1092 days) for the full cohort and 475 days (range 97–1076 days) for the 63 cases in the test subset. Lead time distribution was 87 patients (39.7%) at 3–12 months, 76 (34.7%) at 12–24 months and 56 (25.6%) at 24–36 months. Subsequent diagnostic imaging localised PDA primarily to the pancreatic head (64%), with fewer cases in the body (18%) and tail (18%).

AI-driven automated segmentation showed significantly smaller pancreatic volumes in the pre-diagnostic group than in the controls (68.9 (23.8) mL vs 74.7 (23.6) mL; p=0.005). A similar trend was observed in the test subset (67.1 (22.7) mL vs 72.9 (24.1) mL), but this did not reach statistical significance (p=0.09). Pancreatic volumes from independent validation cohorts were consistent, measuring 71.2 (24.8) mL in the multi-institutional set (n=539) and 68.7 (17.5) mL in the NIH-PCT dataset (n=80).

Robust predictive performance driven primarily by filtered radiomic features

From 968 initial radiomic features per pancreatic volume, the mRMR algorithm selected 40 for the REDMOD ensemble classifier. These included four first-order statistical features and 36 higher-order grey-level textural features, of which 36 (90%) originated from wavelet-filtered images, highlighting the central role of multi-scale filtered information in capturing subtle pre-clinical pancreatic alterations. Figure 4 shows the 40 features selected by mRMR.

On the test subset (n=493; 63 pre-diagnostic, 430 control), REDMOD achieved an AUC of 0.82 (95% CI 0.81 to 0.83) for detecting stage 0 PDA. At the optimal classification threshold (0.41, derived from the Youden Index), the model correctly identified 46 of 63 pre-diagnostic CT scans, yielding a sensitivity of 73.0% (95% CI 60.0% to 78.7%), and 349 of 430 control CT scans, yielding a specificity of 81.1% (95% CI 75.2% to 93.1%). The overall performance was strong with an accuracy of 80.1% (95% CI 75.3% to 88.6%), precision of 36.2% (95% CI 31.5% to 60.9%) and F1 score of 48.4% (95% CI 44.3% to 58.1%). Based on the observed AUC of 0.82, the control–case ratio of ~6.8:1 and a significance level (α of 0.05), our test subset size of 493 subjects provided >80% statistical power to detect the observed effect size. This post-hoc analysis confirms that, despite the inherent constraints in accruing pre-diagnostic CT scans, the study was sufficiently powered to validate the performance of REDMOD. Furthermore, given that REDMOD was trained on SMOTE-balanced data but applied to an imbalanced test subset (~14% prevalence), we evaluated the potential for calibration drift using post-hoc calibration (see online supplemental results section S6) to ensure the reliability of the probabilistic outputs in a clinically representative setting. The findings (Expected Calibration Error 0.04, Brier score 0.189) (see online supplemental figure 1) support that REDMOD provides both high discrimination and reliable risk estimation for an early detection paradigm.

Radiologist review of misclassified scans did not reveal systemic anatomical abnormalities or technical artefacts to explain the errors. Two radiologists, blinded to model predictions, also stratified the test subset into a ‘normal pancreas’ subgroup (n=341) and a ‘confounding imaging features’ subgroup (n=152) with the latter comprising cases with cystic lesions, fatty replacement, ductal dilation or irregularity, calcifications or distinct fibrosis/scarring and focal atrophy. We found no statistically significant difference in performance between these groups (AUC 0.79 vs 0.82; DeLong test p=0.90).

Finally, the REDMOD performance remained stable across technical variations: accuracy was comparable by CT vendor (Siemens 78.8% vs non-Siemens 81.8%; p=0.48) and by slice thickness (≤3 mm 80.7% vs >3 mm 76.6%; p=0.49), indicating resilience to common variations in image acquisition.

High sensitivity for stage 0 PDA detection >2 years prior to diagnosis

A critical measure of early detection is the lead time achievable prior to clinical diagnosis. REDMOD maintained high sensitivity across extended pre-diagnostic intervals:

• 3–12 months before diagnosis: 75.0% (15/20)

• 12–24 months before diagnosis: 75.0% (18/24)

• >24 months before diagnosis: 68.4% (13/19)

Importantly, performance was consistent across sources with comparable sensitivity for scans from our institution (72.2% (13/18)) and from external hospitals (73.3% (33/45)), suggesting generalisability across diverse imaging environments. Likewise, the sensitivity to predict pre-diagnostic PDA was spatially invariant (p=0.36), with sensitivities being 79% (30/38) for head lesions, 58% (7/12) for body lesions, 73% (8/11) for tail lesions and 50% (1/2) for multifocal lesions.

High specificity across multi-institutional and public control cohorts

In the large external validation set of 539 control CT scans, REDMOD achieved a specificity of 81.3% (438/539), mirroring its performance on the test subset. On the public NIH-PCT dataset (n=80), specificity was even higher at 87.5% (70/80), further supporting robust generalisability in correctly classifying normal pancreatic tissue.

Ablation studies demonstrate the critical role of filtered features and ensemble architecture

Performance is robust to variations in pancreas segmentation

To test whether minor segmentation inaccuracies drove classification errors we compared the predictions of REDMOD using fully automated contours versus human-refined contours. Of the 98 cases misclassified in the test subset (17 pre-diagnostic, 81 control), radiologists recommended minor segmentation corrections in 50 instances (9 pre-diagnostic, 41 control). Substituting radiomic features from these refined segmentations produced nearly identical results to the originals (accuracy 81.1% vs 80.1%; specificity 82.3% vs 81.1%; AUC 0.82; p=0.62–1.0). These findings confirm that the performance of REDMOD is not materially affected by small segmentation variations, underscoring the reliability of the fully automated pipeline and its ability to extract stable radiomic signatures in a high-performance real-world setting. Figure 6 shows the methodological validation strategies of REDMOD.

REDMOD outperforms expert radiologists

In the head-to-head evaluation, REDMOD showed significantly superior accuracy for detecting pre-diagnostic PDA compared with two independent radiologists (AUC 0.82 vs 0.69; p<0.001; figure 7). REDMOD achieved a sensitivity of 73.0% (95% CI 61.0% to 82.4%), nearly double that of the pooled radiologists (38.9% (95% CI 30.2% to 47.5%); p<0.001).

Given the limited number of pre-diagnostic CT scans within each lead time bin, formal statistical comparison was not feasible but qualitative evaluation indicated a progressive widening of the sensitivity advantage of REDMOD with increasing diagnostic interval:

• 3–12 months: 75.0% vs 50.0% (1.5-fold higher)

• 12–24 months: 75.0% vs 41.0% (1.8-fold higher)

• >24 months: 68.0% vs 23.0% (nearly 3-fold higher)

Performance of the individual radiologists further highlighted the trade-off between sensitivity and specificity. Reader 1 had sensitivity of 55.5% with specificity of 87.2%, while Reader 2 showed lower sensitivity (22.2%) but higher specificity (97.2%). In contrast, REDMOD achieved both balanced and superior performance, with sensitivity of 73.0% and specificity of 81.1%. Notably, inter-reader agreement between the radiologists was modest (weighted Cohen’s κ=0.22), underscoring the variability and subjectivity inherent to human interpretation.

High concordance and reliability on longitudinal scans

The REDMOD framework showed stable predictive performance in a longitudinal test–retest analysis limited to subjects with serial CT scans available.

• Control cohort (n=61): among subjects with serial CT scans available within ±12 months of the index test CT scan, the predictions of REDMOD were concordant on retest. Consistency reached 92% (47/51) for cases initially classified as true negatives and 70% (7/10) for cases initially classified as false positives.

• Pre-diagnostic cohort (n=10): using earlier pre-index CT scans performed a median of 210 days before the index CT scan (range 6–761 days), the model preserved its true positive classification in 90% (9/10). For these correctly classified serial CT scans the median lead time to PDA diagnosis was 675 days (range 505–993).

Together, these results show that outputs of REDMOD remained reproducible across temporally distinct studies, which is further illustrated in figure 8.

AI-driven radiomics as a solution to the challenge of stage 0 PDA detection

Early detection of PDA at a curable stage is a formidable challenge due to its imperceptibility on conventional imaging during the crucial pre-clinical window (stage 0).4 24 25 This ‘invisibility’ of early-stage disease on standard imaging, combined with the aggressive and rapid progression characteristic of PDA,26 mandates the development of innovative solutions. While studies on smaller datasets have shown initial promise in identifying pre-diagnostic signals,9–13 REDMOD was evaluated on a substantially larger (n=1462) multi-institutional cohort to closely reflect the real-world early detection prevalence (~6.8:1 control–pre-diagnostic case ratio). Under these stringent conditions, REDMOD detected the subtle textural and architectural signatures of stage 0 PDA with high sensitivity (73%) and accuracy (AUC 0.82) at a median lead time of 475 days. This temporal window holds profound significance, as attaining such early detection would substantially augment the probability of cure and improved survival. In fact, modelling studies indicate that increasing the proportion of localised PDAs from 10% to 50% would more than double survival rates,2 27 28 thereby underscoring that the timing of diagnosis is the single most critical determinant of survival outcomes.

Furthermore, REDMOD demonstrated superior accuracy to radiologists (AUC 0.82 vs 0.69; p<0.001). It is imperative to contextualise these metrics within the operational realities of an early detection paradigm. In cohorts characterised by a high control–case ratio, the AUC metric can be misleading because increased specificity across a substantial proportion of normal cases can disproportionately inflate the AUC, thereby obscuring a clinically significant deficiency in sensitivity. This statistical fallacy is evident in the performance metrics of the radiologists. Their high specificities (87–97%) yielded a respectable AUC despite a pooled sensitivity of merely 38.9% and substantial interobserver variability (κ=0.22). The paramount objective of an early detection test is to achieve a clinically efficacious equilibrium: to accurately identify rare instances of disease (sensitivity) while concurrently maintaining an acceptably low false-positive rate (specificity). In this regard, REDMOD showed a superior equilibrium, with a profoundly greater sensitivity (73.0% vs 38.9%) while simultaneously preserving a robust specificity (81.1%). This balanced performance, particularly a nearly threefold higher sensitivity (68.0% vs 23.0%) at the longest lead times (>24 months), suggests that the primary strength of REDMOD lies in its capacity to identify the earliest imaging signatures of visually occult disease. Finally, this complementary performance profile—higher sensitivity from REDMOD and high specificity from radiologists—suggests that an AI-augmented workflow could lead to a synergistic improvement in overall accuracy for early PDA detection. In the absence of visible tumour cues, however, automation bias—where readers uncritically accept AI predictions—remains a risk. To address this, our upcoming prospective AI-PACED (Artificial Intelligence for Pancreatic Cancer Early Detection) trial29 will quantify automation bias and establish optimal override criteria so that algorithmic alerts prompt appropriate risk-stratified evaluation rather than premature intervention.

At the prespecified threshold (0.41; Youden Index), REDMOD delivered 36.2% precision while maintaining 73.0% sensitivity, notably at a visually occult stage where all CT scans had been prospectively reported as negative for PDA. Its precision already exceeds the 3% precision recommended by NICE at the first step of UK cancer referral, where avoiding missed cancers takes precedence over false positives.7 Established early detection programmes reinforce this principle: low-dose CT and mammography accept lower precision initially, relying on follow-up imaging to adjudicate borderline findings30 31 to boost precision. For instance, nodule volume and doubling time stratify lung cancer risk after low-dose CT,32 while mammographic comparison reduces recall rates while balancing initial sensitivity.33 In this context, the 90–92% test–retest concordance of REDMOD and signal persistence of at least 12–24 months support an ‘observe-and-retest’ strategy for intermediate results. Moreover, unlike low-dose CT and mammography, which are predicated on radiologically visible abnormalities, REDMOD identifies PDA signatures before any lesion is apparent. Thus, it extends the window of detection into a pre-diagnostic phase.

Conversely, there is a need for higher precision when invasive procedures are considered. Guided by these principles, the Youden Index-derived operative threshold of REDMOD was intentionally designed to be adjustable to pretest risk. Crucially, this design allows recalibration without retraining, enabling thresholds to be shifted to emphasise sensitivity or precision in accordance with clinical context. Weighted or asymmetrical indices can further penalise false positives where procedural risk is high. This flexibility permits alignment with clinical objectives across the pathway—non-invasive triage early, confirmatory imaging next and stringent precision before surgery.

Methodological advances and their implications for robust early detection

REDMOD integrates automated pancreas segmentation18 for scalability and reproducibility. Our analysis confirmed the robustness of the downstream classification to minor segmentation inaccuracies (AUC unchanged at 0.82; p=0.62–1.0). Furthermore, unlike approaches that focus on a limited number of pre-selected features, REDMOD used an extensive feature engineering pipeline. This pipeline commenced with 968 radiomic features per patient and employed the mRMR algorithm to derive a parsimonious and highly predictive set of 40 features. A key insight is the predominance (90% of selected features) and superior contribution of filtered (wavelet and LoG) features vis-à-vis original unfiltered features (AUC 0.82 vs 0.74, p=0.007) to the predictive power of REDMOD. This suggests that the earliest manifestations of PDA are subtle multiscale textural disruptions and alterations in local intensity gradients, which are more effectively captured by these advanced feature engineering techniques. To elucidate the mechanistic basis of REDMOD, an ablation study systematically removed feature classes to quantify performance degradation. Textural heterogeneity features (eg, specific wavelet-transformed GLCM, GLSZM and GLDM metrics) emerged as the most predictive, which aligns with prior evidence that early carcinogenesis disrupts pancreatic parenchymal architecture at a microscopic level before mass formation.34 35 These findings suggest that REDMOD detects biologically relevant remodelling associated with early malignancy. Furthermore, the use of a heterogeneous ensemble classifier, which averaged the probabilistic outputs of three distinct algorithms, had the highest sensitivity (73.0%) among all configurations with comparable AUC (0.82), suggesting the soft-voting mechanism provides a more balanced classification.

Limitations and future directions

We acknowledge the limitations of this study. First, the study was not designed to evaluate performance across different racial and ethnic groups, a critical consideration for future validation given known disparities in PDA risk among individuals with gNOD.36 Second, while specificity was validated on external cohorts, sensitivity requires validation on public pre-diagnostic datasets which are not yet available. Third, the REDMOD framework was intentionally designed to assess the entire pancreatic gland, reflecting the hypothesis that stage 0 disease manifests as a diffuse field effect rather than a discrete focal lesion. At this early visually occult stage, there is no discernible lesion to serve as ground truth for training a localisation algorithm. Consequently, REDMOD could serve a distinct and critical purpose as an advanced risk-enrichment tool. A ‘high-risk’ flag from the model could prompt further investigation with molecular imaging (eg, using fibroblast activation protein (FAP)-targeted positron emission tomography radiotracers) capable of pinpointing areas of cellular activity to direct endoscopic ultrasound-guided sampling. Moreover, the therapeutic landscape for early stage PDA is undergoing rapid evolution. The emergence of systemic interception strategies, such as novel multi-antigen nanovaccines engineered to elicit a curative antitumour immune response throughout the organ,37 may obviate the necessity for lesion localisation. Within such a paradigm, the primary clinical imperative transitions from identifying a focal lesion for resection to identification of the high-risk individual (HRI) whose immune system warrants activation, a role for which a robust whole-gland risk assessment tool like REDMOD is eminently suitable. Finally, this study validates the diagnostic performance of REDMOD rather than PDA survival outcomes, so classical screening biases such as lead time and length time bias do not directly affect our primary endpoints. Nonetheless, future prospective evaluations must apply methods to account for these biases and competing risks of death, as in hepatocellular carcinoma surveillance,38 39 to define the true net benefit of early interception.

Supplemental material

Patient consent for publication

Not applicable.

Ethics approval

This retrospective minimal risk case-control study received Mayo Clinic Institutional Review Board approval (ID: 19-003018), complied with the HIPAA and was granted a waiver of informed consent.