Publications

Visit PANCAIM on Zenodo!

All publication, open-access datasets and AI algorithms developed during the project can be found on our Zenodo community.

Fully Automatic Deep Learning Framework for Pancreatic Ductal Adenocarcinoma Detection on Computed Tomography

Early detection improves prognosis in pancreatic ductal adenocarcinoma (PDAC), but is challenging as lesions are often small and poorly defined on contrast-enhanced computed tomography scans (CE-CT). Deep learning can facilitate PDAC diagnosis; however, current models still fail to identify small (<2 cm) lesions. In this study, state-of-the-art deep learning models were used to develop an automatic framework for PDAC detection, focusing on small lesions. Additionally, the impact of integrating the surrounding anatomy was investigated. CE-CT scans from a cohort of 119 pathology-proven PDAC patients and a cohort of 123 patients without PDAC were used to train a nnUnet for automatic lesion detection and segmentation (nnUnet_T). Two additional nnUnets were trained to investigate the impact of anatomy integration: (1) segmenting the pancreas and tumor (nnUnet_TP), and (2) segmenting the pancreas, tumor, and multiple surrounding anatomical structures (nnUnet_MS). An external, publicly available test set was used to compare the performance of the three networks. The nnUnet_MS achieved the best performance, with an area under the receiver operating characteristic curve of 0.91 for the whole test set and 0.88 for tumors <2 cm, showing that state-of-the-art deep learning can detect small PDAC and benefits from anatomy information.

Citation: Alves, N.; Schuurmans, M.; Litjens, G.; Bosma, J.S.; Hermans, J.; Huisman, H. Fully Automatic Deep Learning Framework for Pancreatic Ductal Adenocarcinoma Detection on Computed Tomography. Cancers 2022, 14, 376.

Setting the Research Agenda for Clinical Artificial Intelligence in Pancreatic Adenocarcinoma Imaging

Pancreatic ductal adenocarcinoma (PDAC), estimated to become the second leading cause of cancer deaths in western societies by 2030, was flagged as a neglected cancer by the European Commission and the United States Congress. Due to lack of investment in research and development, combined with a complex and aggressive tumour biology, PDAC overall survival has not significantly improved the past decades. Cross-sectional imaging and histopathology play a crucial role throughout the patient pathway. However, current clinical guidelines for diagnostic workup, patient stratification, treatment response assessment, and follow-up are non-uniform and lack evidence-based consensus. Artificial Intelligence (AI) can leverage multimodal data to improve patient outcomes, but PDAC AI research is too scattered and lacking in quality to be incorporated into clinical workflows. This review describes the patient pathway and derives touchpoints for image-based AI research in collaboration with a multi-disciplinary, multi-institutional expert panel. The literature exploring AI to address these touchpoints is thoroughly retrieved and analysed to identify the existing trends and knowledge gaps. The results show absence of multi-institutional, well-curated datasets, an essential building block for robust AI applications. Furthermore, most research is unimodal, does not use state-of-the-art AI techniques, and lacks reliable ground truth. Based on this, the future research agenda for clinically relevant, image-driven AI in PDAC is proposed.

Citation: Schuurmans, M.; Alves, N.; Vendittelli, P.; Huisman, H.; Hermans, J. Setting the Research Agenda for Clinical Artificial Intelligence in Pancreatic Adenocarcinoma Imaging. Cancers 2022, 14, 3498. https://doi.org/10.3390/cancers14143498

Automatic tumour segmentation in H&E-stained whole-slide images of the pancreas

Pancreatic cancer will soon be the second leading cause of cancer-related death in Western society. Imaging techniques such as CT, MRI and ultrasound typically help providing the initial diagnosis, but histopathological assessment is still the gold standard for final confirmation of disease presence and prognosis. In recent years machine learning approaches and pathomics pipelines have shown potential in improving diagnostics and prognostics in other cancerous entities, such as breast and prostate cancer. A crucial first step in these pipelines is typically identification and segmentation of the tumour area. Ideally this step is done automatically to prevent time consuming manual annotation. We propose a multi-task convolutional neural network to balance disease detection and segmentation accuracy. We validated our approach on a dataset of 29 patients (for a total of 58 slides) at different resolutions. The best single task segmentation network achieved a median Dice of 0.885 (0.122) IQR at a resolution of 15.56 μm. Our multi-task network improved on that with a median Dice score of 0.934 (0.077) IQR.

Citation: Vendittelli, Pierpaolo, Esther MM Smeets, and G. J. S. Litjens. "Automatic tumour segmentation in H&E-stained whole-slide images of the pancreas." Medical Imaging 2022: Digital and Computational Pathology. Vol. 12039. SPIE, 2022.

Artificial Intelligence in Pancreatic Ductal Adenocarcinoma Imaging: A Commentary on Potential Future Applications

Cross-sectional imaging such as computed tomography (CT) plays a crucial role in PDAC (pancreatic cancer) management. In the last years, Artificial intelligence (AI) has gained considerable interest in oncology, as it has the potential to support clinicians and ultimately guide decision making at each step of the patient pathway by focusing on well-validated applications at meaningful clinical touchpoints. AI applications can leverage high amounts of data to produce individualized recommendations based on each patient’s clinical picture. But while the number of publications on AI for clinical decision-making in oncology has increased exponentially in the past few years, AI research in PDAC is still at a preliminary stage compared to other cancer diseases, with limited private and public datasets and a lack of independent, external model validation. As a result, no AI applications have been implemented in clinical practice for PDAC. The article identifies the definition of the research questions to be addressed by AI algorithms as a first step towards developing clinically-relevant AI and suggests to do so based on five steps in the patient pathway: Detection, Diagnosis, Staging, Treatment, Treatment monitoring. In each step along the patient pathway, the articles identifies the critical touchpoints that are lacking in clinical practice, where image-based AI could have the greatest impact for patients and clinicians. In general, the authors find that AI holds the potential to bring transformative changes into healthcare, and PDAC is particularly suited to benefit from AI research and the development of commercial applications since current clinical practices still lead to poor patient outcomes.

Citation: Schuurmans, Megan, et al. "Artificial Intelligence in Pancreatic Ductal Adenocarcinoma Imaging: A Commentary on Potential Future Applications." Gastroenterology (2023).

Tutorial on survival modeling with applications to omics data

Identification of genomic, molecular and clinical markers prognostic of patient survival is important for developing personalized disease prevention, diagnostic and treatment approaches. Modern omics technologies have made it possible to investigate the prognostic impact of markers at multiple molecular levels, including genomics, epigenomics, transcriptomics, proteomics and metabolomics, and how these potential risk factors complement clinical characterization of patient outcomes for survival prognosis. However, the massive sizes of the omics datasets, along with their correlation structures, pose challenges for studying relationships between the molecular information and patients’ survival outcomes. We present a general workflow for survival analysis that is applicable to high-dimensional omics data as inputs when identifying survival-associated features and validating survival models. In particular, we focus on the commonly used Cox-type penalized regressions and hierarchical Bayesian models for feature selection in survival analysis, which are especially useful for high-dimensional data, but the framework is applicable more generally. A step-by-step R tutorial using The Cancer Genome Atlas survival and omics data for the execution and evaluation of survival models has been made available at https://ocbe-uio.github.io/survomics.

Citation: Zhi Zhao, John Zobolas, Manuela Zucknick, Tero Aittokallio, Tutorial on survival modeling with applications to omics data, Bioinformatics, Volume 40, Issue 3, March 2024, btae132, https://doi.org/10.1093/bioinformatics/btae132

The PANORAMA Study Protocol: Pancreatic Cancer Diagnosis - Radiologists Meet AI

This publication presents the preregistration of the PANORAMA study protocol, including the BIAS checklist for transparent reporting of biomedical image analysis challenges. The PANORAMA study is a new prospectively designed multi-center study to assess the performance of both radiologists and AI at PDAC detection in routine abdominal CECT scans. PANORAMA’s goals are threefold: To establish the clinical baseline performance of radiologists at PDAC detection through a large-scale, international reader study To establish the state-of-the-art AI performance at PDAC detection through an international AI grand-challenge To compare AI and radiologists, with the end goal of obtaining substantial evidence to start implementing AI to help find PDAC earlier Key aspects of the PANORAMA study protocol have been established in conjunction with an international scientific advisory board of 13 multidisciplinary experts on pancreas AI, clinical workflows, and statistics, as well as a patient representative, to ensure the development and validation of meaningful AI towards clinical translation (Reinke et al., 2021).

Citation: Alves, N., Schuurmans, M., Rutkowski, D., Yakar, D., Haldorsen, I., Liedenbaum, M., Molven, A., Vendittelli, P., Litjens, G., Hermans, J., & Huisman, H. (2024). The PANORAMA Study Protocol: Pancreatic Cancer Diagnosis - Radiologists Meet AI. Zenodo. https://doi.org/10.5281/zenodo.10599559

Explainable multi-omics deep clustering reveals an important role of DNA methylation in PDAC

Patients with pancreatic ductal adenocarcinoma (PDAC) have the lowest survival rate among all cancer patients in Europe. Since western societies have the highest incidence of pancreatic cancer, it has been projected that PDAC will soon become the second leading cause of cancer-related deaths. The main challenge of PDAC treatment is that patients with similar somatic genotypes exhibit a wide range of disease phenotypes. Artificial Intelligence (AI) is currently transforming the field of healthcare and represents a promising technology for integrating various datasets and optimizing evidence-based decision making. However, the interpretability of most AI models is limited and it is challenging to understand how and why a decision is made. In this study, we developed a deep clustering model for PDAC patient stratification using integrated methylation and gene expression data. We placed a specific emphasis on model explainability, with the aim to understand hidden multi-modal patterns learned by the model. The model resulted in two subgroups of PDAC patients with different prognoses and biological factors. We performed several follow-up analyses to measure the relative contribution of each modality to the clustering solution. This multi-omics profile analysis revealed an important role of DNA methylation, partially supported by previous experimental studies. We also show how the model learned the underlying patterns in a multi-modal setting, where individual hidden neurons are specialized either in single data modalities or their combinations. We hope this study will help to promote more explainable AI in real-world clinical applications, where the knowledge of the decision factors is crucial. The code of this project is publicly available in GitHub (https://github.com/albertolzs/edc_mo_pdac).

Citation: López, A., Zobolas, J., Nebdal, D., Lingjærde, O. C., Fleischer, T., & Aittokallio, T. (2024). Explainable multi-omics deep clustering reveals an important role of DNA methylation in PDAC. https://doi.org/10.5281/zenodo.10635657

Automatic quantification of tumor-stroma ratio as a prognostic marker for pancreatic cancer

This study aims to introduce an innovative multi-step pipeline for automatic tumor-stroma ratio (TSR) quantification as a potential prognostic marker for pancreatic cancer, addressing the limitations of existing staging systems and the lack of commonly used prognostic biomarkers. The proposed approach involves a deep-learning-based method for the automatic segmentation of tumor epithelial cells, tumor bulk, and stroma from whole-slide images (WSIs). Models were trained using five-fold cross-validation and evaluated on an independent external test set. TSR was computed based on the segmented components. Additionally, TSR’s predictive value for six-month survival on the independent external dataset was assessed. Median Dice (inter-quartile range (IQR)) of 0.751(0.15) and 0.726(0.25) for tumor epithelium segmentation on internal and external test sets, respectively. Median Dice of 0.76(0.11) and 0.863(0.17) for tumor bulk segmentation on internal and external test sets, respectively. TSR was evaluated as an independent prognostic marker, demonstrating a cross-validation AUC of 0.61±0.12 for predicting six-month survival on the external dataset. Our pipeline for automatic TSR quantification offers promising potential as a prognostic marker for pancreatic cancer. The results underscore the feasibility of computational biomarker discovery in enhancing patient outcome prediction, thus contributing to personalized patient management.

Citation: Vendittelli P, Bokhorst J-M, Smeets EMM, Kryklyva V, Brosens LAA, Verbeke C, et al. (2024) Automatic quantification of tumor-stroma ratio as a prognostic marker for pancreatic cancer. PLoS ONE 19(5): e0301969. https://doi.org/10.1371/journal.pone.0301969

Prediction Variability to Identify Reduced AI Performance in Cancer Diagnosis at MRI and CT

The article evaluates AI prediction variability as an uncertainty quantification (UQ) metric for identifying cases at risk of AI failure in diagnosing cancer at MRI and CT across different cancer types, data sets, and algorithms. Multicenter data sets and publicly available AI algorithms from three previous studies that evaluated detection of pancreatic cancer on contrast-enhanced CT images, detection of prostate cancer on MRI scans, and prediction of pulmonary nodule malignancy on low-dose CT images were analyzed retrospectively. In total, 18 022 images were used for training and 838 images were used for testing. The authors find that an artificial intelligence (AI) prediction uncertainty quantification metric consistently identified reduced AI performance in cancer diagnosis at MRI and CT across different cancer types, data sets, and algorithms.

Citation: Alves, Natália, et al. "Prediction variability to identify reduced AI performance in cancer diagnosis at MRI and CT." Radiology 308.3 (2023): e230275. https://doi.org/10.1148/radiol.230275

Examining properness in the external validation of survival models with squared and logarithmic losses

Scoring rules promote rational and honest decision-making, which is becoming increasingly important for automated procedures in `auto-ML'. In this paper we survey common squared and logarithmic scoring rules for survival analysis and determine which losses are proper and improper. We prove that commonly utilised squared and logarithmic scoring rules that are claimed to be proper are in fact improper, such as the Integrated Survival Brier Score (ISBS). We further prove that under a strict set of assumptions a class of scoring rules is strictly proper for, what we term, `approximate' survival losses. Despite the difference in properness, experiments in simulated and real-world datasets show there is no major difference between improper and proper versions of the widely-used ISBS, ensuring that we can reasonably trust previous experiments utilizing the original score for evaluation purposes. We still advocate for the use of proper scoring rules, as even minor differences between losses can have important implications in automated processes such as model tuning. We hope our findings encourage further research into the properties of survival measures so that robust and honest evaluation of survival models can be achieved.

A Large-Scale Neutral Comparison Study of Survival Models on Low-Dimensional Data

This work presents the first large-scale neutral benchmark experiment focused on single-event, right-censored, low-dimensional survival data. Benchmark experiments are essential in methodological research to scientifically compare new and existing model classes through proper empirical evaluation. Existing benchmarks in the survival literature are often narrow in scope, focusing, for example, on high-dimensional data. Additionally, they may lack appropriate tuning or evaluation procedures, or are qualitative reviews, rather than quantitative comparisons. This comprehensive study aims to fill the gap by neutrally evaluating a broad range of methods and providing generalizable conclusions. We benchmark 18 models, ranging from classical statistical approaches to many common machine learning methods, on 32 publicly available datasets. The benchmark tunes for both a discrimination measure and a proper scoring rule to assess performance in different settings. Evaluating on 8 survival metrics, we assess discrimination, calibration, and overall predictive performance of the tested models. Using discrimination measures, we find that no method significantly outperforms the Cox model. However, (tuned) Accelerated Failure Time models were able to achieve significantly better results with respect to overall predictive performance as measured by the right-censored log-likelihood. Machine learning methods that performed comparably well include Oblique Random Survival Forests under discrimination, and Cox-based likelihood-boosting under overall predictive performance. We conclude that for predictive purposes in the standard survival analysis setting of low-dimensional, right-censored data, the Cox Proportional Hazards model remains a simple and robust method, sufficient for practitioners.

Can our experience with surveillance for inherited pancreatic cancer help to identify early pancreatic cancer in the general population?

Screening of the general population for cancer is a matter of primary prevention reducing the burden of disease. Whilst this is successful for several cancers including breast, colon and prostate, the situation to screen and hence prevent pancreatic cancer is different. The organ is not as accessible to simple physical exam or biological samples (fecal or blood test). Neither exists a blood test such as PSA that is cost-effective. Reviewing the evidence from screening risk groups for pancreatic cancer, one must conclude that there is no rational at present to screen the general population, for a lack of appropriate tests.

Citation: Löhr, JM., Öhlund, D., Söreskog, E. et al. Can our experience with surveillance for inherited pancreatic cancer help to identify early pancreatic cancer in the general population?. Familial Cancer 23, 399–403 (2024). https://doi.org/10.1007/s10689-024-00363-6