What is the name of the methodology?

Pathology Foundation Models & Multiple-Instance Learning (MIL) for Outcome and Therapy-Response Prediction

OI Name

Geert Litjens

What is the methodology good for?

Pathology foundation models are ChatGPT-like models for brightfield microscopy, but instead of text they have been trained on hundreds of thousands (sometimes even millions) of pathology slides, through which they acquired an inherent ‘understanding’ of tissue morphology and composition.

Given a dataset of digitized (whole-slide) pathology images, we use pretrained pathology foundation models to extract features from local tissue regions, and use multiple-instance learning to aggregate these into slide- or patient-level predictions, such as treatment outcome, therapy success/resistance, or survival/recurrence risk, without needing pixel-level annotations. We can use explainability techniques like attention heatmaps to inspect relevant areas within the whole-slide images

What is/are the main advantages of this methodology over related technologies?

Foundation model embeddings capture rich, generalizable morphological features learned from large pathology image corpora, improving robustness across scanners and centers compared to hand-crafted features. MIL removes the need for costly manual region-level annotation, since only a slide- or patient-level label is required for training.

What are the most important limitations of the methodology?

Although pathology foundation models have reduced the need for very large training datasets, we still need reasonably sized datasets (at least 10s of cases). Attention-based MIL provides only coarse interpretability, which can limit clinical trust and adoption. External validation across multiple cohorts is needed before results can be considered clinically robust.

What type of samples are compatible with methodology?

What future develops to the methodology are you planning, in any?

Extend to multimodal integration, combining pathology embeddings with genomic, transcriptomic, or radiology data. 
Improve domain adaptation of foundation models to specific tumor types and staining protocols 
Develop better interpretability/explainability methods for attention-based predictions
Explore spatial biomarker discovery from learned tile embeddings

If someone outside your lab wants to use the methodology, what is the best option?

A) What do you need to provide them to make it work?

- Train a person from their lab

- You will likely need to perform the experiments in your lab, in collaboration (for novel/complex cohorts)

B) Is there any minimal expertise/equipment others need to work with the methodology?

Digital pathology data handling (whole-slide image formats), access to GPU compute, and basic deep learning/Python skills. No wet-lab expertise is required.

Name one or more people in your lab that are experienced with the methodology

Almost everyone, I would assign collaborators based on the cancer of interest (e.g. Ivan Slootweg -> skin cancers, Clement Grisi -> prostate cancer, etc.)

Who originally developed the methodology?

The pipeline builds on publicly available pathology foundation models developed by external groups, combined with an in-house multiple-instance learning framework developed in our lab. Example publications include: https://pubmed.ncbi.nlm.nih.gov/38194735/, https://arxiv.org/abs/2603.14187 or https://pubmed.ncbi.nlm.nih.gov/37437452/ 

Additional information to provide:

Lead: Turning routine pathology slides into predictive biomarkers: our pipeline learns directly from whole-slide images to forecast how a patient's tumor will respond to treatment.

Quote: We believe the real value of foundation models in pathology isn't just better classification, it's giving every digitized slide the potential to answer a clinical question it was never explicitly labeled for.