PET scans allows to visualize a physiological process, rather than the anatomy of a human body. As the technology is rather new, it leaves little doubt that PET scans contain a lot of disease-specific information that is underused or even completely overlooked in current clinical practice. Classically, the identification of such disease-specific information is troublesome as it involves the collection of a lot of clinically annotated data. However, recently foundation models have been proposed as means to mitigate the need for large-scale annotated data. Foundation models use self-supervised learning to learn general purpose, compact representations or “embeddings” of images. Then, using simple downstream models, these embeddings can be leveraged for specific tasks such as the outcome prediction of a therapy or the detection of aggressive tumors in the lungs.
In this internship you will contribute to the first foundation model for nuclear medicine. You will build on top of the popular dino(v2) framework and use self-supervised learning to create robust embeddings for 2D PET scans, by finetuning the model proposed by Meta on a large dataset of PET images. Using several problems such as patient stratification, denoising and segmentation, you will assess the quality of the embeddings. The aim of this work is to compare the performance of the general purpose embeddings of Meta to the ones that have been finetuned on PET images. This is a challenging topic, so prior experience with deep learning is required.

Reference:
https://arxiv.org/abs/2304.07193

Python
Deep Learning
PyTorch