Inhalte/ Contents

Patients generate multimodal clinical data (imaging, signals, reports, genomics), but jointly leveraging these sources to support clinical decisions and improve patient outcomes has long been limited by computing and architecture constraints. Recent advances in multimodal AI/LLMs and GPU computing make it feasible to build models that synthesize heterogeneous evidence. This research tutorial centers on Symile (https://arxiv.org/abs/2411.01053) and replacing a softmax-based loss with a sigmoid-based loss (Sigmile): we reproduce the baseline, then try to exploit the sigmoid formulation's parallelization to scale beyond three modalities.  Target group: advanced BSc/MSc students in CS or related fields. Prerequisites: strong Python and deep learning basics (PyTorch preferred); familiarity with representation/contrastive learning is a plus. Outcome: a publishable paper

Kontakt/ Contact

benedikt.peterson@fu-berlin.de

Link zum Vorlesungsverzeichnis/ Link to the course catalogue

folgt/ coming