Self-Supervised Learning, Mutual Information, and Massively Multimodal Models

by Garrett Merz (University of Wisconsin-Madison)

Friday 16 Jan 2026, 12:30 → 14:30 America/Los_Angeles

48/2-224 - Madrone (SLAC)

Self-supervised pretraining is the engine that powers many state-of-the-art AI “Foundation Models”, many of which integrate a number of data streams and formats into unified representations. I will first briefly touch on some recent applications of self-supervised learning to collider physics, and discuss ways in which these models can be useful for high-energy and astrophysical applications. I will then explore the ongoing debate about the role that mutual information plays in determining when and how these models learn, and discuss a new dataset-generating framework we have developed for investigating these questions in highly multimodal settings.