In 2017, Vaswani et al. declared “attention is all you need.” What once seemed exaggerated has become increasingly plausible. Transformers now power models across language, vision, and science. But reusing design choices from language models raises a key question: are they optimal for other modalities? This talk examines Transformer-based time-series foundation models, showing how their rank structure, frequency biases, and uncertainty modeling reveal the need for modality-specific design choices.

About the speaker: Annan Yu is a final-year Ph.D. candidate in the Center for Applied Mathematics at Cornell University, advised by Alex Townsend. His research at Cornell spans numerical linear algebra and the theory of deep learning. He collaborates closely with Michael Mahoney and Benjamin Erichson at Lawrence Berkeley National Laboratory on advancing the expressivity, robustness, and efficiency of state-space models. He has also worked with Amazon Web Services on understanding Transformer-based time-series foundation models.