Transformers On Large-Scale Graphs with Bayan Bruss - #641

Published: Aug. 7, 2023, 4:15 p.m.

b'Today we\\u2019re joined by Bayan Bruss, Vice President of Applied ML Research at Capital One. In our conversation with Bayan, we covered a pair of papers his team presented at this year\\u2019s ICML conference. We begin with the paper Interpretable Subspaces in Image Representations, where Bayan gives us a dive deep into the interpretability framework, embedding dimensions, contrastive approaches, and how their model can accelerate image representation in deep learning. We also explore GOAT:\\xa0A Global Transformer on Large-scale Graphs, a scalable global graph transformer. We talk through the computation challenges, homophilic and heterophilic principles, model sparsity, and how their research proposes methodologies to get around the computational barrier when scaling to large-scale graph models.\\n\\nThe complete show notes for this episode can be found at twimlai.com/go/641.'