Podcasts Technology The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Rethinking Model Size: Train Large, Then Compress with Joseph Gonzalez - #378

Rethinking Model Size: Train Large, Then Compress with Joseph Gonzalez - #378

Published: May 25, 2020, 1:59 p.m.

Today we\u2019re joined by Joseph Gonzalez, Assistant Professor in the EECS department at UC Berkeley. \n\nIn our conversation, we explore Joseph\u2019s paper \u201cTrain Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers,\u201d which looks at compute-efficient training strategies for models. We discuss the two main problems being solved; 1) How can we rapidly iterate on variations in architecture? And 2) If we make models bigger, is it really improving any efficiency?