BROADCASTS.com

Published: June 1, 2023, 9:42 a.m.

b'

# Summary

\\n

Did you ever had the experience that you where training a network, investing a lot of time in finding the right hyper parameters and testing different initializations to push that validation accuracy over certain threshold? Only to then find out when putting the model into production, that it significantly underperforms?

\\n

If you did, then you experienced one common problem with deep neural networks. The performance gap between in and out-of distribution generalization.

\\n

Today on the show PhD Rahim Entezari is giving us a wonderful tour through his PhD journey investigating ways to understand and improve generalization performance of deep neural networks.

\\n

Rahim will explain how one can improve generalization by different methods in data or in parameter space.

\\n

We will discuss how using different forms of sparsity, or the efficient creation of deep ensemble networks by permutation of network configurations can improve generalization from a parameter space perspective.

\\n

Or, from a data perspective where we discuss how data quality and data diversity effects the generalization performance of modern deep neural networks.

\\n

I hope you enjoy this interview, full of interesting concepts and ideas from deep learning theory.

\\n

\\n

# TOC

\\n

00:00:00 Introduction

\\n

00:02:18 Background Knowledge

\\n

00:06:56 Guest Introduction

\\n

00:12:35 Generalization from a Data or Parameter Perspective

\\n

00:16:21 In and out of distribution Generalization

\\n

00:20:30 Structured and Unstructured Sparsity

\\n

00:29:55 Generalization in Parameter space

\\n

00:46:56 Generalization in Data space

\\n

\\n

# Sponsors

\\n

Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

\\n

Belichberg GmbH: We do digital transformations as your innovation partner - https://belichberg.com/

\\n

\\n

# References

\\n

Rahim Entezari - https://www.linkedin.com/in/rahimentezari/

'