BROADCASTS.com

b'

Google the phrase \\u201cAI over-hyped\\u201d, and you\\u2019ll find literally dozens of articles from the likes of Forbes, Wired, and Scientific American, all arguing that \\u201cAI isn\\u2019t really as impressive at it seems from the outside,\\u201d and \\u201cwe still have a long way to go before we come up with *true* AI, don\\u2019t you know.\\u201d

Amusingly, despite the universality of the \\u201cAI is over-hyped\\u201d narrative, the statement that \\u201cWe haven\\u2019t made as much progress in AI as you might think\\u2122\\ufe0f\\u201d is often framed as somehow being an edgy, contrarian thing to believe.

All that pressure not to over-hype AI research really gets to people\\u200a\\u2014\\u200aresearchers included. And they adjust their behaviour accordingly: they over-hedge their claims, cite outdated and since-resolved failure modes of AI systems, and generally avoid drawing straight lines between points that clearly show AI progress exploding across the board. All, presumably, to avoid being perceived as AI over-hypers.

Why does this matter? Well for one, under-hyping AI allows us to stay asleep\\u200a\\u2014\\u200ato delay answering many of the fundamental societal questions that come up when widespread automation of labour is on the table. But perhaps more importantly, it reduces the perceived urgency of addressing critical problems in AI safety and AI alignment.

Yes, we need to be careful that we\\u2019re not over-hyping AI. \\u201cAI startups\\u201d that don\\u2019t use AI are a problem. Predictions that artificial general intelligence is almost certainly a year away are a problem. Confidently prophesying major breakthroughs over short timescales absolutely does harm the credibility of the field.

But at the same time, we can\\u2019t let ourselves be so cautious that we\\u2019re not accurately communicating the true extent of AI\\u2019s progress and potential. So what\\u2019s the right balance?

That\\u2019s where Sam Bowman comes in. Sam is a professor at NYU, where he does research on AI and language modeling. But most important for today\\u2019s purposes, he\\u2019s the author of a paper titled, \\u201cWhen combating AI hype, proceed with caution,\\u201d in which he explores a trend he calls under-claiming\\u200a\\u2014\\u200aa common practice among researchers that consists of under-stating the extent of current AI capabilities, and over-emphasizing failure modes in ways that can be (unintentionally) deceptive.

Sam joined me to talk about under-claiming and what it means for AI progress on this episode of the Towards Data Science podcast.

***

Intro music: 

- Artist: Ron Gelinas

- Track Title: Daybreak Chill Blend (original mix)

- Link to Track: https://youtu.be/d8Y2sKIgFWc 

***

Chapters: 

\\n
• 2:15 Overview of the paper
\\n
• 8:50 Disappointing systems
\\n
• 13:05 Potential double standard
\\n
• 19:00 Moving away from multi-modality
\\n
• 23:50 Overall implications
\\n
• 28:15 Pressure to publish or perish
\\n
• 32:00 Announcement discrepancies
\\n
• 36:15 Policy angle
\\n
• 41:00 Recommendations
\\n
• 47:20 Wrap-up
\\n