BROADCASTS.com

b'

Imagine you\\u2019re a big hedge fund, and you want to go out and buy yourself some data. Data is really valuable for you \\u2014 it\\u2019s literally going to shape your investment decisions and determine your outcomes.

But the moment you receive your data, a cold chill runs down your spine: how do you know your data supplier gave you the data they said they would? From your perspective, you\\u2019re staring down 100,000 rows in a spreadsheet, with no way to tell if half of them were made up \\u2014 or maybe more for that matter.

This might seem like an obvious problem in hindsight, but it\\u2019s one most of us haven\\u2019t even thought of. We tend to assume that data is data, and that 100,000 rows in a spreadsheet is 100,000 legitimate samples.

The challenge of making sure you\\u2019re dealing with high-quality data, or at least that you have the data you think you do, is called data observability, and it\\u2019s surprisingly difficult to solve for at scale. In fact, there are now entire companies that specialize in exactly that \\u2014 one of which is Zectonal, whose co-founder Dave Hirko will be joining us for today\\u2019s episode of the podcast.

Dave has spent his career understanding how to evaluate and monitor data at massive scale. He did that first at AWS in the early days of cloud computing, and now through Zectonal, where he\\u2019s working on strategies that allow companies to detect issues with their data \\u2014 whether they\\u2019re caused by intentional data poisoning, or unintentional data quality problems. Dave joined me to talk about data observability, data as a new vector for cyberattacks, and the future of enterprise data management on this episode of the TDS podcast.

***

Intro music:

- Artist: Ron Gelinas

- Track Title: Daybreak Chill Blend (original mix)

- Link to Track: https://youtu.be/d8Y2sKIgFWc

***

Chapters:

\\n
• 0:00 Intro
\\n
• 3:00 What is data observability?
\\n
• 10:45 \\u201cFunny business\\u201d with data providers
\\n
• 12:50 Data supply chains
\\n
• 16:50 Various cybersecurity implications
\\n
• 20:30 Deep data inspection
\\n
• 27:20 Observed direction of change
\\n
• 34:00 Steps the average person can take
\\n
• 41:15 Challenges with GDPR transitions
\\n
• 48:45 Wrap-up
\\n