Ali Ghodsi The Past, Present, and Future of Big Data [Founders Field Guide, EP.18]

Published: Jan. 28, 2021, 10:30 a.m.

b"My Guest today is Ali Ghodsi, founder and CEO of Databricks, a data analytics platform for data scientists and developers. He's also the founder of Apache Spark, the open-source project that Databricks is built on, and is an accomplished researcher at UC Berkley's computer science department. Our conversation ranges from the origins of distributed computing to modern data infrastructure, how companies can leverage their massive datasets, and the transformation of Databricks through its phases of growth as a business. While technical, it's exactly the kind of conversation I like to have on this show. I hope you enjoy my conversation with Ali Ghodsi.\\xa0\\n \\xa0\\n For the full show notes, transcript, and links to mentioned content check out\\xa0https://www.joincolossus.com/episodes/4919706/ghodsi-the-past-present-and-future-of-big-data\\xa0\\n \\xa0\\n This episode of Founder's Field Guide is sponsored by Klaviyo.\\xa0\\xa0Klaviyo is the ultimate marketing platform for ecommerce.\\n With targeted segmentation, email automation, SMS marketing, and more, Klaviyo helps you create your ideal customer experience. See why Klaviyo's trusted by more than 50,000 brands, like Living Proof, Solo Stove, and Nomad to help them grow their business.\\n For a free trial check out\\xa0https://www.klaviyo.com/founders.\\xa0\\n \\xa0\\n This episode is also sponsored by\\xa0Vanta.\\xa0\\xa0Vanta\\xa0has built software that makes it easier to both get and maintain your SOC 2 report, at a fraction of the normal cost.\\xa0Founders Field Guide listeners can redeem a $1k off coupon at\\xa0vanta.com/patrick.\\xa0\\n \\xa0\\n Founder's Field Guide is a property of Colossus Inc. For more episodes of Founder's Field Guide go to\\xa0https://www.joincolossus.com/episodes.\\xa0 \\xa0\\n Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week.\\xa0 Sign up here -\\xa0https://www.joincolossus.com/newsletter.\\n Follow Patrick on Twitter at @patrick_oshag\\n Follow Colossus on Twitter at\\xa0@JoinColossus\\n \\xa0\\n Show Notes\\n [00:02:48] \\u2013 [First question] \\u2013 What is Databricks\\n [00:03:34] \\u2013 History of distributed computing\\n [00:05:35] \\u2013 Hardware that made this all possible\\n [00:07:20] \\u2013 Early challenges in building out these systems\\n [00:09:43] \\u2013 What has made networking technology better\\n [00:10:35] \\u2013 Doing something in storage vs with memory\\n [00:11:45] \\u2013 Origins of Hadoop\\n [00:12:42] \\u2013 Use cases of distributed data in 2010 that weren\\u2019t possible in 2000\\n [00:13:35] \\u2013 Origins of Spark\\n [00:15:25] \\u2013 Early Spark and then the transformation into Databricks\\n [00:16:50] \\u2013 Early uses cases\\n [00:17:37] \\u2013 Their relationship to the open-source project\\n [00:21:07] \\u2013 What customers need in order to work with Databricks\\n [00:23:11] \\u2013 Their customer interaction\\n [00:26:27] \\u2013 How they think about making investments\\n [00:28:24] \\u2013 Their competitive advantage\\n [00:30:13] \\u2013 Other companies in moving the needle in building distributed computing industry\\n [00:32:10] \\u2013 Walls that need to be broken down today\\n [00:34:02] \\u2013 Best practices for companies when it comes to their data\\n \\xa0\\xa0\\xa0\\xa0\\xa0\\xa0\\xa0\\xa0\\xa0\\xa0\\xa0 [00:34:13] \\u2013 Jeff Lawson Podcast Episode\\n [00:38:47] \\u2013 Lessons being a CEO\\n [00:39:53] \\u2013 Working at the University of Berkeley\\u2019s AMPLab\\n [00:41:56] \\u2013 What excites him about the future\\n [00:43:29] \\u2013 Kindest thing anyone has done for him"