Podcasts Technology The Cloudcast Economics & Optimization of AI/ML

Economics & Optimization of AI/ML

Published: Aug. 30, 2023, 5 a.m.

b'

Luis Ceze (@luisceze, Founder/CEO @OctoML) talks about barriers to entry for AI & ML, the economics of funding, training, fine tuning, inferencing and optimizations.

SHOW: 749

CLOUD NEWS OF THE WEEK - http://bit.ly/cloudcast-cnotw

NEW TO CLOUD? CHECK OUT - "CLOUDCAST BASICS"

SHOW SPONSORS:

  • CloudZero \\u2013 Cloud Cost Visibility and Savings
  • \\u200b\\u200bCloudZero provides immediate and ongoing savings with 100% visibility into your total cloud spend
  • Reduce the complexities of protecting your workloads and applications in a multi-cloud environment. Panoptica provides comprehensive cloud workload protection integrated with API security to protect the entire application lifecycle.\\xa0 Learn more about Panoptica at panoptica.app

SHOW NOTES:

Topic 1 - Welcome to the show. You have an interesting background with roots in both VC markets and academia. Tell us a little bit about your background.

Topic 2 - Generative AI is now all the rage. But as more people dig into AI/ML in general, they find out quickly there are a few barriers to entry. Let\\u2019s address some of them as you have an extensive history here. The first barrier I believe most people hit is complexity. The tools to ingest data into models and deployment of models has improved but what about the challenges implementing that into production applications? How do folks overcome this first hurdle?

Topic 3 - The next hurdle I think most organizations hit is where to place the models. Where to train them, where to fine tune them and where to run them could be the same or different places. Can you talk a bit about placement of models? Also, as a follow up, how does GPU shortages play into this and can models be fine tuned to work around this?

Topic 4 - Do you see the AI/ML dependence on GPU\\u2019s continuing into the future? Will there be an abstraction layer or another technology coming that will allow the industry to move away from GPU\\u2019s from more mainstream applications?

Topic 5 - The next barrier but very related to the previous one is cost. There are some very real world tradeoffs between cost and performance when it comes to AI/ML. What cost factors need to be considered besides hardware costs? Data ingestion and data gravity comes to mind as a hidden cost that can add up quickly if not properly considered. Another one is latency. Maybe you arrive at an answer but at a slower rate that is more economical. How do organizations optimize for cost?

Topic 6 - Do most organizations tend to use an \\u201coff the shelf model\\u201d today? Maybe an open source model that they train with their private data? I would expect this to be the fastest way to production, why build your own model when the difference is in your data? How does data privacy factor into this scenario?

FEEDBACK?

'