Podcasts Technology Screaming in the Cloud Learning eBPF with Liz Rice

Learning eBPF with Liz Rice

Published: May 2, 2023, 10 a.m.

b'

Liz Rice, Chief Open Source Officer at Isovalent, joins Corey on Screaming in the Cloud to discuss the release of her newest book, Learning eBPF, and the exciting possibilities that come with eBPF technology. Liz explains what got her so excited about eBPF technology, and what it was like to write a book while also holding a full-time job. Corey and Liz also explore the learning curve that comes with kernel programming, and Liz illustrates why it\\u2019s so important to be able to explain complex technologies in simple terminology.


About Liz

Liz Rice is Chief Open Source Officer with eBPF specialists Isovalent, creators of the Cilium cloud native networking, security and observability project. She sits on the CNCF Governing Board, and on the Board of OpenUK. She was Chair of the CNCF\'s Technical Oversight Committee in 2019-2022, and Co-Chair of KubeCon + CloudNativeCon in 2018. She is also the author of Container Security, and Learning eBPF, both published by O\'Reilly.

She has a wealth of software development, team, and product management experience from working on network protocols and distributed systems, and in digital technology sectors such as VOD, music, and VoIP. When not writing code, or talking about it, Liz loves riding bikes in places with better weather than her native London, competing in virtual races on Zwift, and making music under the pseudonym Insider Nine.

Links Referenced:



Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.


Corey: Welcome to Screaming in the Cloud. I\\u2019m Corey Quinn. Our returning guest today is Liz Rice, who remains the Chief Open Source Officer with Isovalent. But Liz, thank you for returning, suspiciously closely timed to when you have a book coming out. Welcome back.


Liz: [laugh]. Thanks so much for having me. Yeah, I\\u2019ve just\\u2014I\\u2019ve only had the physical copy of the book in my hands for less than a week. It\\u2019s called Learning eBPF. I mean, obviously, I\\u2019m very excited.


Corey: It\\u2019s an O\\u2019Reilly book; it has some form of honeybee on the front of it as best I can tell.


Liz: Yeah, I was really pleased about that. Because eBPF has a bee as its logo, so getting a [early 00:01:17] honeybee as the O\\u2019Reilly animal on the front cover of the book was pretty pleasing, yeah.


Corey: Now, this is your second O\\u2019Reilly book, is it not?


Liz: It\\u2019s my second full book. So, I\\u2019d previously written a book on Container Security. And I\\u2019ve done a few short reports for them as well. But this is the second, you know, full-on, you can buy it on Amazon kind of book, yeah.


Corey: My business partner wrote Practical Monitoring for O\\u2019Reilly and that was such an experience that he got entirely out of observability as a field and ran running to AWS bills as a result. So, my question for you is, why would anyone do that more than once?


Liz: [laugh]. I really like explaining things. And I had a really good reaction to the Container Security book. I think already, by the time I was writing that book, I was kind of interested in eBPF. And we should probably talk about what that is, but I\\u2019ll come to that in a moment.


Yeah, so I\'ve been really interested in eBPF, for quite a while and I wanted to be able to do the same thing in terms of explaining it to people. A book gives you a lot more opportunity to go into more detail and show people examples and get them kind of hands-on than you can do in their, you know, 40-minute conference talk. So, I wanted to do that. I will say I have written myself a note to never do a full-size book while I have a full-time job because it\\u2019s a lot [laugh].


Corey: You do have a full-time job and then some. As we mentioned, you\\u2019re the Chief Open Source Officer over at Isovalent, you are on the CNCF governing board, you\\u2019re on the board of OpenUK, and you\\u2019ve done a lot of other stuff in the open-source community as well. So, I have to ask, taking all of that together, are you just allergic to things that make money? I mean, writing the book as well on top of that. I\\u2019m told you never do it for the money piece; it\\u2019s always about the love of it. But it seems like, on some level, you\\u2019re taking it to an almost ludicrous level.


Liz: Yeah, I mean, I do get paid for my day job. So, there is that [laugh]. But so, yeah\\u2014


Corey: I feel like that\\u2019s the only way to really write a book is, in turn, to wind up only to just do it for\\u2014what someone else is paying you to for doing it, viewing it as a marketing exercise. It pays dividends, but those dividends don\\u2019t, in my experience from what I\\u2019ve heard from everyone say, pay off as of royalties on book payments.


Liz: Yeah, I mean, it\\u2019s certainly, you know, not a bad thing to have that income stream, but it certainly wouldn\\u2019t make you\\u2014you know, I\\u2019m not going to retire tomorrow on the royalty stream unless this podcast has loads and loads of people to buy the book [laugh].


Corey: Exactly. And I\\u2019m always a fan of having such [unintelligible 00:03:58]. I will order it while we\\u2019re on the call right now having this conversation because I believe in supporting the things that we want to see more of in the world. So, explain to me a little bit about what it is. Whatever you talking about learning X in a title, I find that that\\u2019s often going to be much more approachable than arcane nonsense deep-dive things.


One of the O\\u2019Reilly books that changed my understanding was Linux Kernel Internals, or Understanding the Linux Kernel. Understanding was kind of a heavy lift at that point because it got very deep very quickly, but I absolutely came away understanding what was going on a lot more effectively, even though I was so slow I needed a tow rope on some of it. When you have a book that started with learning, though, I imagined it assumes starting at zero with, \\u201cWhat\\u2019s eBPF?\\u201d Is that directionally correct, or does it assume that you know a lot of things you don\\u2019t?


Liz: Yeah, that\\u2019s absolutely right. I mean, I think eBPF is one of these technologies that is starting to be, particularly in the cloud-native world, you know, it comes up; it\\u2019s quite a hot technology. What it actually is, so it\\u2019s an acronym, right? EBPF. That acronym is almost meaningless now.


So, it stands for extended Berkeley Packet Filter. But I feel like it does so much more than filtering, we might as well forget that altogether. And it\\u2019s just become a term, a name in its own right if you like. And what it really does is it lets you run custom programs in the kernel so you can change the way that the kernel behaves, dynamically. And that is\\u2026 it\\u2019s a superpower. It\\u2019s enabled all sorts of really cool things that we can do with that superpower.


Corey: I just pre-ordered it as a paperback on Amazon and it shows me that it is now number one new release in Linux Networking and Systems Administration, so you\\u2019re welcome. I\\u2019m sure it was me that put it over the top.


Liz: Wonderful. Thank you very much. Yeah [laugh].


Corey: Of course, of course. Writing a book is one of those things that I\\u2019ve always wanted to do, but never had the patience to sit there and do it or I thought I wasn\\u2019t prolific enough, but over the holidays, this past year, my wife and business partner and a few friends all chipped in to have all of the tweets that I\\u2019d sent bound into a series of leather volumes. Apparently, I\\u2019ve tweeted over a million words. And\\u2026 yeah, oh, so I have to write a book 280 characters at a time, mostly from my phone. I should tweet less was really the takeaway that I took from a lot of that.


But that wasn\\u2019t edited, that wasn\\u2019t with an overall theme or a narrative flow the way that an actual book is. It just feels like a term paper on steroids. And I hated term papers. Love reading; not one to write it.


Liz: I don\\u2019t know whether this should make it into the podcast, but it reminded me of something that happened to my brother-in-law, who\\u2019s an artist. And he put a piece of video on YouTube. And for unknowable reasons if you mistyped YouTube, and you spelt it, U-T-U-B-E, the page that you would end up at from Google search was a YouTube video and it was in fact, my brother-in-law\\u2019s video. And people weren\\u2019t expecting to see this kind of art movie about matches burning. And he just had the worst comment\\u2014like, people were so mean in the comments. And he had millions of views because people were hitting this page by accident, and he ended up\\u2014


Corey: And he made the cardinal sin of never read the comments. Never break that rule. As soon as you do that, it doesn\\u2019t go well. I do read the comments on various podcast platforms on this show because I always tell people to insulted all they want, just make sure you leave a five-star review.


Liz: Well, he ended up publishing a book with these comments, like, one comment per page, and most of them are not safe for public consumption comments, and he just called it Feedback. It was quite something [laugh].


Corey: On some level, it feels like O\\u2019Reilly books are a little insulated from the general population when it comes to terrible nonsense comments, just because they tend to be a little bit more expensive than the typical novel you\\u2019ll see in an airport bookstore, and again, even though it is approachable, Learning eBPF isn\\u2019t exactly the sort of title that gets people to think that, \\u201cOoh, this is going to be a heck of a thriller slash page-turner with a plot.\\u201d \\u201cWell, I found the protagonist unrelatable,\\u201d is not sort of the thing you\\u2019re going to wind up seeing in the comments because people thought it was going to be something different.


Liz: I know. One day, I\\u2019m going to have to write a technical book that is also a murder mystery. I think that would be, you know, quite an achievement. But yeah, I mean, it\\u2019s definitely aimed at people who have already come across the term, want to know more, and particularly if you\\u2019re the kind of person who doesn\\u2019t want to just have a hand-wavy explanation that involves boxes and diagrams, but if, like me, you kind of want to feel the code, and you want to see how things work and you want to work through examples, then that\\u2019s the kind of person who might\\u2014I hope\\u2014enjoy working through the book and end up with a possible mental model of how eBPF works, even though it\\u2019s essentially kernel programming.


Corey: So, I keep seeing eBPF in an increasing number of areas, a bunch of observability tools, a bunch of security tools all tend to tie into it. And I\\u2019ve seen people do interesting things as far as cost analysis with it. The problem that I run into is that I\\u2019m not able to wind up deploying it universally, just because when I\\u2019m going into a client engagement, I am there in a purely advisory sense, given that I\\u2019m biasing these days for both SaaS companies and large banks, that latter category is likely going to have some problems if I say, \\u201cOh, just take this thing and go ahead and deploy it to your entire fleet.\\u201d If they don\\u2019t have a problem with that, I have a problem with their entire business security posture. So, I don\\u2019t get to be particularly prescriptive as far as what to do with it.


But if I were running my own environment, it is pretty clear by now that I would have explored this in some significant depth. Do you find that it tends to be something that is used primarily in microservices environments? Does it effectively require Kubernetes to become useful on day one? What is the onboard path where people would sit back and say, \\u201cAh, this problem I\\u2019m having, eBPF sounds like the solution.\\u201d


Liz: So, when we write tools that are typically going to be some sort of infrastructure, observability, security, networking tools, if we\\u2019re writing them using eBPF, we\\u2019re instrumenting the kernel. And the kernel gets involved every time our application wants to do anything interesting because whenever it wants to read or write to a file, or send receive network messages, or write something to the screen, or allocate memory, or all of these things, the kernel has to be involved. And we can use eBPF to instrument those events and do interesting things. And the kernel doesn\\u2019t care whether those processes are running in containers, under Kubernetes, just running directly on the host; all of those things are visible to eBPF.


So, in one sense, doesn\\u2019t matter. But one of the reasons why I think we\\u2019re seeing eBPF-based tools really take off in cloud-native is that you can, by applying some programming, you can link events that happened in the kernel to specific containers in specific pods in whatever namespace and, you know, get the relationship between an event and the Kubernetes objects that are involved in that event. And then that enables a whole lot of really interesting observability or security tools and it enables us to understand how network packets are flowing between different Kubernetes objects and so on. So, it\\u2019s really having this vantage point in the kernel where we can see everything and we didn\\u2019t have to change those applications in any way to be able to use eBPF to instrument them.


Corey: When I see the stories about eBPF, it seems like it\\u2019s focused primarily on networking and flow control. That\\u2019s where I\\u2019m seeing it from a security standpoint, that\\u2019s where I\\u2019m seeing it from cost allocation aspect. Because, frankly, out of the box, from a cloud provider\\u2019s perspective, Kubernetes looks like a single-tenant application with a really weird behavioral pattern, and some of that crosstalk gets very expensive. Is there a better way than either using eBPF and/or VPC flow logs to figure out what\\u2019s talking to what in the Kubernetes ecosystem, or is BPF really your first port of call?


Liz: So, I\\u2019m coming from a position of perspective of working for the company that created the Cilium networking project. And one of the reasons why I think Cilium is really powerful is because it has this visibility\\u2014it\\u2019s got a component called Hubble\\u2014that allows you to see exactly how packets are flowing between these different Kubernetes identities. So, in a Kubernetes environment, there\\u2019s not a lot of point having network flows that talk about IP addresses and ports when what you really want to know is, what\\u2019s the Kubernetes namespace, what\\u2019s the application? Defining things in terms of IP addresses makes no sense when they\\u2019re just being refreshed and renewed every time you change pods. So yeah, Kubernetes changes the requirements on networking visibility and on firewalling as well, on network policy, and that, I think, is you don\\u2019t have to use eBPF to create those tools, but eBPF is a really powerful and efficient platform for implementing those tools, as we see in Cilium.


Corey: The only competitor I found to it that gives a reasonable explanation of why random things are transferring multiple petabytes between each other in the middle of the night has been oral tradition, where I\\u2019m talking to people who\\u2019ve been around there for a while. It\\u2019s, \\u201cSo, I\\u2019m seeing this weird traffic pattern at these times a day. Any idea what that might be?\\u201d And someone will usually perk up and say, \\u201cOh, is it\\u2014\\u201d whatever job that they\\u2019re doing. Great. That gives me a direction to go in.


But especially in this era of layoffs and as environments exist for longer and longer, you have to turn into a bit of a data center archaeologist. That remains insufficient, on some level. And some level, I\\u2019m annoyed with trying to understand or needing to use tooling like this that is honestly this powerful and this customizable, and yes, on some level, this complex in order to get access to that information in a meaningful sense. But on the other, I\\u2019m glad that that option is at least there for a lot of workloads.


Liz: Yeah. I think, you know, that speaks to the power of this new generation of tooling. And the same kind of applies to security forensics, as well, where you might have an enormous stream of events, but unless you can tie those events back to specific Kubernetes identities, which you can use eBPF-based tooling to do, then how do you\\u2014the forensics job of tying back where did that event come from, what was the container that was compromised, it becomes really, really difficult. And eBPF tools\\u2014like Cilium has a sub-project called Tetragon that is really good at this kind of tying events back to the Kubernetes pod or whether we want to know what node it was running on what namespace or whatever. That\\u2019s really useful forensic information.


Corey: Talk to me a little bit about how broadly applicable it is. Because from my understanding from our last conversation, when you were on the show a year or so ago, if memory serves, one of the powerful aspects of it was very similar to what I\\u2019ve seen some of Brendan Gregg\\u2019s nonsense doing in his kind of various talks where you can effectively write custom programming on the fly and it\\u2019ll tell you exactly what it is that you need. Is this something that can be instrument once and then effectively use it for basically anything, [OTEL 00:16:11]-style, or instead, does it need to be effectively custom configured every time you want to get a different aspect of information out of it?


Liz: It can be both of those things.


Corey: \\u201cIt depends.\\u201d My least favorite but probably the most accurate answer to hear.


Liz: [laugh]. But I think Brendan did a really great\\u2014he\\u2019s done many talks talking about how powerful BPF is and built lots of specific tools, but then he\\u2019s also been involved with Bpftrace, which is kind of like a language for\\u2014a high-level language for saying what it is that you want BPF to trace out for you. So, a little bit like, I don\\u2019t know, awk but for events, you know? It\\u2019s a scripting language. So, you can have this flexibility.


And with something like Bpftrace, you don\\u2019t have to get into the weeds yourself and do kernel programming, you know, in eBPF programs. But also there\\u2019s gainful employment to be had for people who are interested in that eBPF kernel programming because, you know, I think there\\u2019s just going to be a whole range of more tools to come, you know>? I think we\\u2019re, you know, we\\u2019re seeing some really powerful tools with Cilium and Pixie and [Parker 00:17:27] and Kepler and many other tools and projects that are using eBPF. But I think there\\u2019s also a whole load of more to come as people think about different ways they can apply eBPF and instrument different parts of an overall system.


Corey: We\\u2019re doing this over audio only, but behind me on my wall is one of my least favorite gifts ever to have been received by anyone. Mike, my business partner, got me a thousand-piece puzzle of the Kubernetes container landscape where\\u2014


Liz: [laugh].


Corey: This diagram is psychotic and awful and it looks like a joke, except it\\u2019s not. And building that puzzle was maddening\\u2014obviously\\u2014but beyond that, it was a real primer in just how vast the entire container slash Kubernetes slash CNCF landscape really is. So, looking at this, I found that the only reaction that was appropriate was a sense of overwhelmed awe slash frustration, I guess. It\\u2019s one of those areas where I spend a lot of time focusing on drinking from the AWS firehose because they have a lot of products and services because their product strategy is apparently, \\u201cYes,\\u201d and they\\u2019re updating these things in a pretty consistent cadence. Mostly. And even that feels like it\\u2019s multiple full-time jobs shoved into one.


There are hundreds of companies behind these things and all of them are in areas that are incredibly complex and difficult to go diving into. EBPF is incredibly powerful, I would say ridiculously so, but it\\u2019s also fiendishly complex, at least shoulder-surfing behind people who know what they\\u2019re doing with it has been breathtaking, on some level. How do people find themselves in a situation where doing a BPF deep dive make sense for them?


Liz: Oh, that\\u2019s a great question. So, first of all, I\\u2019m thinking is there an AWS Jigsaw as well, like the CNCF landscape Jigsaw? There should be. And how many pieces would it have? [It would be very cool 00:19:28].


Corey: No, because I think the CNCF at one point hired a graphic designer and it\\u2019s unclear that AWS has done such a thing because their icons for services are, to be generous here, not great. People have flashcards that they\\u2019ve built for is what services does logo represent? Haven\\u2019t a clue, in almost every case because I don\\u2019t care in almost every case. But yeah, I\\u2019ve toyed with the idea of doing it. It\\u2019s just not something that I\\u2019d ever want to have my name attached to it, unfortunately. But yeah, I want someone to do it and someone else to build it.


Liz: Yes. Yeah, it would need to refresh every, like, five minutes, though, as they roll out a new service.


Corey: Right. Because given that it appears from the outside to be impenetrable, it\\u2019s similar to learning VI in some cases, where oh, yeah, it\\u2019s easy to get started with to do this trivial thing. Now, step two, draw the rest of the freaking owl. Same problem there. It feels off-putting just from a perspective of you must be at least this smart to proceed. How do you find people coming to it?


Liz: Yeah, there is some truth in that, in that beyond kind of Hello World, you quite quickly start having to do things with kernel data structures. And as soon as you\\u2019re looking at kernel data structures, you have to sort of understand, you know, more about the kernel. And if you change things, you need to understand the implications of those changes. So, yeah, you can rapidly say that eBPF programming is kernel programming, so why would anybody want to do it? The reason why I do it myself is not because I\\u2019m a kernel programmer; it\\u2019s because I wanted to really understand how this is working and build up a mental model of what\\u2019s happening when I attach a program to an event. And what kinds of things can I do with that program?


And that\\u2019s the sort of exploration that I think I\\u2019m trying to encourage people to do with the book. But yes, there is going to be at some point, a pretty steep learning curve that\\u2019s kernel-related but you don\\u2019t necessarily need to know everything in order to really have a decent understanding of what eBPF is, and how you might, for example\\u2014you might be interested to see what BPF programs are running on your existing system and learn why and what they might be doing and where they\\u2019re attached and what use could that be.


Corey: Falling down that, looking at the process table once upon a time was a heck of an education, one week when I didn\\u2019t have a lot to do and I didn\\u2019t like my job in those days, where, \\u201cOh, what is this Avahi daemon that constantly running? MDNS forwarding? Who would need that?\\u201d And sure enough, that tickled something in the back of my mind when I wound up building out my networking box here on top of BSD, and oh, yeah, I want to make sure that I can still have discovery work from the IoT subnet over to whatever it is that my normal devices live. Ah, that\\u2019s what that thing always running for. Great for that one use case. Almost never needed in other cases, but awesome. Like, you fire up a Raspberry Pi. It\\u2019s, \\u201cWhy are all these things running when I\\u2019m just want to have an embedded device that does exactly one thing well?\\u201d Ugh. Computers have gotten complicated.


Liz: I know. It\\u2019s like when you get those pop-ups on\\u2014well certainly on Mac, and you get pop-ups occasionally, let\\u2019s say there\\u2019s such and such a daemon wants extra permissions, and you think I\\u2019m not hitting that yes button until I understand what that daemon is. And it turns out, it\\u2019s related, something completely innocuous that you\\u2019ve actually paid for, but just under a different name. Very annoying. So, if you have some kind of instrumentation like tracing or logging or security tooling that you want to apply to all of your containers, one of the things you can use is a sidecar container approach. And in Kubernetes, that means you inject the sidecar into every single pod. And\\u2014


Corey: Yes. Of course, the answer to any Kubernetes problem appears to be have you tried running additional containers?


Liz: Well, right. And there are challenges that can come from that. And one of the reasons why you have to do that is because if you want a tool that has visibility over that container that\\u2019s inside the pod, well, your instrumentation has to also be inside the pod so that it has visibility because your pod is, by design, isolated from the host it\\u2019s running on. But with eBPF, well eBPF is in the kernel and there\\u2019s only one kernel, however many containers were running. So, there is no kind of isolation between the host and the containers at the kernel level.


So, that means if we can instrument the kernel, we don\\u2019t have to have a separate instance in every single pod. And that\\u2019s really great for all sorts of resource usage, it means you don\\u2019t have to worry about how you get those sidecars into those pods in the first place, you know that every pod is going to be instrumented if it\\u2019s instrumented in the kernel. And then for service mesh, service mesh usually uses a sidecar as a Layer 7 Proxy injected into every pod. And that actually makes for a pretty convoluted networking path for a packet to sort of go from the application, through the proxy, out to the host, back into another pod, through another proxy, into the application.


What we can do with eBPF, we still need a proxy running in userspace, but we don\\u2019t need to have one in every single pod because we can connect the networking namespaces much more efficiently. So, that was essentially the basis for sidecarless service mesh, which we did in Cilium, Istio, and now we\\u2019re using a similar sort of approach with Ambient Mesh. So that, again, you know, avoiding having the overhead of a sidecar in every pod. So that, you know, seems to be the way forward for service mesh as well as other types of instrumentation: avoiding sidecars.


Corey: On some level, avoiding things that are Kubernetes staples seems to be a best practice in a bunch of different directions. It feels like it\\u2019s an area where you start to get aligned with the idea of service meesh\\u2014yes, that\\u2019s how I pluralize the term service mesh and if people have a problem with that, please, it\\u2019s imperative you\\u2019ve not send me letters about it\\u2014but this idea of discovering where things are in a variety of ways within a cluster, where things can talk to each other, when nothing is deterministically placed, it feels like it is screaming out for something like this.


Liz: And when you think about it, Kubernetes does sort of already have that at the level of a service, you know? Services are discoverable through native Kubernetes. There\\u2019s a bunch of other capabilities that we tend to associate with service mesh like observability or encrypted traffic or retries, that kind of thing. But one of the things that we\\u2019re doing with Cilium, in general, is to say, but a lot of this is just a feature of the networking, the underlying networking capability. So, for example, we\\u2019ve got next generation mutual authentication approach, which is using SPIFFE IDs between an application pod and another application pod. So, it\\u2019s like the equivalent of mTLS.


But the certificates are actually being passed into the kernel and the encryption is happening at the kernel level. And it\\u2019s a really neat way of saying we don\\u2019t need\\u2026 we don\\u2019t need to have a sidecar proxy in every pod in order to terminate those TLS connections on behalf of the application. We can have the kernel do it for us and that\\u2019s really cool.


Corey: Yeah, at some level, I find that it still feels weird\\u2014because I\\u2019m old\\u2014to have this idea of one shared kernel running a bunch of different containers. I got past that just by not requiring that [unintelligible 00:27:32] workloads need to run isolated having containers run on the same physical host. I found that, for example, running some stuff, even in my home environment for IoT stuff, things that I don\\u2019t particularly trust run inside of KVM on top of something as opposed to just running it as a container on a cluster. Almost certainly stupendous overkill for what I\\u2019m dealing with, but it\\u2019s a good practice to be in to start thinking about this. To my understanding, this is part of what AWS\\u2019s Firecracker project starts to address a bit more effectively: fast provisioning, but still being able to use different primitives as far as isolation boundaries go. But, on some level, it\\u2019s nice to not have to think about this stuff, but that\\u2019s dangerous.


Liz: [laugh]. Yeah, exactly. Firecracker is really nice way of saying, \\u201cActually, we\\u2019re going to spin up a whole VM,\\u201d but we don\\u2019t ne\\u2014when I say \\u2018whole VM,\\u2019 we don\\u2019t need all of the things that you normally get in a VM. We can get rid of a ton of things and just have the essentials for running that Lambda or container service, and it becomes a really nice lightweight solution. But yes, that will have its own kernel, so unlike, you know, running multiple kernels on the same VM where\\u2014sorry, running multiple containers on the same virtual machine where they would all be sharing one kernel, with Firecracker you\\u2019ll get a kernel per instance of Firecracker.


Corey: The last question I have for you before we wind up wrapping up this episode harkens back to something you said a little bit earlier. This stuff is incredibly technically nuanced and deep. You clearly have a thorough understanding of it, but you also have what I think many people do not realize is an orthogonal skill of being able to articulate and explain those complex concepts simply an approachably, in ways that make people understand what it is you\\u2019re talking about, but also don\\u2019t feel like they\\u2019re being spoken to in a way that\\u2019s highly condescending, which is another failure mode. I think it is not particularly well understood, particularly in the engineering community, that there are\\u2014these are different skill sets that do not necessarily align congruently. Is this something you\\u2019ve always known or is this something you\\u2019ve figured out as you\\u2019ve evolved your career that, oh I have a certain flair for this?


Liz: Yeah, I definitely didn\\u2019t always know it. And I started to realize it based on feedback that people have given me about talks and articles I\\u2019d written. I think I\\u2019ve always felt that when people use jargon or they use complicated language or they, kind of, make assumptions about how things are, it quite often speaks to them not having a full understanding of what\\u2019s happening. If I want to explain something to myself, I\\u2019m going to use straightforward language to explain it to myself [laugh] so I can hold it in my head. And I think people appreciate that.


And you can get really\\u2014you know, you can get quite in-depth into something if you just start, step by step, build it up, explain everything as you go along the way. And yeah, I think people do appreciate that. And I think people, if they get lost in jargon, it doesn\\u2019t help anybody. And yeah, I very much appreciate it when people say that, you know, they saw a talk or they read something I wrote and it meant that they finally grokked whatever that concept was that that I was trying to explain. I will say at the weekend, I asked ChatGPT to explain DNS in the style of Liz Rice, and it started off, it was basically, \\u201cHello there. I\\u2019m Liz Rice and I\\u2019m here to explain DNS in very simple terms.\\u201d I thought, \\u201cOkay.\\u201d [laugh].


Corey: Every time I think I\\u2019ve understood DNS, there\\u2019s another level to it.


Liz: I\\u2019m pretty sure there is a lot about DNS that I don\\u2019t understand, yeah. So, you know, there\\u2019s always more to learn out there.


Corey: There\\u2019s certainly is. I really want to thank you for taking time to speak with me today about what you\\u2019re up to. Where\\u2019s the best place for people to find you to learn more? And of course, to buy the book.


Liz: Yeah, so I am Liz Rice pretty much everywhere, all over the internet. There is a GitHub repo that accompanies the books that you can find that on GitHub: lizRice/learning-eBPF. So, that\\u2019s a good place to find some of the example code, and it will obviously link to where you can download the book or buy it because you can pay for it; you can also download it from Isovalent for the price of your contact details. So, there are lots of options.


Corey: Excellent. And we will, of course, put links to that in the [show notes 00:32:08]. Thank you so much for your time. It\\u2019s always great to talk to you.


Liz: It\\u2019s always a pleasure, so thanks very much for having me, Corey.


Corey: Liz Rice, Chief Open Source Officer at Isovalent. I\\u2019m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you\\u2019ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you\\u2019ve hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that you have somehow discovered this episode by googling for knitting projects.


Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.


'