BROADCASTS.com

b'

Jeremy Snyder, Founder of FireTail, joins Corey on Screaming in the Cloud to discuss his career journey and what led him to start FireTail. Jeremy reveals what\\u2019s changed in cloud since he was an AE and AWS, and walks through how the need for customization in cloud security has led to a boom in the number of security companies out there. Corey and Jeremy also discuss the costs of cloud security, and Jeremy points out some of his observations in the world of cloud security pricing and packaging.\\xa0

About Jeremy

Jeremy is the founder and CEO of FireTail.io, an end-to-end API security startup. Prior to FireTail, Jeremy worked in M&A at Rapid7, a global cyber leader, where he worked on the acquisitions of 3 companies during the pandemic. Jeremy previously led sales at DivvyCloud, one of the earliest cloud security posture management companies, and also led AWS sales in southeast Asia. Jeremy started his career with 13 years in cyber and IT operations. Jeremy has an MBA from Mason, a BA in computational linguistics from UNC, and has completed additional studies in Finland at Aalto University. Jeremy speaks 5 languages and has lived in 5 countries. Once, Jeremy went 5 days without seeing another human, but saw plenty of reindeer.

Links Referenced:

\\n
• Firetail: https://firetail.io\\n
\\n
• Email: jeremy@firetail.io
  \\n
\\n

Transcript

Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.

Corey: Welcome to Screaming in the Cloud. I\\u2019m Corey Quinn. My guest today is Jeremy Snyder, who\\u2019s the founder at Firetail. Jeremy, thank you for joining me today. I appreciate you taking the time from your day to suffer my slings and arrows.

Jeremy: My pleasure, Corey. I\\u2019m really happy to be here.

Corey: So, we\\u2019ll get to a point where we talk about what you\\u2019re up to these days, but first, I want to dive into the jobs of yesteryear because over a decade ago, you did a stint at AWS doing sales. And not to besmirch your hard work, but it feels like at the time, that must have been a very easy job. Because back then it really felt across the board like the sales motion was basically responding to, \\u201cWell, why should we do business with you?\\u201d And the response is, \\u201cOh, you misunderstand. You have 87 different accounts scattered throughout your organization. I\\u2019m just here to give you visibility, governance, and possibly some discounting over that.\\u201d It feels like times have changed in a lot of ways since then. Is that accurate?

Jeremy: Well, yeah, but I will correct a couple of things in there. In my days\\u2014

Corey: Oh, please.

Jeremy: \\u2014almost nobody had more than one account. I was in the one account, no VPCs, you know, you only separate your workloads by tagging days of AWS. So, our job was a lot, actually, harder at the time because people couldn\\u2019t wrap their heads around the lack of subnetting, the lack of workload segregation. All of that was really, like, brand new to people, and so you were trying to tell them like, \\u201cHey, you\\u2019re going to be launching something on an EC2 instance that\\u2019s in the same subnet as everybody else\\u2019s EC2 instance.\\u201d And people were really worried about lateral traffic and sniffing and what could their neighbors or other customers on AWS see. And by the way, I mean, this was the customers who even believed it was real. You know, a lot of the conversations we went into with people was, \\u201cOh, so Amazon bought too many servers and you\\u2019re trying to sell us excess capacity.\\u201d

Corey: That legend refuses to die.

Jeremy: And, you know, it is a legend. That is not at all the genesis of AWS. And you know, the genesis is pretty well publicized at this point; you can go just google, \\u201chow did AWS started?\\u201d You can find accurate stuff around that.

Corey: I did it a few years ago with multiple Amazon execs and published it, and they said definitively that that story was not true. And you can say a lot about AWS folks, and I assure you, I do, but I also do not catch them lying to my face, ever. And as soon as that changes, well, now we\\u2019re going to have a different series of [laugh] conversations that are a lot more pointed. But they\\u2019ve earned some trust there.

Jeremy: Yeah, I would agree. And I mean, look, I saw it internally, the way that Amazon built stuff was at such a breakneck pace, that challenge that they had that was, you know, the published version of events for why AWS got created, developers needed a place to test code. And that was something that they could not get until they got EC2, or could not get in a reasonably enough timeframe for it to be, you know, real-time valid or relevant for what was going on with the company. So, you know, that really is the genesis of things, and you know, the early services, SQS, S3, EC2, they all really came out of that journey. But yeah, in our days at AWS, there was a lot of ease, in the sense that lots of customers had pent-up frustrations with their data center providers or their colo providers and lots of customers would experience bursts and they would have capacity constraints and they would need a lot of the features that AWS offered, but we had to overcome a lot of technical misunderstandings and trust issues and, you know, oh, hey, Amazon just wants to sniff our data and they want to see what we\\u2019re up to, and explain to them how encryption works and why they have their own keys and all these things. You know, we had to go through a lot of that. So, it wasn\\u2019t super easy, but there was some element of it where, you know, just demand actually did make some aspects easy.

Corey: What have you seen change since, well I guess ten years ago and change now? And let\\u2019s be clear, you don\\u2019t work in AWS sales, but you also are not oblivious to what the market is doing.

Jeremy: For sure. For sure. I left AWS in 2011 and I\\u2019ve stayed in the cloud ecosystem pretty much ever since. I did spend some time working for a system integrator where all we did was migrate customers to AWS. And then I spent about five, six years working on cloud security primarily focused on AWS, a lot of GCP, a little bit of Azure.

So yeah, I mean, I certainly stay up to date with what\\u2019s going on in the state of cloud. I mean, look, Cloud has evolved from this kind of, you know, developer-centric, very easy-to-launch type of platform into a fully-fledged enterprise IT platform and all of the management structures and all of the kind of bells and whistles that you would want that you probably wanted from your old VMware networks but never really got, they\\u2019re all there now. It is a very different ballgame in terms of what the platform actually enables you to do, but fundamentally, a lot of the core building block constructs and the primitives are still kind of driving the heart of it. It\\u2019s just a lot of nicer packaging.

What I think is really interesting is actually how customers\\u2019 usage of cloud platforms has changed over time. And I always think of it and kind of like the, going back to my days, what did I see from my customers? And it was kind of like the month zero, \\u201cI just don\\u2019t believe you.\\u201d Like, \\u201cThis thing can\\u2019t be real, I don\\u2019t trust it, et cetera.\\u201d Month one is, I\\u2019m going to assign some developer to work on some very low-priority, low-risk workload. In my days, that was SharePoint, by the way. Like, nine times out of ten, the first workload that customers stood up was a SharePoint instance that they had to share across multiple locations.

Corey: That thing falls over all the time anyway. May as well put it in the cloud where it can do so without taking too much else down with it. Was that the thinking or?

Jeremy: Well, and the other thing about it at the time, Corey, was that, like, so many customers worked in this, like, remote-first world, right? And so, SharePoint was inevitably hosted at somebody\\u2019s office. And so, the workers at that office were so privileged over the workers everywhere else. The performance gap between consuming SharePoint in one location versus another was like, night and day. So, you know, employees in headquarters were like, \\u201cYeah, SharePoint\\u2019s great.\\u201d Employees in branch offices were like, \\u201cThis thing is terrible,\\u201d you know? \\u201cIt\\u2019s so slow. I hate it, I hate it, I hate it.\\u201d

And so, Cloud actually became, like, this neutral location to move SharePoint to that kind of had an equal performance for every office. And so, that was, I think, one of the reasons and it was also, you know, it had capacity problems, and customers were right at that point, uploading tons of static documents to it, like Word documents, Office attachments, et cetera, and so they were starting to have some of these, like, real disk sprawl problems with SharePoint. So, that was kind of the month one problem. And only after they get through kind of month two, three, and four, and they go through, \\u201cI don\\u2019t understand my bill,\\u201d and, \\u201cHelp me understand security implications,\\u201d then they think about, like, \\u201cHey, should we go back and look at how we\\u2019re running that SharePoint stuff and maybe do it more efficiently and, like, move those static Office documents onto S3?\\u201d And so on, and so on.

And that\\u2019s kind of one of the big things that I\\u2019ve changed that I would say is very different from, like, 2011 to now, is there\\u2019s enough sophistication around understanding that, like, you don\\u2019t just translate what you\\u2019re doing in your office or in your data center to what you\\u2019re doing on cloud. Or if you do, you\\u2019re not getting the most out of your investment.

Corey: I\\u2019m curious to get your take on how you have seen cloud adoption patterns differ, specifically tied to geo. I mean, I tend to see it from a world where there\\u2019s a bifurcation of between born-in-the-cloud SaaS-type companies where one workload is 80% of their bill or whatnot, and of the big enterprises where the largest single component is 3%. So, it\\u2019s a very different slice there. But I\\u2019m curious what you would see from a sales perspective, looking across a lot of different geographic boundaries because we\\u2019re all, on some level, biased based upon where we tend to spend our time doing business. I\\u2019m in San Francisco, which is its very own strange universe that has a certain perspective about itself that is occasionally accurate, but not usually. But it\\u2019s a big world out there.

Jeremy: It is. One thing that I would say it\\u2019s interesting. I spent my AWS days based in Singapore, living in Singapore at the time, and I was working with customers across Southeast Asia. And to your point, Corey, one of the most interesting things was this little bit of a leapfrog effect. Data centers in Asia-Pac, especially in places like the Philippines, were just terrible.

You know, the Philippines had, like, the second highest electricity rates in Asia at the time, only behind Japan, even though the GDP per capita gap between those two countries is really large. And yet you\\u2019re paying, like, these super-high electricity rates. Secondarily, data centers in the Philippines were prone to flooding. And so, a lot of companies in the Philippines never went the data center route. You know, they just hosted servers in their offices, you know, they had a bunch of desktop machines in a cubicle, that kind of situation because, like, data centers themselves were cost prohibitive.

So, you saw this effect a little bit like cell phones in a lot of the developing world. Landline infrastructure was too expensive or never got done for whatever reason, and people went straight to cell phones. So actually, what I saw in a lot of emerging markets in Asia was, screw the data center; we\\u2019re going to go straight to cloud. So, I saw a lot of Asia-Pac get a little bit ahead of places like Europe where you had, for instance, a lot of long-term data center contracts and you had customers really locked in. And we saw this over the next, let\\u2019s say between, like, say, 2014 and 2018 when I was working with a systems integrator, and then started working on cloud security.

We saw that US customers and Asia-Pac customers didn\\u2019t have these obligations; European customers, a lot of them were still working off their lease, and still, you know, I\\u2019m locked into let\\u2019s just say Equinix Frankfurt for another five years before I can think about cloud migration. So, that\\u2019s definitely one aspect that I observed. Second thing I think is, like, the earlier you started, the earlier you reached the point where you realize that actually there is value in a lot of managed services and there actually is value in getting away from the kind of server mindset around EC2.

Corey: It feels like there\\u2019s a lot of, I want to call it legacy thinking, in some ways, except that\\u2019s unfair because legacy remains a condescending engineering term for something that makes money. The problem that you have is that you get bound by choices you didn\\u2019t necessarily realize you were making, and then something becomes revenue-bearing. And now there\\u2019s a different way to do it, or you learn more about the platform, or the platform itself evolves, and, \\u201cOh, I\\u2019m going to rewrite everything to take advantage of this,\\u201d isn\\u2019t happening. So, it winds up feeling like, yeah, we\\u2019re treating the cloud like a data center. And sometimes that\\u2019s right; sometimes that\\u2019s a problem, but ultimately, it still becomes a significant challenge. I mean, there\\u2019s no way around it. And I don\\u2019t know what the right answer is, I don\\u2019t know what the fix is going to be, but it always feels like I\\u2019m doing something wrong somewhere.

Jeremy: I think a lot of customers go through that same set of feelings and they realize that they have the active runway problem, where you know, how do you do maintenance on an active runway? You kind of can\\u2019t because you\\u2019ve got flights going in and out. And I think you\\u2019re seeing this in your part of the world at SFO with a lot of the work that got done in, like, 2018, 2019 where they kind of had to close down a runway and had, like, near misses because they consolidated all flights onto the one active runway, right? It is a challenge. And I actually think that some of the evolution that I\\u2019ve seen our customers go through over the last, like, two, three years, is starting to get away from that challenge.

So, to your point, when you have revenue-bearing workloads that you can\\u2019t really modify and things are pretty tightly coupled, it is very hard to make change. But when you start to have it where things are broken down into more microservices, it makes it a lot easier to cycle out Service A for Service B, or let\\u2019s say more accurately, Service A1 with Service A2 where you can kind of just, like, plug and play different APIs, and maybe, you know, repoint services at the new stuff as they come online. But getting to that point is definitely a painful process. It does require architectural changes and often those architectural changes aren\\u2019t at the infrastructure level; they\\u2019re actually inside the application or they\\u2019re between things like applications and third-party dependencies where the customers may not have full control over the dependencies, and that does become a real challenge for people to break down and start to attack. You\\u2019ve heard of the Strangler Methodology?

Corey: Oh, yes. Both in terms of the Boston Strangler, as well\\u2014

Jeremy: [laugh]. Right.

Corey: As the Strangler design pattern.

Jeremy: Yeah, yeah. But I think, like, getting to that is challenging until, like, once you understand that you want to do that, it makes a lot of sense. But getting to the starting point for that journey can be really challenging for a lot of customers because it involves stakeholders that are often not involved on infrastructure conversations, and organizational dysfunction can really creep in there, where you have teams that don\\u2019t necessarily play nice together, not for any particular reason, but just because historically they haven\\u2019t had to. So, that\\u2019s something that I\\u2019ve seen and definitely takes a little bit of cultural work to overcome.

Corey: When you take a look across the board of cloud adoption, it\\u2019s interesting to have seen the patterns that wound up unfolding. Your career path, though, seem to have gotten away from the selling cloud and into some strange directions leading up to what you\\u2019re doing now, where you founded Firetail. What do you folks do?

Jeremy: We do API security. And it really is kind of the culmination of, like, the last several years and what we saw. I mean, to your point, we saw customers going through kind of Phase One, Two, Three of cloud adoption. Phase One, the, you know, for lack of a better phrase, lift-and-shift and Phase Two, the kind of first step on the path towards quote-unquote, \\u201cEnlightenment,\\u201d where they start to see that, like, actually, we can get better operational efficiency if we, you know, move our databases off of EC2 and on to RDS and we move our static content onto S3.

And then Phase Three, where they realize actually EC2 kind of sucks, and it\\u2019s a lot of management overhead, it\\u2019s a lot of attack surface, I hate having to bake AMIs. What I really want to do is just drop some code on a platform and run my application. And that might be serverless. That might be containerized, et cetera. But one path or the other, where we pretty much always see customers ending up is with an API sitting on a network.

And that API is doing two things. It front-ends a data set and at front-ends a set of functionality, and most cases. And so, what that really means is that the thing that sits on the network that does represent the attack surface, both in terms of accessing data or in terms of let\\u2019s say, like, abusing an application is an API. And that\\u2019s what led us to where I am today, what led me and my co-founder Riley to, you know, start the company and try to make it easier for customers to build more secure APIs. So yeah, that\\u2019s kind of the change that I\\u2019ve observed over the last few years that really, as you said, lead to what I\\u2019m doing now.

Corey: There is a lot of, I guess, challenge in the entire space when we bound that to\\u2014even API security, though as soon as you going down the security path it starts seeming like there\\u2019s a massive problem, just in terms of proliferation of companies that each do different things, that each focus on different parts of the story. It feels like everything winds up spitting out huge amounts of security-focused, or at least security-adjacent telemetry. Everything has findings on top of that, and at least in the AWS universe, \\u201cOh, we have a service that spits out a lot of that stuff. We\\u2019re going to launch another service on top of it that, of course, cost more money that then winds up organizing it for you. And then another service on top of that that does the same thing yet again.\\u201d And it feels like we\\u2019re building a tower of these things that are just\\u2026 shouldn\\u2019t just be a feature in the original underlying thing that turns down the noise? \\u201cWell, yes, but then we couldn\\u2019t sell you three more things around it.\\u201d

Jeremy: Yeah, I mean\\u2014

Corey: Agree? Disagree?

Jeremy: I don\\u2019t entirely disagree. I think there is a lot of validity on what you just said there. I mean, if you look at like the proliferation of even the security services, and you see GuardDuty and Config and Security Hub, or things like log analysis with Athena or log analysis with an ELK stack, or OpenSearch, et cetera, I mean, you see all these proliferation of services around that. I do think the thing to bear in mind is that for most customers, like, security is not a one size fits all. Security is fundamentally kind of a risk management exercise, right? If it wasn\\u2019t a risk management exercise, then all security would really be about is, like, keeping your data off of networks and making sure that, like, none of your data could ever leave.

But that\\u2019s not how companies work. They do interact with the outside world and so then you kind of always have this decision and this trade-off to make about how much data you expose. And so, when you have that decision, then it leads you down a path of determining what data is important to your organization and what would be most critical if it were breached. And so, the point of all of that is honestly that, like, security is not the same for you as it is for me, right? And so, to that end, you might be all about Security Hub, and Config instead of basic checks across all your accounts and all your active regions, and I might be much more about, let\\u2019s say I\\u2019m quote-unquote, \\u201cDigital-native, cloud-native,\\u201d blah, blah, blah, I really care about detection and response on top of events.

And so, I only care about log aggregation and, let\\u2019s say, GuardDuty or Athena analysis on top of that because I feel like I\\u2019ve got all of my security configurations in Infrastructure as Code. So, there\\u2019s not a right and wrong answer and I do think that\\u2019s part of why there are a gazillion security services out there.

Corey: On some level, I\\u2019ve been of the opinion for a while now that the cloud providers themselves should not necessarily be selling security services directly because, on some level, that becomes an inherent conflict of interest. Why make the underlying platform more secure or easier to use from a security standpoint when you can now turn that into a revenue source? I used to make comments that Microsoft Defender was a classic example of getting this right because they didn\\u2019t charge for it and a bunch of antivirus companies screamed and whined about it. And then of course, Microsoft\\u2019s like, \\u201cOh, Corey saying nice things about us. We can\\u2019t have that.\\u201d And they started charging for it. So okay, that more or less completely subverts my entire point. But it still feels squicky.

Jeremy: I mean, I kind of doubt that\\u2019s why they started charging for it. But\\u2014

Corey: Oh, I refuse to accept that I\\u2019m not that influential. There we are.

Jeremy: [laugh]. Fair enough.

Corey: Yeah, I just can\\u2019t get away from the idea that it feels squicky when the company providing the infrastructure now makes doing the secure thing on top of it into an investment decision.

Jeremy: Yeah.

Corey: \\u201cDo you want the crappy, insecure version of what we build or do you want the top-of-the-line secure version?\\u201d That shouldn\\u2019t be a choice people have to make. Because people don\\u2019t care about security until right after they really should have cared about security.

Jeremy: Yeah. Look, and I think the changes to S3 configuration, for instance, kind of bear out your point. Like, it shouldn\\u2019t be the case that you have to go through a lot of extra steps to not make your S3 data public, it should always be the case that, like, you have to go through a lot of steps if you want to expose your data. And then you have explicitly made a set of choices on your own to make some data public, right? So, I kind of agree with the underlying logic. I think the counterargument, if there is one to be made, is that it\\u2019s not up to them to define what is and is not right for your organization.

Because again, going back to my example, what is secure for you may not be secure for me because we might have very different modes of operation, we might have very different modes of building our infrastructure, deploying our infrastructure, et cetera. And I think every cloud provider would tell you, \\u201cHey, we\\u2019re just here to enable customers.\\u201d Now, do I think that they could be doing more? Do I think that they could have more secure defaults? You know, in general, yes, of course, they could. And really, like, the fundamentals of what I worry about are people building insecure applications, not so much people deploying infrastructure with bad configurations.

Corey: It\\u2019s funny, we talk about this now. Earlier today, I was lamenting some of the detritus from some of my earlier builds, where I\\u2019ve been running some of these things in my old legacy single account for a while now. And the build service is dramatically overscoped, just because trying to get the security permissions right, was an exercise in frustration at the time. It was, \\u201cNope, that\\u2019s not it. Nope, blocked again.\\u201d

So, I finally said to hell with it, overscope it massively, and then with a, \\u201cTodo: fix this later,\\u201d which of course, never happened. And if there\\u2019s ever a breach on something like that, I know that I\\u2019ll have AWS wagging its finger at me and talking about the shared responsibility model, but it\\u2019s really kind of a disaster plan of their own making because there\\u2019s not a great way to say easily and explicitly\\u2014or honestly, by default the way Google Cloud does\\u2014of okay, by default, everything in this project can talk to everything in this project, but the outside world can\\u2019t talk to any of it, which I think is where a lot of people start off. And the security purists love to say, \\u201cThat\\u2019s terrible. That won\\u2019t work at a bank.\\u201d You\\u2019re right, it won\\u2019t, but a bank has a dedicated security apparatus, internally. They can address those things, whereas your individual student learner does not. And that\\u2019s how you wind up with open S3 bucket monstrosities left and right.

Jeremy: I think a lot of security fundamentalists would say that what you just described about that Google project structure, defeats zero trust, and you know, that on its own is actually a bad thing. I might counterargue and say that, like, hey, you can have a GCP project as a zero trust, like, first principle, you know? That can be the building block of zero trust for your organization and then it\\u2019s up to you to explicitly create these trust relationships to other projects, and so on. But the thing that I think in what you said that really kind of does resonate with me in particular as an area that AWS\\u2014and really this case, just AWS\\u2014should have done better or should do better, is IAM permissions. Because every developer in the world that I know has had that exact experience that you described, which is, they get to a point where they\\u2019re like, \\u201cOkay, this thing isn\\u2019t working. It\\u2019s probably something with IAM.\\u201d

And then they try one thing, two things, and usually on the third or fourth try, they end up with a star permission, and maybe a comment in that IAM policy or maybe a Jira ticket that, you know, gets filed into backlog of, \\u201cReview those permissions at some point in the future,\\u201d which pretty much never happens. So, IAM in particular, I think, is one where, like, Amazon should do better, or should at least make it, like, easy for us to kind of graphically build an IAM policy that is scoped to least permissions required, et cetera. That one, I\\u2019ll a hundred percent agree with your comments and your statement.

Corey: As you take a look across the largest, I guess, environments you see, and as well as some of the folks who are just getting started in this space, it feels like, on some level, it\\u2019s two different universes. Do you see points of commonality? Do you see that there is an opportunity to get the individual learner who\\u2019s just starting on their cloud journey to do things that make sense without breaking the bank that they then can basically have instilled in them as they start scaling up as they enter corporate environments where security budgets are different orders of magnitude? Because it seems to me that my options for everything that I\\u2019ve looked at start at tens of thousands of dollars a year, or are a bunch of crappy things I find on GitHub somewhere. And it feels like there should be something between those two.

Jeremy: In terms of training, or in terms of, like, tooling to build\\u2014

Corey: In terms of security software across the board, which I know\\u2014

Jeremy: Yeah.

Corey: \\u2014is sort of a vague term. Like, I first discovered this when trying to find something to make sense of CloudTrail logs. It was a bunch of sketchy things off GitHub or a bunch of very expensive products. Same thing with VPC flow logs, same thing with trying to parse other security alerting and aggregate things in a sensible way. Like, very often it\\u2019s, oh, there\\u2019s a few very damning log lines surrounded by a million lines of nonsense that no one\\u2019s going to look through. It\\u2019s the needle in a haystack problem.

Jeremy: Yeah, well, I\\u2019m really sorry if you spent much time trying to analyze VPC flow logs because that is just an exercise in futility. First of all, the level of information that\\u2019s in them is pretty useless, and the SLA on actually, like, log delivery, A, whether it\\u2019ll actually happen, and B, whether it will happen in a timely fashion is just pretty much non-existent. So\\u2014

Corey: Oh, from a security perspective I agree wholeheartedly, but remember, I\\u2019m coming from a billing perspective, where it\\u2019s\\u2014

Jeremy: Ah, fair enough.

Corey: \\u2014huh, we\\u2019re taking a petabyte in and moving 300 petabytes between availability zones. It\\u2019s great. It\\u2019s a fun game called find whatever is chatty because, on some level, it\\u2019s like, run two of whatever that is\\u2014or three\\u2014rather than having it replicate. What is the deal here? And just try to identify, especially in the godforsaken hellscape that is Kubernetes, what is that thing that\\u2019s talking? And sometimes flow logs are the only real tool you\\u2019ve got, other than oral freaking tradition.

Jeremy: But God forbid you forgot to tag your [ENI 00:24:53] so that the flow log can actually be attributed to, you know, what workload is responsible for it behind the scenes. And so yeah, I mean, I think that\\u2019s a\\u2014boy that\\u2019s a case study and, like, a miserable job that I don\\u2019t think anybody would really want to have in this day and age.

Corey: The timing of this is apt. I sent out my newsletter for the week a couple hours before this recording, and in the bottom section, I asked anyone who\\u2019s got an interesting solution for solving what\\u2019s talking to what with VPC flow logs, please let me know because I found this original thing that AWS put up as part of their workshops and a lab to figure this out, but other than that, it\\u2019s more or less guess-and-check. What is the hotness? It\\u2019s been a while since I explored the landscape. And now we see if the audience is helpful or disappoints me. It\\u2019s all on you folks.

Jeremy: Isn\\u2019t the hotness to segregate every microservice into an account and run it through a load balancer so that it\\u2019s like much more properly tagged and it\\u2019s also consumable on an account-by-account basis for better attribution?

Corey: And then everything you see winds up incurring a direct fee when passing through that load balancer, instead of the same thing within the same subnet being able to talk to one another for free.

Jeremy: Yeah, yeah.

Corey: So, at scale\\u2014so yes, for visibility, you\\u2019re absolutely right. From a, I would like to spend less money giving it directly to Amazon, not so much.

Jeremy: [unintelligible 00:26:08] spend more money for the joy of attribution of workload?

Corey: Not to mention as well that coming into an environment that exists and is scaled out\\u2014which is sort of a prerequisite for me going in on a consulting project\\u2014and saying, \\u201cOh, you should rebuild everything using serverless and microservice principles,\\u201d is a great way to get thrown out of the engagement in the first 20 minutes. Because yes, in theory, anyone can design something great, that works, that solves a problem on a whiteboard, but most of us don\\u2019t get to throw the old thing away and build fresh. And when we do great, I\\u2019m greenfielding something; there\\u2019s always constraints and challenges down the road that you don\\u2019t see coming. So, you finally wind up building the most extensible thing in the universe that can handle all these things, and your business dies before you get to MVP because that takes time, energy and effort. There are many more companies that have died due to failure to find product-market fit than have died because, \\u201cOh hey, your software architecture was terrible.\\u201d If you hit the market correctly, there is budget to fix these things down the road, whereas your code could be pristine and your company\\u2019s still dead.

Jeremy: Yeah. I don\\u2019t really have a solution for you on that one, Corey [laugh].

Corey: [laugh].

Jeremy: I will come back to your one question\\u2014

Corey: I was hoping you did.

Jeremy: Yeah, sorry. I will come back to the question about, you know, how should people kind of get started in thinking about assessing security. And you know, to your point, look, I mean, I think Config is a low-ish cost, but should it cost anything? Probably not, at least for, like, basic CIS foundation benchmark checks. I mean, like, if the best practice that Amazon tells everybody is, \\u201cTurn on these 40-ish checks at last count,\\u201d you know, maybe those 40-ish checks should just be free and included and on in everybody\\u2019s account for any account that you tag as production, right?

Like, I will wholeheartedly agree with that sentiment, and it would be a trivial thing for Amazon to do, with one kind of caveat\\u2014and this is something that I think a lot of people don\\u2019t necessarily understand\\u2014collecting all the required data for security is actually really expensive. Security is an extremely data-intensive thing at this day and age. And I have a former coworker who used to hate the expression that security is data science, but there is some truth in it at this point, other than the kind of the magic around it is not actually that big because there\\u2019s not a lot of, let\\u2019s say, heuristic analysis or magic that goes into what queries, et cetera. A lot of security is very rule-based. It\\u2019s a lot of, you know, just binary checks: is this bit set to zero or one?

And some of those things are like relatively simple, but what ends up inevitably happening is that customers want more out of it. They don\\u2019t just want to know, is my security good or bad? They want to know things like is it good or bad now relative to last week? Has it gotten better or worse over time? And so, then you start accumulating lots of data and time series data, and that becomes really expensive.

And secondarily, the thing that\\u2019s really starting to happen more and more in the security world is correlation of multiple layers of data, infrastructure with applications, infrastructure with operating system, infrastructure with OS and app vulnerabilities, infrastructure plus vulnerabilities plus Kubernetes configurations plus API sitting at the edge of that. Because realistically, like, so many organizations that are built out at scale, the truth of the matter is, is just like on their operating system vulnerabilities, they\\u2019re going to have tens of thousands, if not millions of individual items to deal with and no human can realistically prioritize those without some context around it. And that is where the data, kind of, management becomes really expensive.

Corey: I hear you. Particularly the complaints about AWS Config, which many things like Control Tower setup for you. And on some level, it is a tax on using the cloud as the cloud should be used because it charges for evaluation of changes to your environment. So, if you\\u2019re spinning things up all the time and then turning them down when they\\u2019re not in use, that incurs a bunch of Config charges, whereas if you\\u2019ve treat it like a big dumb version of your data center where you just spin [unintelligible 00:30:13] things forever, your Config charge is nice and low. When you start seeing it entering the top ten of your spend on services, something is very wrong somewhere.

Jeremy: Yeah. I would actually say, like, a good compromise in my mind would be that we should be included with something like business support. If you pay for support with AWS, why not include Config, or some level of Config, for all the accounts that are in scope for your production support? That would seem like a very reasonable compromise.

Corey: For a lot of folks that have it enabled but they don\\u2019t see any direct value from it either, so it\\u2019s one of those things where not knowing how to turn it off becomes a tax on what you\\u2019re doing, in some cases. In SCPs, but often with Control Tower don\\u2019t allow you to do that. So, it\\u2019s your training people who are learning this in their test environments to avoid it, but you want them to be using it at scale in an enterprise environment. So, I agree with you, there has to be a better way to deliver that value to customers. Because, yeah, this thing is now, you know, 3 or 4% of your cloud bill, it\\u2019s not adding that much value, folks.

Jeremy: Yeah, one thing I will say just on that point, and, like, it\\u2019s a super small semantic nitpick that I have, I hate when people talk about security as a tax because I think it tends to kind of engender the wrong types of relationships to security. Because if you think about taxes, two things about them, I mean, one is that they\\u2019re kind of prescribed for you, and so in some sense, this kind of Control Tower implementation is similar because, like you know, it\\u2019s hard for you to turn off, et cetera, but on the other hand, like, you don\\u2019t get to choose how that tax money is spent. And really, like, you get to set your security budget as an organization. Maybe this Control Tower Config scenario is a slight outlier on that side, but you know, there are ways to turn it off, et cetera.

The other thing, though, is that, like, people tend to relate to tax, like, this thing that they really, really hate. It comes once a year, you should really do everything you can to minimize it and to, like, not spend any time on it or on getting it right. And in fact, like, there\\u2019s a lot of people who kind of like to cheat on taxes, right? And so, like, you don\\u2019t really want people to have that kind of mindset of, like, pay as little as possible, spend as little time as possible, and yes, let\\u2019s cheat on it. Like, that\\u2019s not how I hope people are addressing security in their cloud environments.

Corey: I agree wholeheartedly, but if you have a service like Config, for example\\u2014that\\u2019s what we\\u2019re talking about\\u2014and it isn\\u2019t adding value to you, and you just you don\\u2019t know what it does, how it works, than it [unintelligible 00:32:37]\\u2014or more or less how to turn it off, then it does effectively become directly in line of a tax, regardless of how people want to view the principle of taxation. It\\u2019s a\\u2014yeah, security should not be a tax. I agree with you wholeheartedly. The problem is, is it is\\u2014

Jeremy: It should be an enabler.

Corey: \\u2014unclea\\u2014yeah, the relationship between Config and security in many cases is fairly attenuated in a lot of people\\u2019s minds.

Jeremy: Yeah. I mean, I think if you don\\u2019t have, kind of, ideas in mind for how you want to use it or consume it, or how you want to use it, let\\u2019s say as an assessment against your own environment, then it\\u2019s particularly vexing. So, if you don\\u2019t know, like, \\u201cHey, I\\u2019m going to use Config. I\\u2019m going to use Config for this set of rules. This is how I\\u2019m going to consume that data and how I\\u2019m going to then, like, pass the results on to people to make change in the organization,\\u201d then it\\u2019s particularly useless.

Corey: Yeah. I really want to thank you for taking the time to speak with me. If people want to learn more, where\\u2019s the best place for them to find you?

Jeremy: Easy, breezy. We are just firetail.io. That\\u2019s \\u2018fire\\u2019 like the, you know, flaming substance, and \\u2018tail\\u2019 like the tail of an animal, not like a story. But yeah, just firetail.io.

And if you come now, we\\u2019ve actually got, like, a white paper that we just put out around API security and kind of analyzing ten years of API-based data breaches and trying to understand what actually went wrong in most of those cases. And you\\u2019re more than welcome to grab that off of our website. And if you have any questions, just reach out to me. I\\u2019m just jeremy@firetail.io.

Corey: And we\\u2019ll put links to all of that in the [show notes 00:34:03]. Thank you so much for your time. I appreciate it.

Jeremy: My pleasure, Corey. Thanks so much for having me.

Corey: Jeremy Snyder, founder and CEO at Firetail. I\\u2019m Cloud Economist Corey Quinn, and this is Screaming in the Cloud. If you\\u2019ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you\\u2019ve hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment pointing out that listening to my nonsense is a tax on you going about your day.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.