Podcasts Technology Screaming in the Cloud Exciting Times in Cloud Security with Chris Farris

Exciting Times in Cloud Security with Chris Farris

Published: March 21, 2023, 10 a.m.

b'

Chris Farris, Cloud Security Nerd at Turbot, joins Corey on Screaming in the Cloud to discuss the latest events in cloud security, which leads to an interesting analysis from Chris on how legal departments obscure valuable information that could lead to fewer security failures in the name of protecting company liability, and what the future of accountability for security failures looks like. Chris and Corey also discuss the newest dangers in cloud security and billing practices, and Chris describes his upcoming cloud security conference, fwd:cloudsec.


About Chris


Chris Farris has been in the IT field since 1994 primarily focused on Linux, networking, and security. For the last 8 years, he has focused on public-cloud and public-cloud security. He has built and evolved multiple cloud security programs for major media companies, focusing on enabling the broader security team\\u2019s objectives of secure design, incident response and vulnerability management. He has developed cloud security standards and baselines to provide risk-based guidance to development and operations teams. As a practitioner, he\\u2019s architected and implemented multiple serverless and traditional cloud applications focused on deployment, security, operations, and financial modeling.


Chris now does cloud security research for Turbot and evangelizes for the open source tool Steampipe. He is one of the organizers of the fwd:cloudsec conference (https://fwdcloudsec.org) and has given multiple presentations at AWS conferences and BSides events.


When not building things with AWS\\u2019s building blocks, he enjoys building Legos with his kid and figuring out what interesting part of the globe to travel to next. He opines on security and technology on Mastodon, Twitter and his website https://www.chrisfarris.com

Links Referenced:




Transcript


Announcer: Hello, and welcome to Screaming in the Cloud with your host, Chief Cloud Economist at The Duckbill Group, Corey Quinn. This weekly show features conversations with people doing interesting work in the world of cloud, thoughtful commentary on the state of the technical world, and ridiculous titles for which Corey refuses to apologize. This is Screaming in the Cloud.


Corey: Welcome to Screaming in the Cloud. I\\u2019m Corey Quinn and we are here today to learn exciting things, steal exciting secrets, and make big trouble for Moose and Squirrel. Maybe that\\u2019s the podcast; maybe that\\u2019s the KGB, we\\u2019re not entirely sure. But I am joined once again by Chris Farris, cloud security nerd at Turbot, which I will insist on pronouncing as \\u2018Turbo.\\u2019 Chris, thanks for coming back.


Chris: Thanks for having me.


Corey: So, it\\u2019s been a little while and it\\u2019s been an uneventful time in cloud security with nothing particularly noteworthy happening, not a whole lot of things to point out, and honestly, we\\u2019re just sort of scraping the bottom of the barrel for news\\u2026 is what I wish I could say, but it isn\\u2019t true. Instead, it\\u2019s, \\u201cOh, let\\u2019s see what disastrous tire fire we have encountered this week.\\u201d What\\u2019s top of mind for you as we record this?


Chris: I think the most interesting one I thought was, you know, going back and seeing the guilty plea from Nickolas Sharp, who formerly was an employee at Ubiquiti and apparently had, like, complete access to everything there and then ran amok with it.


Corey: Mm-hm.


Chris: The details that were buried at the time in the indictment, but came out in the press releases were he was leveraging root keys, he was leveraging lifecycle policies to suppress the CloudTrail logs. And then of course, you know, just doing dumb things like exfiltrating all of this data from his home IP address, or exfiltrating it from his home through a VPN, which have accidentally dropped and then exposed his home IP address. Oops.


Corey: There\\u2019s so much to dive into there because I am not in any way shape or form, saying that what he did was good, or I endorse any of those things. And yeah, I think he belongs in prison for what he did; let\\u2019s be very clear on this. But I personally did not have a business relationship with him. I am, however, Ubiquiti\\u2019s customer. And after\\u2014whether it was an insider threat or whether it was someone external breaching them, Krebs On Security wound up doing a whole write-up on this and was single-sourcing some stuff from the person who it turned out, did this.


And they made a lot of hay about this. They sued him at one point via some terrible law firm that\\u2019s entire brand is suing media companies. And yeah, just wonderful, wonderful optics there and brilliant plan. But I don\\u2019t care about the sourcing. I don\\u2019t care about the exact accuracy of the reporting because what I\\u2019m seeing here is that what is not disputed is this person, who whether they were an employee or not was beside the point, deleted all of the audit logs and then as a customer of Ubiquiti, I received an email saying, \\u201cWe have no indication or evidence that any customer data was misappropriated.\\u201d Yeah, you just turn off your logs and yeah, you could say that always and forever and save money on logging costs. [unintelligible 00:03:28] best practice just dropped, I guess. Clowns.


Chris: So, yeah. And there\\u2019s definitely, like, compliance and standards and everything else that say you turn on your logs and you protect your logs, and service control policies should have been able to detect that. If they had a security operations center, you know, the fact that somebody was using root keys should have been setting off red flags and causing escalations to occur. And that wasn\\u2019t happening.


Corey: My business partner and I have access to our AWS org, and when I was setting this stuff up for what we do here, at a very small company, neither of us can log in with root credentials without alarms going off that alert the other. Not that I don\\u2019t trust the man; let\\u2019s be very clear here. We both own the company.


Chris: In business together. Yes.


Corey: Ri\\u2014exactly. It is, in many ways, like a marriage in that one of us can absolutely ruin the other without a whole lot of effort. But there\\u2019s still the idea of separation of duties, visibility into what\\u2019s going on, and we don\\u2019t use root API keys. Let me further point out that we are not pushing anything that requires you to send data to us. We\\u2019re not providing a service that is software powered to people, much less one that is built around security. So, how is it that I have a better security posture than Ubiquiti?


Chris: You understand AWS and in-depth cloud better. You know, it really comes down to how do you, as an AWS customer, understand all of the moving parts, all of the security tooling, all of the different ways that something can happen. And Amazon will say, \\u201cWell, it\\u2019s in the documentation,\\u201d but you know, they have, what, 357 services? Are you reading the security pages of all of those? So, user education, I agree, you should have, and I have on all of my accounts, if anything pops up, if any IAM change happens, I\\u2019m getting text messages. Which is great if my account got compromised, but is really annoying when I\\u2019m actually making a change and my phone is blowing up.

Corey: Yeah. It\\u2019s worth pointing out as well that yes, Ubiquiti is publicly traded\\u2014that is understood and accepted\\u2014however, 93% of it is owned by their CEO-founder god-king. So, it is effectively one person\\u2019s personal fiefdom. And I tend to take a very dim view as a direct result. When you\\u2019re in cloud and you have suffered a breach, you have severely screwed something up somewhere. These breaches are never, \\u201cSomeone stole a whole bunch of drives out of an AWS data center.\\u201d You have misconfigured something somewhere. And lashing out at people who reported on it is just a bad look.


Chris: Definitely. Only error\\u2014now, of course, part of the problem here is that our legal system encourages people to not come forward and say, \\u201cI screwed up. Here\\u2019s how I screwed up. Everybody come learn from my mistakes.\\u201d The legal professions are also there to manage risk for the company and they\\u2019re like, \\u201cDon\\u2019t say anything. Don\\u2019t say anything. Don\\u2019t even tell the government. Don\\u2019t say anything.\\u201d


Whereas we all need to learn from these errors. Which is why I think every time I do see a breach or I do see an indictment, I start diving into it to learn more. I did a blog post on some of the things that happened with Drizly and GitHub, and you know, I think the most interesting thing that came out of Drizly case was the ex-CEO of Drizly, who was CEO at the time of the breach, now has following him, for the rest of his life, an FTC order that says he must implement a security program wherever he goes and works. You know, I don\\u2019t know what happens when he becomes a Starbucks barista or whatever, but that is on him. That is not on the company; that is on him.


And I do think that, you know, we will start seeing more and more chief executive officers, chief security or information security officers becoming accountable to\\u2014or for the breaches and being personally accountable or professionally accountable for it. I think we kind of need it, even though, you know, there\\u2019s only so much a CISO can do.

Corey: One of the things that I did when I started consulting independently on AWS bills back in 2016 was, while I was looking at customer environments, I also would do a quick check for a few security baseline things. And I stopped doing it because I kept encountering a bunch of things that needed attention and it completely derailed the entire stated purpose of the engagement. And, frankly, I don\\u2019t want to be running a security consultancy. There\\u2019s a reason I focus on AWS bills. And people think I\\u2019m kidding, but I swear to you I\\u2019m not, when I say that the reason is in part because no one has a middle-of-the-night billing emergency. It is strictly a business-hours problem. Whereas with security, wake up.


In fact, the one time I have been woken up in the middle of the night by a customer phone call, they were freaking out because it was a security incident and their bill had just pegged through the stratosphere. It\\u2019s, \\u201cCool. Fix the security problem first, then we\\u2019ll worry about the bill during business hours. Bye.\\u201d And then I stopped leaving my phone off of Do Not Disturb at night.


Chris: Your AWS bill is one of your indicators of compromise. Keep an eye on it.


Corey: Oh, absolutely. We\\u2019ve had multiple engagements discover security issues on that. \\u201cSo, what are these instances in Australia doing?\\u201d \\u201cWe don\\u2019t have anything there.\\u201d \\u201cI believe you\\u2019re being sincere when you say this.\\u201d

Chris: Yes.

Corey: However.


Chris: \\u201cLast month, you\\u2019re at $1,000 and this month, you\\u2019re at $50,000. And oh, by the way, it\\u2019s the ninth, so you might want to go look at that.\\u201d


Corey: Here\\u2019s the problem that you start seeing in large-scale companies though. You or I wind up posting our IAM credentials on GitHub somewhere in public\\u2014and I do this from time to time, intentionally with absolutely no permissions attached to a thing\\u2014and I started look at the timeline of, \\u201cOkay 3, 2, 1, go,\\u201d with the push and now I start counting. What happens? At what time does the quarantine policy apply? When do I get an email alert? When do people start trying to exploit it? From where are they trying to exploit it?


It\\u2019s a really interesting thing to look into, just from the position of how this stuff all fits together and works. And that\\u2019s great, but there\\u2019s a whole \\u2018nother piece to it where if you or I were to do such a thing and actually give it admin credentials, okay, my, I don\\u2019t know, what, $50, $100 a month account that I use for a lot of my test stuff now starts getting charged enormous piles of money that winds up looking like a mortgage in San Francisco, I\\u2019m going to notice that. But if you have a company that spending, I don\\u2019t know, between ten and $20 million a month, do you have any idea how much Bitcoin you\\u2019ve got to be mining in that account to even make a slight dent in the overall trajectory of those accounts?


Chris: In the overall bill, a lot. And in a particularly mismanaged account, my experience is you will notice it if you\\u2019re monitoring billing anomalies on a per-account basis. I think it\\u2019s important to note, you talked about that quarantine policy. If you look at what actually Amazon drops a deny on, it\\u2019s effectively start EC2 instances and change IAM policies. It doesn\\u2019t prevent anybody from listing all your buckets and exfiltrating all your data. It doesn\\u2019t prevent anybody from firing up Lambdas and other less commonly used resources. Don\\u2019t assume oh, Amazon dropped the quarantine policy. I\\u2019m safe.


Corey: I was talking to somebody who spends $4 a month on S3 and they wound up suddenly getting $60 grand a day and Lambda charges, because max out the Lambda concurrency in every region and set it to mine crypto for 15 minutes apiece, yeah, you\\u2019ll spend $60,000 a day to get, what $500 in crypto. But it\\u2019s super economical as long as it\\u2019s in someone else\\u2019s account. And then Amazon hits them with a straight face on these things, where, \\u201cPlease pay the bill.\\u201d Which is horrifying when there\\u2019s several orders of magnitude difference between your normal bill and what happens post-breach. But what I did my whole post on \\u201c17 Ways to Run Containers on AWS,\\u201d followed by \\u201c17 More Ways to Run Containers on AWS,\\u201d and [unintelligible 00:12:00] about three services away from having a third one ready to go on that, the point is not, \\u201cToo many ways to run containers,\\u201d because yes, that is true and it\\u2019s also amusing to me\\u2014less so to the containers team at AWS which does not have a sense of humor or sense of self-awareness of which they have been alerted\\u2014and fine, but every time you\\u2019re running a container, it is a way to turn it into a crypto mining operation, in some way shape or form, which means there are almost 40-some-odd services now that can reasonably be used to spin up cryptocurrency mining. And that is the best-case breach scenario in a bunch of ways. It costs a bunch of money and things to clean up, but \\u2018we lost customer data.\\u2019 That can destroy companies.


Chris: Here\\u2019s the worst part. Crypto mining is no longer profitable even when I\\u2019ve got stolen API keys because bitcoin\\u2019s in the toilet. So, now they are going after different things. Actually, the most recent one is they look to see if your account is out of the SCS sandbox and if so, they go back to the tried-and-true way of doing internet scams, which is email spam.


Corey: For me, having worked in operations for a very long time, I\\u2019ve been in situations where I worked at Expensify and had access to customer data there. I have worked in other finance companies\\u2014I worked at Blackrock. Where I work now, I have access to customer billing data. And let me be serious here for a second, I take all of these things seriously, but I also in all of those roles slept pretty well at night. The one that kept me up was a brief stint I did as the Director of Tech Ops at Grindr over ten years ago because unlike the stuff where I\\u2019m spending the rest of my career and my time now, it\\u2019s not just money anymore.

Whereas today, if I get popped, someone can get access to what a bunch of companies are paying AWS. It\\u2019s scandalous, and I will be sued into oblivion and my company will not exist anymore and I will have a cloud hanging over my head forever. So, I have to be serious about it\\u2014


Chris: But nobody will die.

Corey: Nobody dies. Whereas, \\u201cOh, this person is on Grindr and they\\u2019re not out publicly,\\u201d or they live in a jurisdiction where that is punishable by imprisonment or death, you have blood on your hands, on some level, and I have never wanted that kind of responsibility.


Chris: Yeah. It\\u2019s reasonably scary. I\\u2019ve always been happy to say that, you know, the worst thing that I had to do was keep the Russians off CNN and my friends from downloading Rick and Morty.

Corey: Exactly. It\\u2019s, \\u201cOh, heavens, you\\u2019re winding up costing some giant conglomerate somewhere theoretical money on streaming subscriptions.\\u201d It\\u2019s not material to the state of the world. And part of it, too, is\\u2014what\\u2019s always informed my approach to things is, I\\u2019m not a data hoarder in the way that it seems our entire industry is. For the Last Week in AWS newsletter, the data that I collect and track is pretty freaking small.


It\\u2019s, \\u201cYou want to sign up for the lastweekinaws.com newsletter. Great, I need your email address.\\u201d I don\\u2019t need your name, I don\\u2019t need the company you work at. You want to give me a tagged email address? Fine. You want to give me some special address that goes through some anonymizing thing? Terrific. I need to know where I\\u2019m sending the newsletter. And then I run a query on that for metrics sometimes, which is this really sophisticated database query called a count. How many subscribers do I have at any given point because that matters to our sponsors. But can we get\\u2014you give us any demographic? No, I cannot. I can\\u2019t. I have people who [unintelligible 00:15:43] follow up surveys sometimes and that\\u2019s it.

Chris: And you\\u2019re able to make money doing that. You don\\u2019t have to collect, okay, you know, Chris\\u2019s zip code is this and Bob\\u2019s zip code is that and Frank\\u2019s zip code is the other thing.

Corey: Exactly.

Chris: Or job titles, or you know, our mother\\u2019s maiden name or anything else like that.

Corey: I talk about what\\u2019s going on in the world of AWS, so it sort of seems to me that if you\\u2019re reading this stuff every week, either because of the humor or in spite of the humor, you probably are in a position where services and goods tied to that ecosystem would be well-received by you or one of the other 32,000 people who happen to be reading the newsletter or listening to the podcast or et cetera, et cetera, et cetera. It\\u2019s an old-timey business model. It\\u2019s okay, I want to wind up selling, I don\\u2019t know, expensive wristwatches. Well, maybe I\\u2019ll advertise in a magazine that caters to people who have an interest in wristwatches, or caters to a demographic that traditionally buys those wristwatches. And okay, we\\u2019ll run an ad campaign and see if it works.

Chris: It\\u2019s been traditional advertising, not the micro-targeting stuff. And you know, television was the same way back in the broadcast era, you know? You watched a particular show, people of that demographic who watched that particular show had certain advertisers they wanted.


Corey: That part of the challenge I\\u2019ve seen too, from sponsors of this show, for example, is they know it works, but they\\u2019re trying to figure out how to do any form of attribution on this. And my answer\\u2014which sounds self-serving, but it\\u2019s true\\u2014is, there\\u2019s no effective way to do it because every time you try, like, \\u201cEnter this coupon code,\\u201d yeah, I assure you, some of these things wind up costing millions of dollars to deploy at large companies at scale and they provide value for doing it. No one\\u2019s going to punch in a coupon code to get 10% off or something like that. Procurement is going to negotiate custom contracts and it\\u2019s going to be brought up maybe by someone who heard the podcast ad. Maybe it just sits in the back of their mind until they hear something and it just winds of contributing to a growing awareness of these things.

You\\u2019re never going to do attribution that works on things like that. People try sometimes to, \\u201cOh, you\\u2019ll get $25 in credit,\\u201d or, \\u201cWe\\u2019ll give you a free t-shirt if you fill out the form.\\u201d Yeah, but now you\\u2019re biasing for people who find that a material motivator. When I\\u2019m debating what security suite I\\u2019m going to roll out at my enterprise I don\\u2019t want a free t-shirt for that. In fact, if I get a free t-shirt and I wear that shirt from the vendor around the office while I\\u2019m trying to champion bringing that thing in, I look a little compromised.

Chris: Yeah. Yeah, I am\\u2014[laugh] I got no response to that [laugh].

Corey: No, no. I hear you. One thing I do want to talk about is the last time we spoke, you mentioned you were involved in getting fwd:cloudsec\\u2014a conference\\u2014off the ground. Like all good cloud security conferences, it\\u2019s named after an email subject line.

It is co-located with re:Inforce this year in Anaheim, California. Somewhat ominously enough, I used to live a block-and-a-half away from the venue. But I don\\u2019t anymore and in fact, because nobody checks the global event list when they schedule these things, I will be on the other side of the world officiating a wedding the same day. So, yet again, I will not be at re:Inforce.

Chris: That is a shame because I think you would have made an excellent person to contribute to our call for papers and attend. So yes, fwd:cloudsec is deliberately actually named after a subject line because all of the other Amazon conferences seem to be that way. And we didn\\u2019t want to be going backwards and thinking, you know, past tense. We were looking forward to our conference. Yeah, so we\\u2019re effectively a vendor-neutral cloud security conference. We liked the idea of being able to take the talks that Amazon PR would never allow on stage at re:Inforce and run with it.

Corey: I would question that. I do want to call that out because I gave a talk at re:Invent one year about a vulnerability I found and reported, with the help of two other people, Scott Piper and Brandon Sherman, to the AWS security team. And we were able to talk about that on stage with Zack Glick, who at the time, was one of basically God\\u2019s own prototypes, working over in the AWS environment next to Dan [Erson 00:19:56]. Now, Dan remains the salt of the earth, and if he ever leaves basically just short the entire US economy. It\\u2019s easier. He is amazing. I digress. The point being is that they were very open about talking about an awful lot of stuff that I would never have expected that they would be okay with.

Chris: And last year at re:Inforce, they had an excellent, excellent chalk talk\\u2014but it was a chalk talk, not recorded\\u2014on how ransomware attacks operate. And they actually, like, revealed some internal, very anonymized patterns of how attacks are working. So, they\\u2019re starting to realize what we\\u2019ve been saying in the cloud security community for a while, which is, we need more legitimate threat intelligence. On the other hand, they don\\u2019t want to call it threat intelligence because the word threat is threatening, and therefore, you know, we\\u2019re going to just call it, you know, patterns or whatever. And our conference is, again, also multi-cloud, a concept that until recently, AWS, you know, didn\\u2019t really want to acknowledge that there were other clouds and that people would use both of them [crosstalk 00:21:01]\\u2014

Corey: Multi-cloud security is a nightmare. It\\u2019s just awful.

Chris: Yeah, I don\\u2019t like multi-cloud, but I\\u2019ve come to realize that it is a thing. That you will either start at a company that says, \\u201cWe\\u2019re AWS and we\\u2019re uni-cloud,\\u201d and then next thing, you know, either some rogue developer out there has gone and spun up an Azure subscription or your acquire somebody who\\u2019s in GCP, or heaven forbid, you have to go into some, you know, tinhorn dictator\\u2019s jurisdiction and they require you to be on-prem or leverage Oracle Cloud or something. And suddenly, congratulations, you\\u2019re now multi-cloud. So yes, our goal is really to be the things that aren\\u2019t necessarily onstage or aren\\u2019t all just, \\u201cIt\\u2019s great.\\u201d Even your talk was how great the incident response and vulnerability remediation process was.

Corey: How great my experience with it was at the time, to be clear. Because I also have gotten to a point where I am very aware that, in many cases when dealing with AWS, my reputation precedes me. So, when I wind up tweeting about a problem or opening a support case, I do not accept as a given that my experience is what everyone is going to experience. But a lot of the things they did made a lot of sense and I was frankly, impressed that they were willing to just talk about anything that they did internally. Because previously that had not been a thing that they did in open forums like that.

Chris: But you go back to the Glue incident where somebody found a bug and they literally went and went to every single CloudTrail event going back to the dawn of the service to validate that, okay, the, only two times we ever saw this happen were between the two researcher\\u2019s accounts who disclosed it. And so, kudos to them for that level of forward communication to their customers because yeah, I think we still haven\\u2019t heard anything out of Azure for last year\\u2019s\\u2014or a year-and-a-half ago\\u2019s Wiz findings.

Corey: Well, they did do a broad blog post about this that they put out, which I thought, \\u201cOkay, that was great. More of this please.\\u201d Because until they start talking about security issues and culture and the remediation thereof, I don\\u2019t give a shit what they have to say about almost anything else because it all comes back to security. The only things I use Azure for, which admittedly has some great stuff; their computer vision API? Brilliant\\u2014but the things I use them for are things that I start from a premise of security is not important to that service.

The thing I use it for on the soon-to-be-pivoted to Mastodon Twitter thread client that I built, it writes alt-text for images that are about to be put out publicly. Yeah, there\\u2019s no security issue from that perspective. I am very hard-pressed to imagine a scenario in which that were not true.

Chris: I can come up with a couple, but you know\\u2014

Corey: It feels really contrived. And honestly, that\\u2019s the thing that concerns me, too: the fact that I finally read, somewhat recently, an AWS white paper talking about\\u2014was it a white paper or was it blog post? I forget the exact media that it took. But it was about how they are seeing ransomware attacks on S3, which was huge because before that, I assumed it was something that was being made up by vendors to sell me something.

Chris: So, that was the chalk talk.

Corey: Yes.

Chris: They finally got the chalk talk from re:Inforce, they gave it again at re:Invent because it was so well received and now they have it as a blog post out there, so that, you know, it\\u2019s not just for people who show up in the room, they can hear it; it\\u2019s actually now documented out there. And so, kudos to the Amazon security team for really getting that sort of threat intelligence out there to the community.

Corey: Now, it\\u2019s in writing, and that\\u2019s something that I can cite as opposed to, \\u201cWell, I was at re:Invent and I heard\\u2014\\u201d Yeah, we saw the drink tab. We know what you might have thought you heard or saw at re:Invent. Give us something we can take to the board.

Chris: There were a lot of us on that bar tab, so it\\u2019s not all you.

Corey: Exactly. And it was my pleasure to do it, to be clear. But getting back to fwd:cloudsec, I\\u2019m going to do you a favor. Whether it\\u2019s an actual favor or the word favor belongs in quotes, the way that I submit CFPs, or conference talks, is optimized because I don\\u2019t want to build a talk that is never going to get picked up. Why bother to go through all the work until I have to give it somewhere?

So, I start with a catchy title and then three to five sentences. And if people accept it, great, then I get to build the talk. This is a forcing function in some ways because if you get a little delayed, they will not move the conference for you. I\\u2019ve checked. But the title of a talk that I think someone should submit for fwd:cloudsec is, \\u201cI Am Smarter Than You, so Cloud Security is Easy.\\u201d

And the format and the conceit of the talk is present it with sort of a stand-it-up-to-take-it-down level of approach where you are over-confident in the fact that you are smarter than everyone else and best practices don\\u2019t apply to you and so much of this stuff is just security theater designed as a revenue extraction mechanism as opposed to something you should actually be doing. And talk about why none of these things matter because you use good security and you know, it\\u2019s good because you came up with it and there\\u2019s no way that you could come up with something that you couldn\\u2019t break because you\\u2019re smart. It says so right in the title and you\\u2019re on stage and you have a microphone. They don\\u2019t. Turn that into something. I feel like there\\u2019s a great way to turn that in a bunch of different directions. I\\u2019d love to see someone give that talk.

Chris: I think Nickolas Sharp thought that too.

Corey: [laugh]. Exactly. In fact, that will be a great way to bring it back around at the end. And it\\u2019s like, \\u201cAnd that\\u2019s why I\\u2019m better at security than you are. If you have any questions beyond this, you can reach me at whatever correctional institute I go in on Thursday.\\u201d Exactly. There\\u2019s ways to make it fun and engaging. Because from my perspective, talks have to be entertaining or people don\\u2019t pay attention.

Chris: They\\u2019re either entertaining, or they\\u2019re so new and advanced. We\\u2019re definitely an advanced cloud security practice thing. They were 500 levels. Not to brag or anything, but you know, you want the two to 300-level stuff, you can go CCJ up the street. We\\u2019re hitting and going above and beyond what a lot of the [unintelligible 00:27:18]\\u2014

Corey: I am not as advanced on that path as you are; I want to be very clear on this. You speak, I listen. You\\u2019re one of those people when it comes to security. Because again, no one\\u2019s life is hanging in the balance with respect to what I do. I am confident in our security posture here, but nothing\\u2019s perfect. Everything is exploitable, on some level.

It\\u2019s also not my core area of focus. It is yours. And if you are not better than I am at this, then I have done something sort of strange, or so of you, in the same way that it is a near certainty\\u2014but not absolute\\u2014that I am better at optimizing AWS bills than you are. Specialists exist for a reason and to discount that expertise is the peak of hubris. Put that in your talk.

Chris: Yeah. So, one talk I really want to see, and I\\u2019ve been threatening to give it for a while, is okay, if there\\u2019s seventeen ways\\u2014or sorry, seventeen times two, soon to be seventeen times three ways to run containers in AWS, there\\u2019s that many ways to exfiltrate credentials from those containers. What are all of those things? Do we have a holistic way of understanding, this is how credentials can be exfiltrated so that we then as defenders can go figure out, okay, how do we build detections and mitigations for this?

Corey: Yeah. I\\u2019m a huge fan of Canarytokens myself, for that exact purpose. There are many devices I have where the only credentials in plain text on disk are things that as soon as they get used, I wind up with a bunch of things screaming at me that there\\u2019s been a problem and telling me where it is. I\\u2019m not saying that my posture is impenetrable. Far from it. But you\\u2019re going to have to work for it a little bit harder than running some random off-the-shelf security scanner against my AWS account and finding, oops, I forgot to turn on a bucket protection.

Chris: And the other area that I think is getting really interesting is, all of the things that have credentials into your Cloud account, whether it\\u2019s something like CircleCI or GitHub. I was having a conversation with somebody just this morning and we were talking about Roles Anywhere, and I was like, \\u201cRoles Anywhere is great if you\\u2019ve got a good strong PKI solution and can keep that private certificate or that certificate you need safe.\\u201d If you just put it on a disk, like, you would have put your AKIA and secret on a desk, congratulations, you haven\\u2019t really improved security. You\\u2019ve just gotten rid of the IAM users that are being flagged in your CSPM tool, and congratulations, you have, in fact, achieved security theater.

Corey: It\\u2019s obnoxious, on some level. And part of the problem is cost and security are aligned and that people care about them right after they really should have cared about them. The difference is you can beg, cry, whine, et cetera to AWS for concessions, you can raise another round of funding; there have solutions with money. But security? That ship has already sailed.

Chris: Yeah. Once the data is out, the data is out. Now, I will say on the bill, you get reminded of it every month, about three or four days after. It\\u2019s like, \\u201cOh. Crap, yeah, I should have turned off that EC2 instance. I just burned $100.\\u201d Or, \\u201cOh hey, we didn\\u2019t turn off that application. I just burned $100,000.\\u201d That doesn\\u2019t happen on security. Security events tend to be few and far between; they\\u2019re just much bigger when they happen.

Corey: I really want to thank you for taking the time to chat with me. I\\u2019m sure I\\u2019ll have you back on between now and re:Inforce slash fwd:cloudsec or anything else we come up with that resembles an email subject line. If people want to learn more and follow along with your adventures\\u2014as they should\\u2014where\\u2019s the best place for him to find you these days?

Chris: So, I am now pretty much living on Mastodon on the InfoSec Exchange. And my website, chrisfarris.com is where you can find the link to that because it\\u2019s not just at, you know, whatever. You have to give the whole big long URL in Mastodon. It\\u2019s no longer\\u2014

Corey: Yeah. It\\u2019s like a full-on email address with weird domains.

Chris: Exactly, yeah. So, find me at http colon slash slash infosec dot exchange slash at jcfarris. Or just hit Chris Farris and follow the links. For fwd:cloudsec, we are conveniently located at fwdcloudsec.org, which is F-W-D cloud sec dot org. No colons because I don\\u2019t think those are valid in whois.

Corey: Excellent choice. And of course, links to that go in the [show notes 00:31:32], so click the button. It\\u2019s easier. Thanks again for your time. I really appreciate it.

Chris: Thank you.


Corey: Chris Farris, Cloud Security Nerd at Turbot slash Turbo. I\\u2019m Cloud Economist Corey Quinn and this is Screaming in the Cloud. If you\\u2019ve enjoyed this podcast, please leave a five-star review on your podcast platform of choice, whereas if you\\u2019ve hated this podcast, please leave a five-star review on your podcast platform of choice, along with an angry comment that resembles a lawsuit being filed, and then have it processed-served to me because presumably, you work at Ubiquiti.

Corey: If your AWS bill keeps rising and your blood pressure is doing the same, then you need The Duckbill Group. We help companies fix their AWS bill by making it smaller and less horrifying. The Duckbill Group works for you, not AWS. We tailor recommendations to your business and we get to the point. Visit duckbillgroup.com to get started.

'