Director of IT & Operations at Chef on What it Means to be Secure

Webinar Banner_resized

About Token Security

At Token Security our goal is to teach the core curriculum for modern DevSecOps. Each week we will deep dive with an expert so you walk away with practical advice to apply to your team today. No fluff, no buzzwords.

About This Episode

This week we are joined by Ben Rockwood, Director of IT & Operations at Chef who shares what it means to be secure, and how compliance and controls play a part in that.

About The Hosts

Max Saltonstall

Max Saltonstall loves to talk about security, collaboration and process improvement. He's on the Developer Advocacy team in Google Cloud, yelling at the internet full time. Since joining Google in 2011 Max has worked on video monetization products, internal change management, IT externalization and coding puzzles. He has a degree in Computer Science and Psychology from Yale.

Justin McCarthy

Justin McCarthy is the co-founder and CTO of strongDM, the database authentication platform. He has spent his entire career building highly scalable software. As CTO of Rafter, he processed transactions worth over $1B collectively. He also led the engineering teams at Preact, the predictive churn analytics platform, and Cafe Press..

Justin McCarthy 0:19
This is Justin McCarthy from strongDM. And today we’ve got with us, Ben Rockwood, the director of IT and operations at chef software, who is also I’m told, an all around great guy. Welcome, Ben.

Ben Rockwood 0:27
Thank you, Justin. Glad to be here.

Justin McCarthy 0:29
I know a lot of folks would think of Chef as being software that I download, and I run on my machine or machines. So like, Why on earth would Chef need a director of IT and operations because I just downloaded the right stuff from GitHub. So what are you doing all day?

Ben Rockwood 0:43
What am I doing all day? Well, I mean, we’re a business. And as a business, we have an IT department that needs to do all the it things. And we have customers who want to know that our development environment, our production environments are stable, and I run operations, because we have a hosted product that we are as a SaaS, we do have three products, three core products, right Chef is the configuration management, which obviously is very useful to the compliance regime.

But the other is that we do have a product called InSpec, which is probably the single greatest tool out there for automating compliance checks with your auditors. So that’s a very cool tool that we have this one, perhaps less well known, then the marquee product chef.

Why do you do any of the things that you do? Why do you put a firewall in place? Why do you update the operating system? Why do you go and watch that list for CBEs? And why do you apply them? Do you do it just because it’s a nice idea? And if you do that, because it’s a nice idea, is that all that you need to do? Or is that just what occurred to you to happen? Or is it because I manager said that you should do that? Why are you doing that? What is it that we’re really trying to do here? I mean, we say security?

Okay, great, I’ll do security. You said security. And now I know exactly what you mean. What the fuck does that actually mean? How do you define that? What do you actually need to do? Its compliance, that really sets the stage. And that primarily involves the controls.

So what the fuck is a control? First, let’s understand what the controllers and we can produce a list of like, what are the things that we should do? And then we can go and implement those things? And then that gets into? That’s why we have the firewalls there. Do we need Ingress egress? Or do we just need Ingress? How often do we need to patch? How big a concern is that? And then how does that play into our environment? And then that gets into the topic of risk? Right? Why do we do risk assessments?

Because we only know what we know, what about all the things that we don’t know are specific to our environment that we can’t possibly capture in a bunch of controls, we got to account for that some way what we do we do risk.

Justin McCarthy 2:38
Today, we are going to be discussing compliance primarily, but also all the adjacent topics. And actually, when I think of Chef, I think of often running a knife command on my prompt, I think of running some software locally, I don’t actually think necessarily have an organization that has to do with a lot of compliance. So can you tell me a bit about basically your exposure and why you’re an expert on the subject, and you’re doing some compliance is that the case?

Ben Rockwood 3:01
Doing all kinds of compliance, it’s important to the industry, it’s important to our customers, it’s important to us, I think we’ve moved into an era where more and more business depends on software.

And those businesses depending on their own risk profile have to get some amount of assurance that the software that they’re hearing is developed in a secure environment in a safe environment where they can trust those updates, as well as mitigate their risk.

Some of this is for, you know, regulation purposes. Some of this is just because it’s good practice. So yeah, we’re deeply involved in our company, our customers are as well in the Chef is a great product for anybody who’s trying to maintain a compliant and secure environment.

One of the best things you can do to secure say, a PCI environment is to shift to make sure that you have a baseline configuration in an environment that isn’t going to change your sprawl over time. And we also have a product called inspec, which is a fantastic product for auditing in a DSL language that looks like playing live language.

And the advantage there is is that an auditor non technical person can understand it. So if you’ve ever gone through an audit, and you have to sit there in front of an auditor and log into 100 different systems and take screenshots of the firewall is in fact enabled tools like InSpec and automate that process and produce a report that simplifies your life keeps you continuously compliance, because you could run it all the time. And something that your auditor trusts, and it’s technology that sort of automates and improves the entire process for everyone involved.

Justin McCarthy 4:28
I know one of the soap boxes I often find myself on is sort of educating mostly my team on the distinctions between a security and compliance. So can you talk a little bit about your observation about sort of the boundaries and where one is the customer, the other and how security teams can work together with compliance goals, so that folks don’t end up pitching stuff over walls? For example?

Ben Rockwood 4:49
Yeah, I think we want everything to be secure. We want our cars to be secure, we want our phones to be secure, we want our house to be secure. We want our retirement fund to be secure. We all like security, right? That’s a basic sort of human need. Right? You can even go out to Maslow’s hierarchy in terms of sort of things we all need. But what does that mean? It’s contextual, right?

And it can be difficult for people to agree on what secure actually means. Everyone has their own definition. And so if you step away from kind of the deep technologist, then you step back a little bit and say, what is it that I need to do to be secure, it starts to get really fuzzy, and it starts to grow out of control really quick.

And so where can we look to get a list of what are the things that we should do? Well, compliance is sort of the place we go, there are lots of compliance frameworks out there, from SOC 2 PCI DSS, you’ve got HIPPA, you’ve got ITSM frameworks, like combet, Nikhil, and all kinds of acronyms, but it gives you a place to go and look and start to piece together a set of what we call controls, which are really just principles that help us to begin to define what it is that we need to do.

And we can start to break the huge problem space into pieces that we can then reason about individually, and start to make the right choices in terms of well, how does that need to look in terms of policy? And then how do we go ahead and implement that as an actual technical control? And the thing is, is it really bridges management to technical implementation.

I think that’s why a lot of technologists get really scared off, right? They want to think about this subsystem and how I secure that or this individual data flow, and how do I deal with that? And when you say, No, no, let’s, let’s stop, let’s go back, let’s go upstream. Let’s think about what is the business need? What does the customer need? What is the value? What is it that we really need to secure? And what does that mean to all the different stakeholders, suddenly that problem space grows and grows and grows and people get intimidated and scared off. And I think it’s a shame, because some of the people are scared off the most easily are the people who are most necessary in that process.

Justin McCarthy 6:54
I actually haven’t thought about using the criteria that are defined in compliance regimes as a way of sort of partition the space. So I really like that model. Because I think all of us are vulnerable to the trap of what you actually just alluded to, which is sort of allowing ourselves to sort of stare into the abyss and and actually keep asking, like, but what if about everything. And then pretty soon, we get down to the root need to go ask Intel how they made this, Jeff. And so like that is not a productive path for anyone. And so partitioning the space you can figure out if we assume for a moment that Intel and AMD do their jobs. And if we assume for a moment that AWS has done their job, then what?

Ben Rockwood 7:30
Yeah, and you know, I’m a little different than I think a lot of my peers in the space in that I didn’t really get into security and compliance, all that because somebody said I had to most people, it’s forced on them. And they reject it, and it hurts. And so they push back.

I got into this, because, you know, I was in a small company, and I was growing the department and I got to a point where I felt like I built sort of the best operations team and the most secure infrastructure I could. And I asked around and asked everyone I could How does it look to you tell me what’s wrong with it, criticize it? And I got a lot of Oh, it’s great. It’s good. It’s good, you’re good.

I got nervous that maybe I hadn’t covered all my bases. And so I started looking around and saying, Well, how could I judge independently what I did, and I came into contact with things like I tell, which is old and busted. But I was thinking if it’s old and busted, then it should be easy, right?

So I can look at that. And I can look at some of these other standards that are out there. And then found that they were really useful tools for helping me as an ops manager decide whether or not we done the job well, and so I’m a little more lovey lovey on this stuff, because it’s a tool for me as a manager to judge my performance, and then my team’s performance and not be surprised.

Justin McCarthy 8:38
So of the compliance regimes out there, the one that I have the most recent experience with, and the one that a lot of folks seem to grapple with right now. And it may be increasingly is a common currency and sort of terms of explaining to your customers that you’re approaching the problem in a mature fashion is suck, too. So can you tell me a little bit about what you’ve done with the sock to process and how it’s maybe influenced the shape your team in the shape of your routines?

Ben Rockwood 9:02
Yeah, absolutely. So SOC 2 is probably the most important of the compliance standards out there. Right. It was actually created by tax people AI CPA, it was formerly known to most technologists SAS 70, SAS 70, was Sunday you think about is being associated with like a data center. And it was really used for the design of controls around sort of ambiguous things.

So the problem with SAS 70 was the standard didn’t actually say which had to do. And I think this is one thing I want a lot of people to understand about compliance. In general, you think about SOX 404 some people use hear about right, you need to be a Sox 404 compliant company. The thing is, is if you actually go and pull up, Sarbanes Oxley, you look at section 404, and you actually read it, what it says is you need to have controls to ensure security and reliability of your process.

Okay, well, how do I do that? I don’t know. Just do that. Right. And so if you look at SAS, 70, same thing, SAS 70 was, in one way, a way of kind of certifying against socks for for compliance. And again, somebody just said you should have a bunch of controls that determine what you do, and you should abide by them. Okay, that’s great.

Well, what is that? What is that thing I need to do? I don’t know it’s appropriate to your business. So the problem with SAS 70 was they had no teeth, right. And it was essentially a useful standard. So as a CPA, think it’s the American Institute of CPS came up with an auditing standard that they broke out, and they turned it into soft one sock sock. Sock one is sort of generic.

So it says you should design controls for your business that ensure the confidentiality, integrity and availability and security of your business. That’s normally what’s actually used to fulfill at least a good part of socks for for compliance. So when your financial team does an audit, they’ll go and talk to the IT team and make sure that only people we’re supposed to have access to the financial system habit.

Justin McCarthy 11:03
If you pause there for a moment, as you read those criteria in that regime, it’s pretty obvious that it’s written from the point of view of the question of how do I prevent embezzlement?

Ben Rockwood 11:14
Yeah, pretty much. And that’s what it’s really all about, right? I mean, a lot of those controls, it goes back to the old days of the cash register, right? Separation of duties like this is when we’ve seen technology all the time. And we get really confused about because what does that mean separation of duties.

It comes from the old idea that the person who puts money and takes money in and out of the cash register is not the person who counts at the end of the day so that they can’t steal, right? There’s a check, or a control to ensure that the accounting sort of works out, what do you bring that over to the technology side, it’s really hard because you don’t have that many cooks in the kitchen. And separating these things apart, can difficult. And that’s maybe the important thing for us, as technologists to remember, and this is why it’s actually good to look at these controls.

And understand that control is a principle it’s not a rule, it’s not a policy, it’s a principle that gets implemented as a policy. So the idea of separation of duties is, in the case of technology is the person who has access to a system to go and change, it can’t go and remove themselves from the logs, right, they shouldn’t be able to erase the evidence that they actually wouldn’t made a modification that separation of duties, right? Not that you necessarily have to have somebody say, approve a change independent of you. Right, that doesn’t really matter.

So you know, sock one is all about sort of you decide what you want to do you tell the auditor, this is what we want to do, and they certified that looks like the right list. And then they say, Okay, did you do all those things start to is really making up for the deficiency there. And that it has a prescriptive list of these are the things you should do. And it breaks it out into five different categories. We don’t need to get into here, but it’s a much more sort of prescribed of what those controls are. And then you figure out well, how do we as a business and implement those controls.

Justin McCarthy 12:51
I’ll also say that’s really when the set of criteria graduated, something that’s relevant for a SAS business or a technology business. It looks less like supermarket, and more like something that has a website. And so the right buttons to click and a team perhaps building it.

Ben Rockwood 13:07
Yeah, the beauty of SOC 2 is, is that now it’s pretty reasonable standard, especially if you understand the difference, like everyone who does talk to has to meet the what’s called the security principle. And then there is a privacy principle and an availability principle and availability principle that, but the one everyone has to do to claim sock to is security, it’s a fairly decent list.

And it’s one that I personally anytime I interface with a SaaS service out there, I want their SOC 2 and their SOC 2 report is actually have a great number of things in its first and foremost going to have an architecture diagram, a general description of the technical implementation of the service. And it’s going to have a whole bunch of their controls, right.

And what’s great is, is that I have a lot more confidence in the service that I’m working with, because I know that some fun energy has gotten into this. And so I generally think it’s something that whether you need those reports or not, it’s a good idea for everyone to demand these of the people that they’re working with. And it’s also a great challenge if you have a SaaS service to think about implementing it yourself.

Because I’ll tell you honestly, the thing that concerns me with a lot operations teams, if you go to the ops team and just say, Hey, can you draw me a diagram of your service, you’d be shocked how many operations teams can’t do that. Because we become so technically siloed. Within the team, it’s like, well, we can talk to Bob who’s doing all the server management. And we can talk to Jeanne who’s going ahead and taking care of like the monitoring systems. And then when nobody talks to the financial parts of it, that’s Fred. And we don’t even talk to Fred.

And it turns out, you don’t even know your own system, right. So if you’re that kind of brokenness is happening inside of an operations team, I as a consumer of that service, would probably like to know that they’re that disorganized and use sock to as an opportunity to bring the team together really embrace kind of the DevOps idea of bringing development and operations together, understand the service. And it can be a really useful catalyst for getting your head around the problem and making better for yourself, for your customers and for the world. Right.

Justin McCarthy 15:03
So I know a lot of us that are involved in both requesting, so Dr. Reports, and then producing them. There’s this interesting activity now of sort of describing the vendors that you depend on describing the products you depend on. And then there’s this sort of network and graph that’s built as a result of the way this trust is communicated between parties. I’m curious, just because everybody looks at it with different eyes. When you receive fresh sucks, you report, what page are you jumping to? So in other words, like what are the like top three controls that are the most bang for the buck that you’re going to drill down to immediately and just see whether it passes your sniff test,

Ben Rockwood 15:35
I go to the architecture diagram, first thing off, right, I go to the first section, I go past the at the station, and from the auditor and all that stuff. And I go straight to description of the service. Right, and I look to see how complete that is. A lot of the controls are useful. And the thing is, one thing you need to know is is that if you get a sock to report and every single one of them is green, or nothing found, then it’s probably not super reliable, right? We always got to fail, like one or two or three, there’s some improvement that can be made.

But I want to see how well does a business describe their own service. And normally, because we’re technologists, right, when we look at that report, we know, okay, they’re running a bunch of you see two instances that are behind a bunch of BS, and you know, it’s in East one. And Okay, so I’ve got a pretty good idea about what all that means. And then of course, I’m going to immediately use my technologies brand and be like, I didn’t mention anything about using auto scale group, maybe there’s some different ways in which this could break or their failure modes that I should be aware of in the future.

But a lot of times, let’s be honest, you know, a lot of services and I’m consuming, I’m not going to run the service, right, I just want to have a little confidence that the team running, it has their act together, and I can trust them. And that’s what those reports can give you in a way that you just couldn’t get in any other reasonable way.

Justin McCarthy 16:48
I think of all the problems we have within technical systems of working with dependency management, and I think in every language, every dependency management system, and every OS, every patch management system. And I wonder when we’re going to get started the next evolution and compliance where we can kind of a little bit formalized some of these dependencies at the org level. And somehow I should be able to include in my report, check sums of all the reports that I depend on, and there’s nothing for that maybe I’ll throw it in next year, I’ll just throw it into the service description. Here’s the checksum.

Alright, so one part of at least the SOC 2 compliance process, and this is present in every compliance regime, but it’s articulated reasonably well in suck too. And that’s the sort of sub process of thinking about analyzing and sort of documenting risk. I’d love to hear about how you have approached risk and how you’ve approached that analysis, what cadence and how you’ve sort of distill that down into a final product, whether it’s a report or document or whatever.

Ben Rockwood 17:45
Yeah, I mean, risk is one of the funnest parts of the whole thing. It’s also the most scary, and I think ambiguous, it’s a hard thing to wrap your head around, because it’s so open ended. So what is risk risk is stuff that can go wrong. And any compliance standard that’s available out there understands that it can’t possibly anticipate everything that could actually happen in your environment, because it doesn’t know your environment. And so what it’s going to do is it’s going to say that one of those controls will be that you perform a regular risk assessment.

And that’s going to be tailored to your environment. So how do you actually go and do that risk assessment, I love bringing the whole team together. And we get on a big meeting, I’ll schedule it for like three hours, four hours, but the point is, is the medium long enough that nobody’s looking at their watches. And I opened up a spreadsheet normally, like Google sheets or something like that. And I kind of fill out a couple of columns.

And then we all start to think, what are the most horrible things that can happen? One of the things that scare you, when you go to bed at night? What is the stuff that goes through your head, and hope that doesn’t break out? That doesn’t happen? What if this happens, because all of us already have that. And that’s where you really have to get the practitioners involved, right?

They built it, they know it, they know where its warts and flaws are. And you just go at it, everyone just starts adding things into the spreadsheet, and it’s fun, the stuff that starts to show up. And then of course, once you’ve got to all of those things, people see them and it triggers other ideas on what about that? What about that whether, and then you just start building the spreadsheet, and you’ll quickly go from a handful to a couple hundred within a pretty short period of time.

And once you know, just kind of like a user story mapping or something like that, when the idea sort of slow down, and people are now coming up with weird things like zombie apocalypse and whatnot, they nice, okay, let’s, let’s reel it back. And then you do you know, little sorting and categorizing. And normally, you know, in these meetings, take a couple of breaks, right? Kind of first 30 minutes, everyone puts everything in, then everyone stopped, go away, go have coffee, don’t think about this and come back fresh mind. Try again, go away, come back.

And then when it’s also down, right, we categorize, walk away, come back, and then we start to walk through, you know, what are the actual consequences of the sort of things? And how do we intend to mitigate those risks. And as the process sort of goes along, it’s sort of distills down into a couple of really actionable items, were a couple items, at least that you need, did go back with the team and really analyze how much effort we get to put into this, we know that we could all build perfectly secure environments, right?

Just like you can write completely bug free code, right? The problem is, is that the amount of effort required to achieve it is so arduous that by the time you made it bug free, no one would want the thing that you made, right. And so going through the analysis is for what are the trade offs? And what are we willing to deal with? And what do we mitigate? It’s a fun process. And to do that once every six months is not too much to ask anything.

Justin McCarthy 20:30
who’s in the room for that? And I asked, because actually, the way some of the criteria are written, it’s very easy to see one of the possible risks as I don’t do a good enough job building my product and nobody buys it.

Ben Rockwood 20:43
Yeah, yeah, you can. So this is one of the places where I think risk goes off the rails. Another thing is, is that if you’re a real kind of devil’s advocate type of person, you also say, well, a risk is any outcome that you do not perceive. Therefore, it can be both positive and negative. And I’ll admit, I used to be one of those people are like, well, what are the positive risks? Right? It is true to say that if you run a website, one of the potential risks, or one of the most classical types of risks is you get mentioned on Oprah. And all of a sudden access goes through the roof. Is that positive? Yes. Is it negative? Yeah, it’s also negative, right? So everybody on over again,

Justin McCarthy 21:19
they’re all getting a chef license this year?

Ben Rockwood 21:21
Yeah. I mean, there’s all sorts of things right. And that’s why you want your mind to be sort of as free as possible. And to think about, you know, what can you really deal with my product sucks, and no one will buy it, that would be different part of a different risk process. And we should probably make a distinction here. risk assessments are things that can be done at a lot of different levels. And your CFO and CEO should be doing a business wide risk assessment. And in that risk assessment, a risk to the business to investors is our product is sucks and no one buys it. Right, we talk about risk in terms of compliance. And when we get down, we’re talking about technical implementation, and actual products that we build all that we’re not concerned about, you know, nobody likes my product, we’re going to get more tangible sorts of risks.

Justin McCarthy 22:05
The header of our risk assessment explicitly says that just to try and constrain it. It’s a curious situation that I can imagine there are many organizations out there were a technology. And our product risk assessment, like the one you’ve just described, is probably happening maybe even more frequently, or more rigorously than the other one you described.

Ben Rockwood 22:23
Yeah, it should be happening at all levels in the organization. And again, some people think it’s come down to maturity or whatnot. But to be frank, one of the things I do like about compliance is when you really think about it like this, you probably once in a while, actually sit down with your spouse or your partner, friends, or whatever. And you probably do kind of a little mini risk assessment over a couple of beers about your life, right? Like, what happens if I get sick? What happens if I get hit by a bus, we’ll do this kind of fairly naturally. Think about worst case scenarios, a risk assessment is just a little slightly more structured way of dealing with it and trying to move it towards actual implementation, right?

Justin McCarthy 22:58
Whenever I’m in that meeting, I need to use the bus metaphor, though I always lean on the lottery metaphor,

Ben Rockwood 23:03
you win the lottery,

Justin McCarthy 23:04
well, yeah, if Jane are critical engineer on this thing, wins the lottery tomorrow and decides she’s that we Jane doesn’t have to get hit by bus, she proves out of existence because she buys an island.

Ben Rockwood 23:14
That’s a lot more, I don’t know, pushing out existence is pretty terrible to

Justin McCarthy 23:18
Oh, come on, don’t be cool to just, you know, do ghost everybody. Because you know, the

Ben Rockwood 23:22
unfortunate thing with a lottery metaphors all of us, as technologists are so geeky that we all imagine that we’re going to be sitting there on a really nice laptop in the Caribbean. And we’re just gonna keep working while we’re risking.

Justin McCarthy 23:32
Really what we want to do we tend, I suppose, in particular, if you have an open source product, if there’s something you can just keep working on, anyway, like, exactly.

Very true. Alright, so I would love to hear a little bit about sort of day in the life of achieving some of these actually implementing some of these controls over the course of the year. So I know for us, we rely heavily on manifesting some of these things into tickets that happen. They’re triggered by time they’re triggered by events, and then we sort of they’re very good, concrete steps that we do. And then we proved that we did by logging stuff in the tickets. So any thoughts on sort of how to make some of this stuff real throughout the year?

Ben Rockwood 24:08
Yeah, absolutely. This is an important point that really needs to be emphasized that anybody who’s new to compliance are struggling with compliance, trying to wrap their head around it. Stop thinking that security, and particularly compliance has anything to do with you actually making anything secure. So compliance and audit is not about actually securing things. It’s about proving that you secured things, it’s about proving that you know, the state of your infrastructure, right? And so just say, Oh, yeah, it’s secure. Oh, we got firewalls everywhere.

Oh, yeah, you know, we actually audited our users to six months ago, doesn’t mean anything, if you can’t prove it to somebody else. And so there’s a couple of practical ways you get around that one is start using tickets and places you might not have us tickets before, when you go through and check all the systems to ensure that they’re all running the latest version of the operating system need to create the ticket, and somebody needs to put in that ticket.

Yes, I have checked. And I found that it’s okay. And you need to put that in a comment of a JIRA ticket. Before you close it. You know, some teams have a tendency to like put a description in the ticket that says, you know, go do this thing. And then they go do it. And they just close the ticket is done. Right? Because it implies that I actually, in fact, did that. Well, your auditor is going to look at that and be like, maybe you did nothing. And you just closed it. Yeah,

Justin McCarthy 25:23
I have evidence that you click the Close button.

Ben Rockwood 25:25
Yeah, here’s one out of PCI that I think is hilarious, right. So you need to on a daily basis, I think it is review your firewall box. How do I prove to an auditor that I did in fact, review the logs? Right? How do I do that? So you talked to on it. And one of the most common ways to do this is you email the log to yourself. And then you have a Read Receipt and your email that shows that you opened it looked at it. Now, any reasonable technology is going to be like, okay, so it shows up in your mailbox, you click on it, and you don’t read it, you don’t actually do that.

Okay, maybe you don’t. But the auditor has a control that says that you review it. And they’ve got to verify in some way that you did that. This is also important to mention that there’s two types of sock reports, right, there’s a sock type one and type two. So type one is that you do this at a point in time, a sock to said that you do this over some period of time, typically six months, and people want to say here, but it’s almost always six months, which is also why you go and talk to a company and you say I need your soccer for they’ll say we can get you one right now start to type one now.

And we expect to have our type to done in like six months, or whatever it is right? Which is fairly typical. Somebody who’s just recently implemented it type choose the one you really want, because it shows you’ve done it for some period of time. So this is when all those tickets that you create are super important, right? You need to be able to prove to the auditor that over the course of six months that you did review the logs on a daily basis, if that’s a controlled, you have to implement that when you on a monthly basis where going to check that all of your operating systems of acceptable latest patches that you did in fact do that. And so this is where you can get super interesting with your automation, right.

So certainly in JIRA has an API that you can actually integrate with. So you can actually completely automate this where you know, a ticket is generated every 30 days. And as soon as that ticket is created, it goes off and runs several shell script or runs a tool like inspect, to go and run a profile to ensure that all done and then sort of a pencil that had data into the ticket and closes it. So you have a point in time reference that we needed to do a thing and it’s done. You don’t need to have humans do all this, you can put your mind to the task of automating it. And once you do at first it’s going to feel a little arduous, right?

I’m doing all this stuff for the sake of doing it. And it’s all ridiculous. But I will say you know, you can get some pretty cool reports, like when you actually get a report in your inbox, that shows everybody who escalated to root across all of your infrastructure. That’s actually a really interesting report. Like it only takes you 100 milliseconds to scan that an email. But every so often subtle pop up, what was that? Right? And that’s good data to have.

And so that’s why I generally like to be really positive on a lot of these controls. To some extent, it’s a lot at first, and it seems kind of ridiculous. And you can get really cynical about it. But on the other hand, why wouldn’t you want to know who’s logging into your systems? Why wouldn’t you want to have absolute confidence that the people have access to your systems are the right people and that you don’t have old craft the accounts laying around I mean, most of us are sitting under a mountain of what we like to call technical debt.

Some cases, it’s not technical debt, it’s incompetence, because nobody’s holding us accountable to do this stuff. And we’ve got a million other things to do. So I know I should clean up the accounts. But Bob’s a good guy, he’s probably not going to like use his account, he doesn’t have access to VPN anyway. So it doesn’t matter. I mean, you can’t actually get him to login. I don’t want to go clean up balls, SSH keys, but you really should do that. And it’s good to have a good reason to do that. Once you had the SOC 2 toolkit that actually helped you get your controls worked out in the policy generated, so you could take it from there, when that’d be great.

Justin McCarthy 29:02
That would be great.

Yes. Okay. So Ben, thank you for mentioning that. Yes, certainly, as we went through the process for the first time, we like every technologist out there, tried to gamify it and turn it into, you know, how do I reduce this very human problem into something that I can at least reason about? And seems more like a technical and software product that we’re familiar with working on every day? Um, yeah. So that da to give birth to comply, the generalized compliance toolkit that happens to have a specialty and suck to controls? Yeah. So I’m glad you mentioned it. I don’t know, you tell me what do you see in that project that piqued your interest?

Ben Rockwood 29:36
The great thing about that project is it’s a place to start, right? I think this is the hardest thing about compliance. Okay, we got to do compliance when we’re talking talk to you, right? So like, I gotta do sock too. So what do I do? And the funny thing is, actually, you’ll find this is dirty secret, like he talked to your auditors. A lot of times, you’re like, Okay, what do I do? I paid a lot of money to Deloitte and Touche. Now you tell me what to do. And they’re like, really audit man, like,

Justin McCarthy 30:01
we can’t tell you what to do?

Ben Rockwood 30:03
Well, number one, they can’t tell you what to do. A lot of times when they say they can’t tell you what to do, it means they don’t know. And I won’t get into the fun political reasons for why that is, but they’re just as confused as you and where do you get started? So what do you gotta do, you gotta start with a list of controls, at least and talk to, you’ve already got that, then you’ve got to turn that into a bunch of policies, because the controls are just the principle that you’re going to do it.

So you need an actual thing that says, We, as a company agree that we’re going to do this that’s called the policy, whether you like policy or not, you got to have that thing. And then Okay, well, now I’ve got a blank sheet of paper and Google Docs that says, security policy or user access policy, and you’re like, oh, boy, how do I start this, just getting started is hard.

And a lot a lot of people do is they either go and buy or they get from their auditors, these templates that are some generic policy that auditors never read, that you’re never going to read that you’re just going to fill out, nobody will ever read the stupid thing, but everyone will be fine with it, because it looks super official that everyone will sign off on and those are the worst kinds of policies whatsoever templates, fine to start with, but you need to make it your own.

So the comply framework that strong dm is created is great that it was a bunch of you worked on it, right? That it gives you a nice place to start that looks like something that human would use, and it gets you down the road to the point where you start to feel like I can do this right, I can go ahead and make some modifications to it. I can make it my own and build it over time.

And it gives you that seed from which to start, just like any other boilerplate, we know that you start out with like a big Angular project, right? What do you gotta do? You gotta write a whole bunch of boiler plate, Why should I do that I can go get a bunch of boilerplate out of a GitHub project, that’s best practice that gets me started. And then I’ll slowly morph that into something that’s mine over time, that’s the best way to go about it.

Justin McCarthy 31:51
I of course, love that I can then commit those refinements back to get back. It’s all source control this awesome. I mean, doing everything in Word doc says,

Yeah, Google Docs. Of course, this is ever so slight upgrade, but nothing like branching and merging to make a really crisp boundary between a proposal and a ratified, set of changes. Cool.

And also, just since since we are talking about comply, I do want to put out a call. So actually, we were talking a few minutes ago about collecting evidence. So when you’re going through the audit, the process of collecting and proving that you did that ticket, that’s a big thing. And actually in comply for anyone interested in checking it out, you’ll see that there’s a whole clockwork around opening and closing tickets, that’s part of the product. But then additionally, something that’s missing that anyone out there could contribute, is sumac, some help screenshot.

So I know when I go through this process, I literally encounter just organization and workflow problems related to screenshot. Because many of these systems, you can run chef inspect to get certain parameters. But then there are other things, there’s just no way around it, like you have to screenshot the thing.

Ben Rockwood 32:54
That is real, right? I’m not joking, like a lot of sitting there with an auditors literally sitting about two people in a conference room, while the otter says show me this, and you pull it up on your screen, and he says, Take a screenshot and send it to me. And I mean, I remember the first time I did this, I was like, You’re joking, right? Like, this is all a big joke. And they’re like, nope, this is how it’s done. And I believe it’s all I talked to a couple of people. I mean, it just seems so ridiculous.

Justin McCarthy 33:21
And then there’s an additional sort of perverse reality about it, where by going through this process, then you have increased your standard. And I have increased my standard of what is an acceptable way to, for example, transmit some sensitive information and store some sensitive information that doesn’t apply to this evidence for the security of this of the screenshots themselves. I think there’s an opportunity there for by framework or some other open source product to just help just have something that you can sort of right, securely to and then grant access to the author.

Ben Rockwood 33:53
Yeah, this is a great opportunity to automate things that you’ve wanted automate, never really had the time or energy to put into it. Right. I mean, you’re complaining project is that one of the great things about using chef in my previous companies before I worked here is is that, you know, I could demonstrate to the auditor, that chef was running on every single one of my systems and automatically ran every 30 minutes.

And that was sufficient that my honor to actually took my cookbooks as evidence that fulfilled a large number of the controls, right, skipped all that screenshot, and went straight to you know, it’s all good to go. There’s a lot of opportunity for automation to come in and improvements. And it’s going to be sort of specific to your environment. And I know one place that it’s the hardest right now is actually the network environments, right is logging into your Juniper and Arista near, you know, all these switches and getting all that config and pulling it out.

Because it also needs to be done in a way that the auditor understands it, which is a real great challenge for us, as technologists, to be able to demonstrate this to a non technical individual that we’re filling these requirements, I think you can be cynical about it, and you can be rude about it, or you can sort of take it as a challenge of how well do I know my systems? How well do I know my own operation?

And can I explain it to this poor person from delayed who’s suffering through all my technical dribble, if I can make them understand it, then we really know what we’re doing. We’re really doing things right. And it just gives you a bunch of confidence that I think is really, really productive

Justin McCarthy 35:14
and couldn’t agree more. Okay, so Ben, thank you very much for your thoughts today on compliance and thanks for sharing your experience helping chef comply and best of luck in all your endeavors.

Ben Rockwood 35:25
Thank you so much.