Automate Once, Run Anywhere: The Docker Moment for Security Workflows - Nancy Chauhan & Aseem Shrey

This talk addresses the challenges associated with security automation in software engineering and introduces an open-source project called Ship Shape Studio designed to streamline this process. The speaker discusses the evolution of security engineering compared to software engineering, highlighting the reliance on scripts that lead to issues such as lack of observability, ownership, and fragility in existing automation processes. The session showcases how traditional methods fall short, especially in larger organizations, and proposes a modular solution that replaces scripts with containerized workflows. By using a drag-and-drop interface, Ship Shape Studio allows users to build and manage security automation workflows efficiently, incorporating various existing tools like Trivy and Semgrep, and provides features for analytics, logging, and version control, thus fostering collaboration and standardization within teams.

Good morning everyone. This is a scene and today we are here to talk about open source project that we have been working and a couple of our friends have been working on this to bring the best of software engineering to security engineering. We quickly see how it ended up in the demo. Show of hands how many of you are software developers here? Quite a lot. That's that's

couple of hands. Okay, so I think because we have so many people here. So it's going to sound very familiar to you. It's an AI generated image. So just a disclaimer. So if there is a very small startup, we have the engineers let's assume less than 10 engineers. There are a lot of manual deployed and we have like a lot of scripts. So we work in this

way. We have processes but not so much processes. And then we move to mid-scale. We start having some pipelines. We have checks. We have like a lot of things. We have like a lot of processes in the system. And then when we move to the enterprise, we have everything like observability, alerting. We have a lot of engineers. We have proper pipelines. We have testing staging environments. And

we have like systems. But when we look at security engineering, it's going to look very different and I'm going to ask this question. If we have security engineers in the room. And if you're working in the startup or you might have seen other people in startup, does this flow look semi-familiar to you? Like having a lot of scripts. Scripts deployed on different servers. Scripts glued together. Does

it sound familiar to you? Like anyone? Okay, I see one hand. I see couple of hands. Awesome. Now the second question is for people who might have seen at mid-scale. Does this look familiar? It's pretty much similar like all the scripts. We have different tools, Trivy. Uh we are exploring different tools, but again the same problem like we have scripts glued together. Now I'm going to ask

the question to all those people who are working at like very large organizations. Do you think this is also familiar? Um again the same thing, we have like a lot of scripts, um everything glued together. The scripts are different man Uh does it sound familiar to anyone who is working? Okay, I see couple of hands. Wow. So yeah, we have the same image. Um so for the

security engineering um a lot of things actually evolved in software engineering, but at security engineering it didn't evolve. And that is the problem which we are going to discuss today. Um so this is how security automation looks today. We have different tools, um Wiz, Slack, Trivy, um Semgrep. Any other tools, many other tools. Uh and they are just glued together. Um and yeah, this is how it

looks today. So who are we? Uh I am Nancy. I love to build and ship. Uh I work at Arise AI which is all about AI evaluations and LLM observability of AI agents. Uh I previously worked at open source orgs, uh LocalStack which recently is uh got closed. Uh but anyways, I worked there for a couple of long time. Uh I worked at Blinkit, Exotel. Um I

have been I love open source. I've been contributing to open source. I was part of K8s release team. Uh I also led TAG Environment Sustainability at one point which is now part of Operational Resilience. If any one of you if it sounds familiar to you, this is one of the group in CNCF TAG where you can contribute. Um I'm a CNCF ambassador and I also founded Women

in Cloud Native Community. I'm not as much into CNCF environment as Nancy is. But yeah, I've been uh currently I'm a infrastructure security engineer at Scale AI. I've been hacking and building stuff all my life and I built security from scratch at a bunch of startups that have become decacorn now. So, I've lived this whole journey. I also teach people cybersecurity online on my YouTube channel. Uh

that's about me. Awesome. So, I think we're going to discuss the problem with security automation. Um and it would be really nice if somebody can, you know, like somebody has anything else other than these pointers. I would love We would love to hear. Uh we want this talk to be more and more conversational. Um so, as you might have seen earlier when I talked about scripts, we

have seen the problems with security automation that it lacks observability. There is no ownership. Uh there is no observability. We do not know about any alerting. Uh most of the scripts which are there running in the servers may hi- may have the silent failures and we don't even get to know. And these scripts are fragile and unreliable and because we do not have any traceability, um there

are different compliance and evidence gaps. Uh do you do you feel this problem as well? Like uh the traceability problems. Um anyone? >> Okay, I we see couple of hands and uh built like scripts not systems. Um I want this to be conversational. If anyone has any other points which they might have seen with security automations. Other than this. Anyone? Okay. Okay, we're going to keep it

in the end. Um I'm pretty sure there will be more and more points. So, every company automates security. But, few operate those automations like software which we have seen in couple of slides. I've seen this happening again and again that the cost of unreliable security automation is paid in multiple ways. One of the very common thing that you see is company keep on buying a lot of

vendor tools like for every different kind of security stuff you buy one different tool. For example, if for code scanning you have a different tool, for PR scanning maybe a different tool, for cloud security you buy a another tool, then again there goes for CI/CD security have another tool. Essentially, there's a tool sprawl, you have multiple contracts, multiple things that you have to renew every year. Contracts

to negotiate, vendor security onboarding, all those things happen because of this. The second thing that happens is there's a lot of engineering effort that goes wasted just manning these tools because let's be honest, a lot of these tools are not very friendly. With the advent of LLMs, it's easier to just talk to cloud code, but it was not the case till very recently. Even now, a lot

of these enterprise tools are not very user-friendly. You have to get to learn those. In fact, a lot of these companies they have tutorials to just learn how these tools can be used. The third is a lot of open source tools get underutilized because companies tend to buy vendor tooling. And one of the reasons being that you don't want to take the ownership of the risk and

you want to essentially offload that risk to a third-party vendor. But still and building security from a lot of like that a lot of companies I've seen people prefer open source early on, but they have to shift towards a vendor dependency because it's hard to maintain After working 7 years as a security engineer and also as a security consultant with different companies, I've realized one thing that

security engineers they are very good problem solvers. If you ask them how to do a tax surface management or how to do cloud security, they would have 10 different ideas and all of them would be really good. But, they are not necessarily good developers. So, if you have to build something that scales for, let's say, 10 of your AWS accounts or 5,000 of your GCP projects, that's

where they would start faltering. And which is I I don't believe it's because of them, but it's they're not by default good developers. So, that's how it is. The challenge is scripts don't scale. So, something that you have written that works for your organization for a single AWS org or, let's say, a couple of orgs, that's good. Something you can run locally. But, when you have to

scale it up, when you have to ensure that artifacts are maintained, logs are being created, and all the output is being stored and queryable even after, let's say, a year from now, that's when the problems start to show up. The solution being, we thought instead of having scripts, let's create containerized workflows, and just to be clear, I'm not saying that you write the Python script and just

dockerize it and run it. That's not what I'm trying to say. Uh you'll see how we have tried to make it as a modular solution to any kind of security engineering problem. I'll quickly show you a 10,000-ft overview of the diagram. So, this is the the software is called Ship Shape Studio. There's a UI which is essentially a drag-and-drop graph. And you can So, essentially, you can

drag all these security components. For example, let's say, if you have to do port scanning, you would have an Nmap dockerized component on the left side. You can drag and drop it. If you want to, let's say, scan your cloud for security vulnerabilities. Another good open-source project is Prowler. You would have a component for it. You can also build your own component but by just adding your

Docker image for it and specifying the input and output ports for it. That's the UI part. So, first challenge is solved where you don't have to write scripts. You just have to drag and drop components. The second part comes the back-end API that essentially does the heavy lifting for you. So, for example, let's say you have your AWS or credentials, you can add it to the secrets

into it, and the back-end would take care to provision and like give it to the right component who requires that. It also contains analytics and workflow management. So, for example, if you want to trigger a workflow through web hooks, or let's say you want to do a cron kind of trigger, let's say every day you want to do a cloud security scanning, or you want to trigger

for every PR you want to do an AI code review. So, you have an AI open code agent, and you can bring your own key, add your own key, and do a code review like similar to what Code Rabbit and a lot of other tools do. Analytics being Now, for example, you have all these scans, but you want to visually as to how many of these PRs

were actually having critical vulnerabilities, or for example, yesterday we got to know about light light LLM having supply chain attack. So, you could essentially just ask the back-end as to where all we are using light LLM, which of these have the vulnerable dependencies, and you have that all all the metrics available to We didn't want to create our own back-end because we might falter at some places.

So, we thought, "Okay, we'll use Temporal." Temporal is a very, I would say, very battle-tested project, and Snapchat and Uber and a lot of big companies use it as one of their back-end to ship out a lot of these things. So, it gives you retries, it gives you fault tolerance. It essentially ensures that your workflow runs no matter what. the Temporal is essentially runs these workflows on

the workers. These workers are again deployed on Kubernetes, and each of these components are Docker components. So, the attack surface is very much composed. So, even if, let's say, someone runs an app scan, it's and tries to give some kind of sensitive parameters that is supposed like that is supposed to let's say do a shell execution, it's contained to that particular component only. Um that was an

extra thing. And then we have ClickHouse and for analytics and Kafka for event streaming. So, everything that you see, every workflow that runs, you have all the logging so that if if you have to show it to an auditor or if the internal team wants to see why a particular flow is failing, they can see it there on it on the UI itself. I'll show the UI

after some time like in the demo. So, you get all of these available because of the way it's architected right now. Yeah. So, coming to the components, I already mentioned that there's like a security-first components. Uh some of the components that are by default available. So, you can think of it as like n8n or uh make.com where you have these components for dragging and dropping onto the

canvas, the visual builder, and a reliable runtime to ensure that everything runs smoothly. Um so, let's talk about example just before the demo. Um so, what we can build with this? Um let's say you want to build a cloud security posture management tool, uh which in short basically scans your um cloud infrastructure uh to find any misconfigurations and security risk. And it could be anything, AWS, Azure.

Um so, most of the companies would have a cloud and they might be using like other tools like Wiz or Orca. Uh Sorry, Orca. But, we want to do something we want to build something open source and we will be using Prowler for this. Um so, Prowler is very open like very popular open source project which basically does the similar work. It scans everything. Uh it scans

your AWS, GCP, and Azure to find out um like what are the different findings and what are the different vulnerabilities. Um so, over here before like this is how like most of the most of us do like um yeah, so this is how we most of us do like before having the tool which we have tried building, uh, which is ShipSec. Um, and it's open source. So,

if anybody wants to check like just feel free to, you know, Google ShipSec. We are going to also have a demo. So, the way we are going to build this is like we will just write a Python script, uh, and maybe just add like the credentials which are relevant to it and then export it to the JSON file and then you will just upload it to S3

bucket. Uh, or maybe ClickHouse. But then comes like you will have to set up the ClickHouse. So, does this sound familiar to anyone who might have worked on these things like, uh, on similar script? Just want to see if anybody has done Okay, I see couple of hands. It's so great when we have like people, uh, who can relate to this stuff. So, now if you see

code on the right, you see that it's a simplified version of what happens. Obviously, it's going to be more complicated. So, what we get is we do not have the retries, there are no guarantees. Um, there is no observability, there is no traceability. Um, then there are no logs and there are no answers. You really don't get to know that what was, uh, failing silently. Why was

it even failing? When did it even fail failed? You will have to do a lot of manual work as like even as he mentioned, uh, with the light LLM thing. Um, I was just checking my Slack as well, uh, and we were just discussing what do we need to, you know, like look at and it's just a manual work which everyone is doing. Um, so, yeah, so

this is something which happens and then there are side effects everywhere. Uh, and again, impossible to audit. So, this is something which we have tried fixing, uh, with ShipSec Studio. Um, this is how it's going to look like. Uh, we tried to resolve this. So, essentially, you will have a visual builder, uh, where you can drag and drop things and, uh, this is the export of the

YAML uh, that you can see over here and you don't have to write any Python code. All these components are like these are composable. So, you can just drag and drop. And because we are using Temporal, as Assim already mentioned in the architecture diagram, so there is a retry retry in in like retry with it. So thanks to the other project Temporal, the failures are handled and

you can see all the failures on this dashboard, like kinds of bubble up to the UI and you can also set alerts for it. Pretty much how we have like observability for all these software toolings. There's a full visibility into every step. And then everything is auditable because like you maintain all the versions, so the compliance is also being taken care of. So yeah, as it is

mentioned all the points we have like it's auditable, version compliant compliant, provable and then there will be logs, traces and history. So let's do the demo. And and I hope demo gods is with us. Uh Okay, so let me How do I do this? Let me ex mirror the screen. Okay. I think you Yeah. So this is how the workflow builder looks like. So this is the

Ship Shape Studio looks like by default. So if you just get clone it and start and just there it would show this You can also connect to AWS and GitHub account. Let's see how we are building. So in the builder you have some of these default workflows. The workflow that Nancy was talking about right now, this is the minimal CSPM, so if you can see here this

is the entry point. By default this I mean you can trigger it through web hooks, schedules or inputs. So if you want to change the account ID here, as you can see, so you can define it into the entry point and through a webhook you can send the AWS account that you want to trigger it for. AWS credential builders, this is loading these secrets from the secret

store that we have available. So, right now there are two secrets added to it. This is different from the one that you saw in on the initial page where you had connect AWS account that uses IAM delegation, but this is where essentially you run through the credentials. Now, if you see here on the top you can see version five. So, every time let's say even if I

change this thing and save it again here, now you can see version six. Now, the reason version got updated because the X and Y coordinate got changed. So, every time you change even the slightest of thing there's a new version. The reason being because let's say if you have to show audit to anyone, then you can show the exact version that was being run at that time.

An example being let's say now instead of AWS credentials I had GCP access GCP credentials and I was passing that. And I ran the GCP report. So, if the auditor comes and I have to show them the audit report for the GCP credentials, I can show which version was running at that time or I could just duplicate the whole workflow and run it for GCP. On the

execution tab you can see all the runs that have run before. I'll quickly run once and show you a previous run. So, you can see the logs here and the timeline all of these here. So, for the workflow is running. It's actually running on my AWS account the account that we have. And you can see the Docker container at the bottom. So, this is Prowler that we

have versioned on our There's logs that you can see, you can all the logs are being streamed onto the platform. This is actual Prowler logs how it looks like right now. And you like we have agent on the platform. So, the agent logs would be here. There's IO, there's network. Let's quickly see a previous one that ran. So, this was the previous version and if you see

the logs here, this is the final output, let's say. And it's truncated here, but the idea is all these logs can also be dumped into an analytics thing that you can essentially see on the UI itself. So, you don't have to worry about like the execution has been done now where the results are being stored. You can do similarly for a lot of other things. There's an

activity which shows all the running things, etc., but by default we have a few of these workflows that are here. For example, cloud security scan is exactly what it was supposed to be. And there's a full code secure sorry, this AI code review it's been very popular these days. So, it's a mess right now, but you can see there's open grip, truffle log, and dependency scanner. These

are for scanning different stuff. And it normalizes it and sorry. There's an open So, we have used open code as an AI agent which essentially uses all the MC pin also gets all the data from here and it comments on the PR for every time you run the PR, it does a code review for it and gives you the data. So, this is the one right now.

Let me quickly jump back to We can show the GitHub repo as well. Oh, yeah. Sorry. so you might have seen they were like truncated things and everything. So, obviously contributions are welcome and that's how we grew. We have really put a lot of hard work in this project and it's open source. Um so, if you have any questions in the end and you would love to

contribute or if you have any feedback because feedback is also a form of contributions, that would be really helpful. Yeah. Uh yeah, coming back to slides Okay. Yeah, sorry. Okay, awesome. So, we tried showing the dashboard and we just want to think like now let's talk about use case that what could be the use cases. Uh now you can build like security products in-house. I know that

we have like a lot of open source projects. Um Um and there are like so many open source projects, but then there comes a lot of hassle where you have 10 different tools and maybe you have 10 different vendors and then you have to write 10 different agreements. So, these are reusable security workflows. So, once you have built it, you can run it anywhere whether it's EC2

machine or maybe Kubernetes cluster or anything. So, no more tools at all and no more hundreds of vendor agreements if you use ShiftLeft. So, once the system solves hundreds of use cases for security, it's open source and actually it can scale. Uh so, now you can use open source tools with this that are available such as uh Trivy and Git GitLeaks. So, all of these are production

ready with good open source licenses that you can use with your company itself. Um and also the other benefit is like teams can just share their work um instead of sharing those Python and bash scripts. So, there's a standardization which we've been talking about which has been the problem um since the beginning and you can mix and match workflows as well essentially to solve um different use

Um so, yeah this changes. You can use the you will have the reusable security workflows. Uh there are no more tool sprawl and um open source actually scales uh because it just makes it very easier to use those things and um these are also production ready. And teams can share workflows not just the scripts and security becomes composable. another use case that you might have already thought

about is that having a marketplace of security workflows. So, I as a security engineer have a lot of these workflows manually at every organization that I worked. And it would have been very handy if those things are available. So, as a security engineer, I can understand that, okay, I know what to do. But for people who are just, like, small companies who don't have budget to hire

a security engineer or buy 10 different security tools, they already have a secure default way that, okay, these are the five workflows that I need to have. And my cloud would be secured. I would have a code review for all the code changes that are being done. So, essentially, just pull the like you pull the studio, run it on a EC2 machine, and you have three or

four of these workflows that are running regularly for your own internal use case. So, yeah, that's the that's the other use case that we have thought of. Uh that's it. Uh spin up your first security workflow today. If you have any feedback, you can scan this one. And yeah. Thank you for coming today. Um any questions? Yeah, any questions? Thank you for the talk. I was very

inspired, and I really love the idea of providing this security workflow as a platform that you are giving. And when I'm thinking about the integration, the first thing I think about is how do I use it in my CI most of the time. But you are basically providing the CI platform of your own with the studio. So, is there an intention to provide a like CLI entry

point to the platform or something like that? Or is it like the studio is a part becomes a part of the daily uh workflow of the developers and becomes the part of the CI. So, uh that's a good point, and I We like we thought about having a CLI to execute these workflows. The other thing is that I mean, you can have a CLI, but having it

into CI/CD like other security checks, you can just trigger these workflows from there. So, that's one of the things that we have thought of so, but the dashboard being the central point of answer for all the security questions. So, even if you trigger it from your CI/CD pipeline, it should still have that storage or it should be running somewhere. So, it would run on the dashboard, but

yeah, you can have a step in your CI/CD that triggers these workflows. So, yeah. Makes sense. Thank you. Thank you for the talk. So, I was wondering if there is anything for managing the life cycle of the different that this tool uncovers or routing them to the correct team that owns the resource. Oh, yeah. I mean, yeah, all those are there with the the screen demo the

whole thing, but those things are there. So, there is like uh management component where essentially all the vulnerabilities you have option to either route it to the right person or you can like deprecate that these are not like false positives anymore. So, that's there on the dashboard. Thank