>>Hello, everyone. My name is John Jane Shake. I work from Iran. Tous Andi. I am here this afternoon very gratefully with Anders Vulcan, who is VP of technology strategy for cloud bees, a Miranda's partner and a well known company in the space that we're going to be discussing. Anders is also a well known entity in this space, which is continuous integration and continuous delivery. Um, you've seen already today some sessions that focus on specific implementations of continuous integration and delivery, um, particularly around security. And, uh, we think this is a critically important topic for anyone in the cloud space, particularly in this increasingly complicated kubernetes space. To understand, um, Miranda's thanks, Uh, if I can recapitulate our own our own strategy and, uh, and language that with complexity on uncertainty consistently increasing with the depth of the technology stacks that we have to deal with consistently, um um elaborating themselves that navigating this requires, um first three implementation of automation to increase speed, which is what C and C d do. Um, and that this speed ba leveraged toe let us ship and iterate code faster. Since that's ultimately the business that all of us air in one way or another. I would like, I guess, toe open this conversation by asking Onders what does he think of that core strategy? >>You know, I think you know, hitting the security thing, right? Right off the bat. You know, security doesn't happen by accident. You know, security is not something that you know, Like a like a server in a restaurant. You know, Sprinkles a little bit of Parmesan cheese right before they serve you the the food. It's not something you Sprinkle on at the end. It's something that has to be baked in from the beginning, not just in the kitchen, but in the supply chain from from from the very beginning. So the you know it's a feature, and if you don't build it, if you're not going to get an outcome that you're not gonna be happy with and I think the you know it's increasingly it's obviously increasingly important and increasingly visible. You know, the you know, the kinds of security problems that we that we see these days can can be, you know, life altering, for for people that are subject to them and and can be, you know, life or death for a company that that's exposed to it. So it's it's it's very, very important. Thio pay attention to it and to work to achieve that as an explicit outcome of the software delivery process. And I think, you know, C i n c d as as process as tooling as culture plays a big part in that because ah, lot of it has to do with, you know, set things up, right? Um run them the same way over and over, you know, get the machine going. Turned the crane. Now, you wanna you wanna make improvements over over time. You know, it's not just, you know, set it and forget it. You know, we got that set up. We don't have to worry about it anymore, but it really is a question of, you know, get the human out of the loop a lot of the times because if if you're dealing with configuring complex systems, you wanna make sure that you get them set up configured, you know, documented Ideally, you know, as code, whether it's a domain specific language or or something like that. And then that's something that you contest against that you can verify against that you can that you can difficult. And then that becomes the basis for for your, you know, for yourself, for pipelines, for your automation around, you know, kind of the software factory floor. So I think automation is a key aspect of that because it, you know, it takes a lot of the drudgery out of it, for one thing, So now the humans have more time to spend on doing on the on the creative things on the things that we're good at a zoo. Humans and it also make sure that, you know, one of the things that computers are really good at is doing the same thing over and over and over and over. Eso that kind of puts that responsibility into the hands of the entity that that knows how to do that well, which is which is the machine eso I think it's, you know, it's a light. It's a deep, deep topic, obviously, but, you know, automation plays into it. Uh, you know, small batch sizes play into it, you know, being able to test very frequently whether that's testing in. You're kind of you're C I pipeline where you're sort of doing building mostly unit testing, maybe some integration testing, but also in layering in the mawr. Serious kinds of testing in terms of security scanning, penetration, testing, vulnerability, scanning. You know those sorts of things which, you know, maybe you do on every single see I Bill. But most people don't because those things tend toe take a little bit longer on. And you know you want your sea ice cycle to be as fast as possible because that's really in service of the developer who has committed code and wants toe kind of see the thumbs up from the system saying it. And, um, so most organizations most organizations are are are focusing on, you know, making sure that there's a follow on pipeline to follow on set of tests that happened after the C I passes successfully and and that's, you know, where a lot of the security scanning and those sorts of things happen. >>It's a It's an interesting problem. I mean, you mentioned, um, what almost sounds like a Lawrence Lessig Ian kind of idea that, you know, code is law in enterprises today, code particularly see, I code ends up being policy, but At the same time, there's, Ah, it seems to me there's a an alternative peril, which is, as you increase speed, particularly when you become more and more dependent on things like containers and layering technology to provide components and capabilities that you don't have to build yourself to your build pipeline, that there are new vulnerabilities, potentially that creep in and can creep in despite automation. Zor at least 1st. 1st order automation is attempts toe to prevent them from creeping in. You don't wanna you wanna freeze people on a six month old version of a key container image. But on the other hand, if the latest version has vulnerabilities, that could be a problem. >>Yeah, I mean, it's, you know, it's it's a it's a it's a double edged sword. It's two sides of the same coin. I think you know, when I talked to a lot of security people, um, you know, people to do it for a living is supposed to mean I just talk about it, um, that Z not completely true. But, um, the ah, lot of times the problem is old vulnerabilities. The thing that I think keeps a lot of people up at night isn't necessarily that the thing at the tip of the releases for particular, you know, well known open source, library or something like that. But that's gonna burn you all the vast majority of the time. And I want to say, like, 80 85% of the time. The vulnerability is that you that you get hosed by are ones that have been known about for years. And so I think the if I had to pick. So if you know, in that sort of two sides of that coin, if I had to pick, I would say Be aggressive in making sure that your third party dependencies are updated frequently and and continuously right, because that is the biggest, biggest cause of of of security vulnerabilities when it comes to third party code. Um, now you know the famous saying, You know, move fast and break things Well, there's certain things you don't want to break. You know you don't want to break a radiation machine that's going to deliver radio radiotherapy to someone because that will endanger their health. So So those sorts of systems, you know, naturally or subject a little bit more kind of caution and scrutiny and rigor and process those sorts of things. The micro service that I run that shows my little avatar when I log in, that one probably gets a little less group. You know, Andre rightfully so. So I think a lot of it has to do. And somebody once said in a I think it was, Ah, panel. I was on a PR say conference, which was, which was kind of a wise thing to say it was Don't spend a million dollars protecting a $5 assets. You know, you wanna be smart and you wanna you wanna figure out where your vulnerabilities they're going to come from and in my experience, and and you know, what I hear from a lot of the security professionals is pay attention to your supply chain. You're you want to make sure that you're up to date with the latest patches of, of all of your third party, you know, open source or close source. It doesn't really matter. I mean, if anything, you know, open source is is more open. Eso You could inspect things a little bit better than the close source, but with both kinds of streams of code that you consume and and use. You wanna make sure that you're you're more up to date as opposed to a less up to date? Um, that generally will be better. Now, can a new version of the library cause problems? You know, introduce bugs? You know, those sorts of things? Yes. That's why we have tests. That's what we have automated tests, regression, sweets, You know, those sorts of things. And so you wanna, you know, you wanna live in a in a world where you feel the confidence as a as a developer, that if I update this library from, you know, one debt owed at 3 to 1 debt owed at 10 to pick up a bunch of, you know, bug fixes and patches and those sorts of things. But that's not going to break some on demand in the test suites that that will run against that ought to cover that that sort of functionality. And I'd rather be in that world of Oh, yeah, we tried to update to that, but it But it broke the tests and then have to go spend time on that, then say, Oh, it broke the test. So let's not update. And then six months later, you do find out. Oh, geez. There was a problem in one that owed at three. And it was fixed in one. That about four. If only we had updated. Um, you know, you look at the, um you look at some of the highest profile security breaches that are out there that you sort of can trace toe third party libraries. It's almost always gonna be that it was out of date and hadn't been patched. That's so that's my you know, opinionated. Take on that. Sure. >>What are the parts of modern C I c D. As opposed to what one would encounter 56 years ago? Maybe if we can imagine that is being before the micro services and containers revolution really took off. >>You know, I think e think you're absolutely right that, you know, not the whole world is not doing. See, I Yeah, and certainly the whole world is not doing city yet. Um, you know, I think you know, as you say, we kind of live in a little bit of an ivory tower. You know, we live in an echo chamber in a little bit of a bubble Aziz vendors in this space. The truth is that I would say less than 50% of the software organizations out there do real. See, I do real CD. The number's probably less than that. Um, you know, I don't have anything to back that up other than just I talked to a lot of folks and work with, you know, with a lot of organizations and like, Yeah, that team does see I that team does Weekly builds You know, those sorts of things. It's it's really all over the place, Onda. Lot of times there's There's definitely, in my experience, a high correlation there with the amount of time that a team or a code base has been around, and the amount of sort of modern technologies and processes and and and so on that are that are brought to it on. And that sort of makes sense. I mean, if you if you're starting with the green field with a blank sheet of paper, you're gonna adopt, you know, the technologies and the processes and the cultures of today. A knot of 5, 10 15 15 years ago, Um but but most organizations air moving in that direction. Right? Andi, I think you know what? What? What? What's really changed in the last few years is the level of integration between the various tools between the various pieces and the amount of automation that you could bring to bear. I mean, I you know, I remember, you know, five or 10 years ago having all kinds of conversations with customers and prospects and and people of conferences and so on and they said, Oh, yeah, we'd like to automate our our software development life cycle, but, you know, we can't We have a manual thing here. We have a manual thing there. We do this kind of testing that we can automate it, and then we have this system, but it doesn't have any guy. So somebody has to sit and click on the screen. And, you know, and I used to say e used to say I don't accept No for an answer of can you automate this right? Everything. Anything can be automated. Even if you just get the little drinking bird. You know that just pokes the mouse. Everyone something. You can automate it, and I Actually, you know, I had one customer who was like, Okay, and we had a discussion and and and and they said, Well, we had this old Windows tool. We Its's an obscure tool. It's no longer updated, but it's it's it's used in a critical part of the life cycle and it can't be automated. And I said, Well, just install one of those Windows tools that allows you to peek and poke at the, you know, mass with my aunt I said so I don't accept your answer. And I said, Well, unfortunately, security won't allow us to install those tools, Eh? So I had to accept No, at that point, but But I think the big change were one of the biggest changes that's happened in the last few years is the systems now have all I'll have a p i s and they all talk to each other. So if you've gotta, you know, if you if you've got a scanning tool, if you've got a deployment tool, if you have a deployment, you know, infrastructure, you know, kubernetes based or, you know, kind of sitting in front of our around kubernetes thes things. I'll talk to each other and are all automated. So one of the things that's happened is we've taken out a lot of the weight states. A lot of the pauses, right? So if you you know, if you do something like a value stream mapping where you sit down and I'll date myself here and probably lose some of the audience with this analogy. But if you remember Schoolhouse Rock cartoons in in the late seventies, early eighties, there was one which was one of my favorites, and and the guy who did the music for this passed away last year, sadly, But, uh, the it was called How a bill Becomes a Law and they personified the bill. So the bill, you know, becomes a little person and, you know, first time passed by the house and then the Senate, and then the president either signs me or doesn't and or he vetoes, and it really sort of did this and what I always talk about with respect to sort of value stream mapping and talking about your processes, put a GoPro camera on your source codes head, and then follow that source code all the way through to your customer understand all of the stuff that happens to it, including nothing, right? Because a lot of times in that elapsed time, nothing keeps happening, right. If we build software the way we were sorry. If we build cars the way we build software, we would install the radio in a car, and then we would park it in a corner of the factory for three weeks. And then we might remember to test the radio before we ship the car out to the customer. Right, Because that's how a lot of us still develop some for. And I think one thing that's changed in the in the last few years is that we don't have these kind of, Well, we did the bill. So now we're waiting for somebody to create an environment and rack up some hardware and install an operating system and install. You know, this that and the other. You know, that that went from manual to we use Scheffer puppet to do it, which then went to we use containers to do it, which then went to we use containers and kubernetes to do it. So whole swaths of elapsed time in our software development life cycles basically went to nothing, right and went to the point where we can weaken, weaken, configure them way to the left and and and follow them all the way through. And that the artifact that we're delivering isn't necessarily and execute herbal. It could be a container, right? So now that starts to get interesting for us in terms of being able to test against that container scan against that container, def. Against that container, Um, you know, and it, you know, it does bring complexity to in terms of now you've got a layered file system in there. Well, what all is in there, you know, And so there's tools for scanning those kinds of things, But But I think that one of the biggest things that's happened is a lot of the natural pause. Points are no longer natural. Pause points their unnatural pause points, and they're now just delays in yourself for delivery. And so what? What a lot of organizations are working on is kind of getting to the point where those sorts of things get get automated and connected, and that's now possible. And it wasn't 55 or 10 years ago. >>So It sounds like a great deal of the speed benefit, which has been quantified many different ways. But is once you get one of these systems working, as we've all experienced enormous, um, is actually done by collapsing out what would have been unused time in a prior process or non paralyze herbal stuff has been made parallel. >>I remember doing a, uh, spent some time with a customer, and they did a value stream mapping, and they they found out at the end that of the 30 days of elapsed time they were spending three days on task. Everything else was waiting, waiting for a build waiting foran install, waiting for an environment, waiting for an approval, having meetings, you know, those sorts of things. And I thought to myself, Oh, my goodness, you know, 90% of the elapsed time is doing nothing. And I was talking to someone Gene Kim, actually, and I said, Oh my God, it was terrible that these you know, these people are screwed and he says, 0 90%. That's actually pretty good, you know? So So I think you know, if you if you think today, you know, if you If you if you look at the teams that are doing just really pure continuous delivery, you know, write some code committed, gets picked up by the sea ice system and passes through CIA goes through whatever coast, see I processing, you need to do security scanning and so on. It gets staged and it gets pushed into production. That stuff can happen in minutes, right? That's new. That's different. Now, if you do that without having the right automated gates in place around security and and and and those sorts of things you know, then you're living a little bit dangerously, although I would argue not necessarily any more dangerously, than just letting that insecure coat sit around for a week before your shipment, right? It's not like that problem is going to fix itself if you just let it sit there, Um, but But, you know, you definitely operated at a higher velocity. Now that's a lot of the benefit that you're tryingto trying to get out of it, right? You can get stuff out to the market faster, or if you take a little bit more time, you get more out to the market in, in in the same amount of time you could turn around and fix problems faster. Um, if you have a vulnerability, you can get it fixed and pushed out much more quickly. If you have a competitive threat that you need to address, you can you know, you could move that that much faster if you have a critical bug. You know, I mean, all security issues or bugs, sort of by definition. But, you know, if you have a functionality bug, you can you can get that pushed out faster. Eso So I think kind of all factors of the business benefit from from this increase in speed. And I think developers due to because anybody you know, any human that has a context switch and step away from something for for for, you know, duration of time longer than a few minutes, you know, you're gonna you're gonna you're gonna you're gonna have to load back up again. And so that's productivity loss. Now, that's a soft cost. But man, is it Is it expensive and is a painful So you see a lot of benefit there. Think >>if you have, you know, an organization that is just starting this journey What would you ask that organization to consider in orderto sort of move them down this path? >>It's by far the most frequent and almost always the first question I get at the end of the talk or or a presentation or something like that is where do we start? How do I know where to start? And and And there's a couple of answers to that. What one is Don't boil the ocean, right? Don't try to fix everything all at once. You know that because that's not agile, right? The be agile about your transformation Here, you know, pick, pick a set of problems that you have and and make a, you know, basically make a burn down list and and do them in order. So find find a pain point that you have right and, you know, just go address that and and try to make it small and actionable and especially early on when you're trying to affect change. And you're tryingto convinced teams that this is the way to go and you may have some naysayers, or you may have people who are skeptical or have been through these processes before that have been you know failures released, not the successes that they that they were supposed to be. You know, it's important to have some wind. So what I always say is look, you know, if you have a pebble in your shoe, you've got a pain point. You know how to address that. You know, you're not gonna address that by changing out your wardrobe or or by buying a new pair of shoes. You know, you're gonna address that by taking your shoe off, shaking it until the pebble falls out there putting the shoe back on. So look for those kinds of use cases, right? So if you're engineers are complaining that whenever I check in the build is broken and we're not doing see, I well, then let's look at doing C I Let's do see eye, right? If you're not doing that. And for most organizations, you know, setting up C I is a very manageable, very doable thing. There's lots of open source tooling out there. There's lots of commercial tooling out there. Thio do that to do it for small teams to do it for large teams and and everything in between. Um, if the problem is Gosh, Every time we push a change, we break something. You know where every time something works in staging it doesn't work in production. Then you gotta look at Well, how are these systems being configured? If you're If you're configuring them manually, stop automate the configuration of them. Um, you know, if you're if you're fixing system manually, don't you know, as a friend of mine says, don't fix, Repave? Um, you know, you don't wanna, you know, there's a story of, you know how how Google operates in their data centers. You know, they don't they don't go look for a broken disk drive and swap it out. You know, when it breaks, they just have a team of people that, like once a month or something, I don't know what the interval is. They just walked through the data center and they pull out all the dead stuff and they throw it out, and what they did was they assume that if the scale that they operate, things are always going to break physical things are always going to break. You have to build a software to assume that breakage and any system that assumes that we're going to step in when a disk drive is broken and fix it so that we can get back to running just isn't gonna work at scale. There's a similarity. There's sort of ah, parallel to that in in software, which is you know, any time you have these kinds of complex systems, you have to assume that they're gonna break and you have to put the things in place to catch those things. The automated testing, whether it's, you know, whether you have 10,000 tests that you that you've written already or whether you have no tests and you just need to go right, your first test that that journey, you've got to start somewhere. But my answer thio their questions generally always just start small, pick a very specific problem. Build a plan around it, you know, build a burned down list of things that you wanna address and just start working your way down that the same way that you would for any, you know, kind of agile project, your transformation of your own processes of your own internal systems. You should use agile processes for those as well, because if you if you go off for six months and and build something. By the time you come back, it's gonna be relevant. Probably thio the problems that you were facing six months ago. >>A Then let's consider the situation of, ah, company that's using C I and maybe sea ice and C d together. Um, and they want to reach what you might call the next level. Um, they've seen obvious benefits they're interested in, you know, in increasing their investment in, you know and cycles devoted to this technology. You don't have to sell them anymore, but they're looking for a next direction. What would you say that direction should be? I >>think oftentimes what organizations start to do is they start to look at feedback loops. So on DAT starts to go into the area of sort of metrics and analytics and those sorts of things. You know what we're we're always concerned about? You know, we're always affected by things like meantime to recovery. Meantime, the detection, what are our cycle times from, you know, ideation, toe codecommit. What's the cycle? Time from codecommit the production, those sorts of things. And you know you can't change what you don't measure eso so a lot of times the next step after kind of getting the rudimentary zoo of C I Orsini or some combination of both in places start to measure. Stop you, Um, and and then but But there. I think you know, you gotta be smart about it, because what you don't want to do is kind of just pull all the metrics out that exists. Barf them up on the dashboard. And the giant television screens say boom metrics, right. You know, Mike, drop go home. That's the wrong way to do it. You want to use metrics very specifically to achieve outcomes. So if you have an outcome that you want to achieve and you can tie it to a metric start looking at that metric and start working that problem once you saw that problem, you can take that metric. And you know, if that's the metric you're showing on the big you know, the big screen TV, you can pop that off and pick the next one and put it up there. I I always worry when you know a little different when you're in a knock or something like that. When when you're looking at the network stuff and so on. But I'm always leery of when I walk into to a software development organization. You know, just a Brazilian different metrics, this whole place because they're not all relevant. They're not all relevant at the same time. Some of them you wanna look at often, some of them you just want to kind of set an alarm on and make sure that, you know, I mean, you don't go down in your basement every day to check that the sump pump is working. What you do is you put a little water detector in there and you have an alarm go off if the water level ever rises above a certain amount. Well, you want to do the same thing with metrics, right? Once you've got in the water out of your basement, you don't have to go down there and look at it all the time. You put the little detector in, and then you move on and you worry about something else. And so organizations as they start to get a little bit more sophisticated and start to look at the analytics, the metrics, um, start to say, Hey, look, if our if our cycle time from from, you know, commit to deploy is this much. And we want it to be this much. What happens during that time, And where can we take slices out of that? You know, without without affecting the outcomes in terms of quality and so on, or or if it's, you know, from from ideation, toe codecommit. You know what? What can we do there? Um, you start to do that. And and then as you get those sort of virtuous cycles of feedback loops happening, you know, you get better and better and better, but you wanna be careful with metrics, you know, you don't wanna, you know, like I said, you don't wanna barf a bunch of metrics up just to say, Look, we got metrics. Metrics are there to serve a particular outcome. And once you've achieved that outcome, and you know that you can continue to achieve that outcome, you turn it into an alarm or a trigger, and you put it out of sight. And you know that. You know, you don't need to have, like, a code coverage metric prominently displayed you you pick a code coverage number that you're happy with you work to achieve that. Once you achieve it, you just worry about not going below that threshold again. So you can take that graph off and just put a trigger on this as if we ever get below this, you know, raising alarm or fail a build or fail a pipeline or something like that and then start to focus on improving another man. Uh, or another outcome using another matter >>makes enormous sense. So I'm afraid we are getting to be out of time. I want to thank you very much on this for joining us today. This has been certainly informative for me, and I hope for the audience, um, you know, thank you very, very much for sharing your insulin.