>>Welcome to the cubes, continuing coverage of AWS reinvent 2021 find Lisa Martin and we are running one of the industry's most important and largest hybrid tech events with AWS in this ecosystem partners. This year, we have two live sets, two remote sites over 100 guests talking about the next decade in cloud innovation. And we're excited to be joined by Patrick Jeanne, the CTO of OutSystems Patrick. Welcome to the program. >>Thank you. I appreciate being one of those 100 guests, >>One of the 100, one of the elite, 100, we'll say it like that. Right? So, so OutSystems has some revolutionary news. You guys are saying, you know, what developer experience needs to change? Tell us more. >>It does. I mean, it needs to change. And I've been in the industry developing applications for too many years dimensions basically since I was 12 years old writing software and, you know, going over that time and thinking about it, doing the traditional software development route. So many applications that take too long was, you know, costly to build so much risk involved in it. Eventually it didn't meet all the requirements. And if you look at the investment we make in software, which is important, I mean, software is a, is a unique differentiator for, for businesses. That investment has such a high risk and a high cost, and that needs to change and it needs to change just because of the complexity that is in that process inherent in it that's. And that is what we are doing and OutSystems is tackling that problem. And, um, from a business standpoint, it must change. >>It must change that that is strong words there. So talk to me about what you're announcing, what, what were the gaps in the market customer feedback? Was it, or were there any catalysts from the pandemic going we've got to change this developer experience and this is the time >>For sure. I mean, if you think about from the pandemic and I mean, we were on a journey for digital transformation. We've been on this journey for a number of years and it really accelerated that the experiences that we have with each other, with you and me, we're not the same studio today. I mean, there's there reasons that we have used this experience remote, we have a technology that can do it, the pandemic accelerated that. And so, so much of the experiences we have are digital experiences. And if you think about it, there's a device in between us. There's going to be a device in between all the people viewing what we're looking at, that experience that, uh, that they will have with us will be basically surfaced through an application on that device. And the pandemic has really accelerated that. And that's an area that we play in, obviously for what's considered low code application development. >>And if you just think about application development in general, that's what powers all of these experiences. And going back to that, you know, statement about that, it needs to change if we need these experiences to be diverse, if we need these experiences to be meaningful, if we need them to make sure that when people engage, as far as what that device is, something that brings, you know, delight and pleasure to them, we need developers across the board. Investing in that today, there is a very constrained market for professional developers, but because of the inherent complexity in software development. And so if you think about how that's almost almost you're limiting access to the people who can create those experiences, that's not a good situation. There's about 25 million developers in the world that would consider themselves developers today, 7, 8, 9, 10 billion devices out there. Think of that disparity between those two numbers. >>And so we need a larger number of people to actually develop applications. So that experience can be much more diverse. We need to expose development to many more people. That is the problem today with software development is that it is complex. It is too specialized. It's too inherit as far as with failure when you get it together. And so either you shy away from that as an organization or as an individual to do development, or you go on these very long development as far as cycles to actually create these applications. What we do is we take the approach of let's make it very simple to get into, you know, some terms and call it citizen developer, low code, basically all they're saying is let's, let's reduce the risk of development. Let's go into a process where we make it accessible to more and more people. You can go through and develop applications with the lower risk. You can build change into that process and you can get value into end users as rapidly as possible. So that's, that is the value proposition. That is what needs to change >>Strong value proposition well said, Patrick, talking about reducing the complexity, uh, the risk as well. So, so go ahead and crack crack open what you guys are actually announcing today. >>Yeah, for sure. So with, we we've been doing this for many years, we have, um, software development, we have 14 million plus as far as end-users using applications that have been developed with the Al systems platform, what we're announcing is taking some of the great benefits that we have to what you'd consider as the first part of that low code process, where you have a, you have a developer that has an idea, and there's a canvas in front of you. You know, you're, you're an artist, right? But again, this is what you are as a developer. And so you go in and you create that application. We've been doing this for many years and it works really well. But thing that we're improving upon now is the ability to do that and scale that out to millions of end-users 10 millions of end-users. So if you think about that inherent speed of developing an application, using a platform like OutSystems, we're taking that same concept and rolling that into an internet scale application, hosting architecture. >>So any developer that uses our systems, basically like it would be comparable to a traditional development team that has application architects, cloud architects, security, engineers, database engineers, a whole team of very smart individuals that generally the, the biggest technology companies in the world can put together. Most companies can't do that. You don't have access to that type of that type of skillset. And so we're providing that with project Neo, which is what we're announcing today in our, um, at our user conference and customer conference, is this brand new as far as platform that allows you to build these applications at scale. And this is initially built on AWS using all the great AWS technologies. If you look at what AWS has done and provided to developers today, it's amazing. It is absolutely amazing. The amount of technologies that you can leverage. It's also daunting because as a traditional developer, you have to go in and choose, you know, what do you do? It's like, there's just massive cognitive load as far as upfront when you're going to design and application and what type of messaging what's at the data store. Well, how do I host my application? What type of network, you know, as far as security do I use, we're taking all that heavy lifting, all that undifferentiated, heavy lifting off of the developers, putting it into the project, Neo platform, allowing a single developer or a small group of developers to actually leverage that best in class architecture on AWS today. >>So when you're talking to developers, what are some of the things that you described as the unique differentiators of project Neo? It sounds like this was really apt and apt time for change, but when you're talking to those folks, what do you say? You know, 1, 2, 3, these are the things that make project Neo unique. >>Yeah. So you're the first is don't worry about the application architecture. Like I mentioned, don't when you go in that, the idea, the concept of that application and what it means to, to deliver some value, whether it's into a business or a hobby or whatever. I mean, however, you're developing application, you're doing it for a reason. You want that value to come out as quickly as possible. You want that experience. And so that first thing is you don't have to worry about the architecture anymore. So in the past, you know, you'd have to think about if it's a very large application, it's millions and millions of end-users. How do you structure that? How do you put it together? That concern is removed from you in that process? The other thing is we solve the problem of software disintegration. So with traditional development, when you develop an application and you get it into the hands of end, users get immediately starts to disintegrate. >>So there will be bugs that will appear. There will be, as far as, um, security flaws that will come up services that you use will become deprecated. We'll swap out cloud services, you know, AWS or Azure or Google, we'll swap out cloud services with different services behind the scenes version that we new versions of those that is software disintegration. As soon as you develop software today and all of these beautiful cloud services that you use and components, they often something will become outdated almost by the time you release it. A lot of times with software development projects, it literally is you start with some version or some component before you can get that out in a traditional mode. Something becomes outdated. We solve that issue. What I like to call software disintegration, we, as far as our systems, ensure we invest in that platform. And so when we need to change out those components, so services, those versions fix is for a security flaws, fixed bugs. >>We do that and it seamless. And so your application, you do not have to rewrite your application. You do not have to go through that process as a tradition, as a developer on our systems like you would, as your traditional developer, we solve that software disintegration issue. So it is it's, it's very empowering to developers to not have to worry about that. There are many, you look at the numbers today about how much is invested in innovation versus maintenance. You know, a lot of companies start out at 70% innovation, 30% as far as maintenance. And then over time that flips and you'll get to 30% of your time spent on innovations development, 70% maintenance, that burden we removed that burden. >>Those are some really powerful statements protect that you mean, I really liked the way that you described software disintegration. I've actually never heard that term before. And it kind of reminded me of, you know, when you buy a brand new car, you drive it off. The lot the value goes down right away, then before you even get things out. And on the consumer side, we know that as soon as we buy the newest iPhone, the next one's going to be out, or there's some part of it, that's going to be outdated in terms of technical debt. I was reading a stat that technical debt is expected to reach and costs businesses 5 trillion us dollars over the next 10 years. How does OutSystems helps customers address the challenges with technical debt and even reduce it? >>Yeah. If you think about the guy, the truest sense of technical debt, it's a, it's a decision that you make in the development process to basically, you know, load up the future with some work that you don't want to do right now. And so we're solving that issue where number one, we, you don't even have to make that decision. So you can go back to that concept of removing that cognitive load of, do I get the software out right now or do I get it out in the right way? And that's really what technical debt technical debt is saying. I need to get it out now. And there are some things I want to do that it'd be better if I did them now, but I'm going to go ahead and push that out into the future. You don't have to do that today with us. >>And so what happens with our systems? We invest in that platform, and this is hard. I mean, this is not an easy thing to do. This is why we have some of the best and brightest engineers focusing on this process at the heart of this, not to get too technical, but the heart of this is what we call the true change engine. But then, um, within our platform, we go through and we look at all of the changes that you need to make. So if you think of that concept of technical debt of like, oh, I want to get this into the hands of man users, but I don't want to invest in the time to do something right. It's always done right. As far as with the OutSystems platform. So we take that, we look at the intent of your change. So it's like a, it's like a process where you tell us the intent. >>When you, as a application developer, you're designing an application, you tell us the intent of the application is to look and feel. It could be some business processes can be some integrations. We determine what's the best way to do that. And then once again, from a software disintegration standpoint, we continue to invest in all the right ways to do that the best way possible. And so, I mean, we have customers that have written applications. That's 10, 15 years ago, they're still using our platform with those same applications they've added to them, but they actually have not rewritten those applications. And so if you think about the normal traditional development process, the technical debt incurred over that type of lifetime would be enormous with us. There's no technical debt. They're still using the same application. They have simply added capabilities to it. We invest in that platform. So they don't have to >>So big business outcomes there, obviously from a developer productivity perspective, but from the company wide perspective, the ability to eliminate technical debt, some significant opportunities there. Talk to me about the existing OutSystems customers. When are they going to be able to take advantage of this? What is the migration or upgrade path that they can take? >>Yeah. And so it's, it was very important to me and, and, uh, and the team, as far as our systems, to be able to integrate, to innovate as far as for customers, without disrupting customers. And we've probably all been through this path of great new technology is awesome. But then to actually utilize that technology when you're a current customer, it creates pain. And so we've invested heavily in making sure that the process is pain-free so you can use project Niamh. So we are announcing it as it was in public preview, as far as now, and then we will release it from GA as far as in the first quarter of next year. So over this timeframe, you'll be able to get in and try it out and all that continue to use your current version, which is OutSystems 11. So what we, what we affectionately call it 11, as far as Alice systems, Al systems, 11 version, and continued to use, and you can continue to use that today. >>Side-by-side and coexistence with the project, Neo and project Neo is a code name. So we will, we will have an official product name is for as at launch, but it's our it's. Our affectionate is kind of a unofficial mascot as Neo. So we call it project Neo bit of a fun thing, and you can use it side by side. And then in the future, you'll be able to migrate applications over, or you can just continue to coexist. I mean, we see a very long lifetime for OutSystems 11, it's a different platform, different technology behind the scenes project, Neos, Kubernetes base, Lennox containers. Based once again on the bill, we went in with the, just looked at it and said, rearchitect re-imagined, how would you do this? If you had the best and brightest, as far as engineers, architects, um, you know, we have, which we do, you know, very smart in those people. >>And we did that. And so we did that for our customers. And so Neo is that how systems 11 still a great choice. If you have applications on it, you can use it. And we have, we anticipate that customers will actually side by side, develop on both in which we have some customers in preview today. And that's the process that they have. They will develop on 11, they will develop on the Neo and they will continue to do that. And there's no, we, we are dedicated to making sure that there's no disruption and no pain in that process. And then when customers are ready to migrate over, if that's what they choose, we'll help them migrate over. >>You make it sound easy. And I was wondering if project Neo had anything to do with the new matrix movie, I just saw the trailer for it the other day. >>It was a happy coincidence. It is not easy. Let me, let me be clear. It is something we have been working on for three years and really this last year really kicked into high gear. And, um, you know, a lot of behind the scenes work, obviously for us, but once again, that's our value proposition. It's we do the hard work. So developers and customers don't have to do that hard work, uh, but no relations in the L I love, I do love the matrix movies. So it's a, it's a nice coincidence. >>It is a nice coincidence. Last question, Patrick, for you, you know, as we wrap up the calendar year 2021, we head into 20, 22. I think we're all very hopeful that 2022 will be a better year than the last two. What are some of the things that you see as absolutely critical for enterprises? What are they most concerned about right now? >>Yeah, I think it's look, I mean, it's, obviously it has been a crazy couple of years. And, um, and if you think about what enterprises want, I mean, they want to provide, uh, a great experiences for their customers, a great experience for their employees. Once again, digital transformation, we're where you don't even kind of talk about digital transformation more because we're in it. And I think that customers need to make sure that the experiences they provide these digital experiences are the best possible experiences. And these are differentiators. These are differentiators for employees is, are differentiators for customers. I believe that software is one of the big differentiators for businesses today and going forward, and that will continue to be so we're where businesses may be invested in supply chains and invested in certain types of technologies. Business will continue to invest in software because software is that differentiator. >>And if you look at where we fit, you can go, you can go buy, you know, some great satisfied where my software as a service off the shelf in the end, you're just like every other business you bought the same thing that everybody else has bought. You can go the traditional development route, where you invest a bunch of money. It's a high risk, takes a long time. And once again, you may not get what you want. We believe what is most important to businesses. Get that unique software that fits like a glove that is great for employees is great for their customers. And it is a unique differentiator for them. And I really see that in 2022, that's going to be big and, and going forward. They're the legs for that type of investment that companies make and their return on that is huge. >>I agree with you on that in terms of software as a differentiator. No, we're seeing every company become a software company in every industry these days to be first to survive in the last 20 months and now to be competitive, it's really kind of a must have. So Patrick, thank you for joining me on the program, talking about project Neo GA in quarter of calendar year 22 exciting stuff. We appreciate your feedback and your insights and congratulations on project Neil. Thanks, Lisa. Appreciate it for Patrick Jean I'm Lisa Martin, and you're watching the cubes continuous coverage of re-invent 2021.

(Upbeat intro music) >> Welcome to theCUBE's continuing coverage of AWS re:Invent 2021. I'm Lisa Martin and we are running one of the industry's most important and largest hybrid tech events with AWS in this ecosystem partners this year. We have two live sets, two remote sets over 100 guests talking about the next decade in cloud innovation. And we're excited to be joined by Patrick Jean the CTO of OutSystems, Patrick welcome to the program. >> Thank you, I appreciate being one of those 100 guests. >> One of the 100, one of the elite 100, we'll say it like that, right? >> Yes. >> So OutSystems has some revolutionary news. You guys are saying, you know what, developer experience needs to change, tell us more. >> It does I mean, it needs to change. And I've been in the industry developing applications for too many years to mention, basically since I was 12 years old writing software and going over that time and thinking about it, doing the traditional software development route. So many applications that take too long was costly to build, so much risk involved in it. Eventually it didn't meet all the requirements. And if you look at the investment we make in software, which is important, I mean, software is a unique differentiator for businesses. That investment has such a high-risk and a high cost and that needs to change. And it needs to change just because of the complexity that is in that process inherent in it. That's and that is what we are doing in OutSystems is tackling that problem. And from a business standpoint, it must change. >> It must change that is strong words there. So talk to me about what you're announcing what were the gaps in the market, customer feedback, were there any catalysts from the pandemic going we've got to change this developer experience and this is the time. >> For sure. I mean, if you think about from the pandemic and I mean, we were on a journey for digital transformation. We've been on this journey for a number of years the pandemic really accelerated that the experiences that we have with each other, you and me are not in the same studio today. I mean, there reasons that we use this experience remotely. We have a technology that can do it. The pandemic accelerated that. And so, so much of the experiences we have are digital experiences. And if you think about it, there's a device in between us. There's going to be a device in between all the people viewing what we're looking at. That experience that they will have with us will be basically surfaced through an application on that device. And the pandemic has really accelerated that. And that's an area that we play in, obviously for what's considered low-code application development. And if you just think about application development in general, that's what powers all of these experiences. And going back to that statement about that it needs to change. If we need these experiences to be diverse, if we need these experiences to be meaningful, if we need them to make sure that when people engage as far as what that device is something that brings, delight and pleasure to them. We need developers across the board investing in that. Today there is a very constrained market for professional developers because of the inherent complexity in software development. And so if you think about how that's almost, almost here limiting access to the people who can create those experiences, that's not a good situation. There's about 25 million developers in the world that would consider themselves developers today, seven, eight, nine, 10 billion devices out there. Think of that disparity between those two numbers. And so we need a larger number of people to actually develop applications so that experience can be much more diverse. We need to expose development to many more people. That is the problem today with software development is that it is complex, it is too specialized. It's too inherent as far with failure when you get it together. And so either you shy away from that as an organization or as an individual. To do development are you going on these very long development as far as cycles to actually create these applications? What we do is we take the approach of let's make it very simple to get into. Sometimes we call it citizen developer, low-code, basically all they're saying is let's reduce the risk of development. Let's go into a process where we make it accessible to more and more people. You can go through and develop applications with the lower risk. You can build change into that process. You can get value into end users as rapidly as possible. So that is the value proposition, that is what needs to change. >> Strong value proposition well said, Patrick. Talking about reducing the complexity, the risk as well. So go ahead and crack open what you guys are actually announcing today. >> Yeah, for sure. So we've been doing this for many years. We have software development, we have 14 million plus as far as end-users using applications that have been developed with the Allo systems platform. What we're announcing is taking some of the great benefits that we have to what you'd consider as the first part of that low-code process. Where you have a developer that has an idea, and there's a canvas in front of you. You're an artist, right, with a canvas that's what you are as a developer. And so you go in and you create that application. We've been doing this for many years and it worked really well. The thing that we're improving upon now is the ability to do that and scale that out to millions of end-users, 10 millions of end-users. So if you think about that inherent speed of developing an application, using a platform like OutSystems, we're taking that same concept and rolling that into an internet scale application, hosting architecture. So any developer that uses OutSystems, basically like it would be comparable to a traditional development team that has application architects, cloud architects, security engineers, database engineers, a whole team of very smart individuals that generally the biggest technology companies in the world can put together. Most companies can't do that, you don't have access to that type of skillset. And so we're providing that with Project Neo, which is what we're announcing today in our, at our user conference and customer conference. Is this brand new as far as platform that allows you to build these applications at scale. And this is initially built on AWS using all the great AWS technologies. If you look at what AWS has done and provided to developers today, it's amazing. It is absolutely amazing. The amount of technologies that you can leverage. It's also daunting because as a traditional developer, you have to go in and choose what do you do? It's like, there's just massive cognitive load. As far as upfront when you go in to design an application. What's up in messaging, what's up at data store, well, how do I host my application? What type of network as far as security do I use? We're taking all that heavy lifting, all that undifferentiated heavy lifting off of the developers, putting it into the Project Neo platform. Allowing a single developer or a small group of developers to actually leverage that best in class architecture on AWS today. >> So when you're talking to developers, what are some of the things that you describe as the unique differentiators of Project Neo? It sounds like this was really apt and apt time for change. But when you're talking to those folks, what do you say you know, one, two three, these are the things that make Project Neo unique. >> Yeah, so the first is don't worry about the application architecture. Like I mentioned when you go in, the idea, the concept of that application and what it means to deliver some value, whether it's into a business or a hobby or whatever. I mean, however you're developing application, you're doing it for a reason. You want that value to come out as quick as possible. You want that experience. And so that first thing is, you don't have to worry about the architecture anymore. So in the past you'd have to think about if it's a very large application, it's millions and millions of end-users. How do you structure that? How do you put it together? That concern is removed from you in that process. The other thing is we solve the problem of software disintegration. So with traditional development, when you develop an application and you get it into the hands of end users it immediately starts to disintegrate. So there will be bugs that will appear. There will be as far as security flaws that will come up services that you use will become deprecated. We'll swap out cloud services by AWS or Azure or Google. swap out cloud services with different services behind the scenes. Version, there'll be new versions of those that is software disintegration. As soon as you develop software today and all of these beautiful cloud services that you use and components. Something will become outdated almost by the time you release it. A lot of times with software development projects, it literally is you start with some version or some component before you can get that out in a traditional mode, something becomes outdated. We solved that issue. What I like to call software disintegration. We, as far as OutSystems, ensure we invest in that platform. And so when we may need to change out those components, those services, those versions fix is for security flaws, fixed bugs, we do that and it's seamless. And so your application, you do not have to rewrite your application. You do not have to go through that process as a tradition, as a developer on OutSystems like you would, as your traditional developer. We solve that software disintegration issue. So it's very empowering to developers to not have to worry about that. There are many, you look at the numbers today about how much is invested in innovation versus maintenance. A lot of companies start out at 70% innovation, 30% as far as maintenance, and then overtime that flips. And you'll get to 30% of your time spent on innovations development, 70% maintenance, that burden, we remove that burden. >> Those were some really powerful statements Patrick that you made and I really liked the way that you described software disintegration. I've actually never heard that term before. And it kind of reminded me of when you buy a brand new car, you drive it off the lot, the value goes down right away then before you even get things out. And on the consumer side, we know that as soon as we buy the newest iPhone, the next one's going to be out, or there's some part of it, that's going to be outdated. In terms of technical debt, I was reading a stat that technical debt is expected to reach in costs of businesses, 5 trillion, US dollars over the next 10 years. How does OutSystems help customers address the challenges with technical debt and even reduce it? >> Yeah, I mean if you think about in the kind of the truest sense of technical debt, it's a decision that you make in the development process to basically load up the future with some work that you don't want to do right now. And so we're solving that issue where not only, you don't even have to make that decision. So you can go back to that concept of removing that cognitive load of, do I get the software out right now or do I get it out in the right way? And that's really what technical debt, technical debt is saying I need to get it out now. And there are some things I want to do that it'd be better if I did them now, but I'm going to go ahead and push that out into the future. You don't have to do that today with us. And so what happens with OutSystems is we invest in that platform. And this is hard. I mean, this is not an easy thing to do. This is why we have some of the best and brightest engineers focusing on this process at the heart of this, not to get too technical, but the heart of this is what we call the true change engine within our platform. We go through and we look at all of the changes that you need to make. So you think of that concept of technical debt of like, ah, I want to get this in the hands of end users, but I don't want to invest in the time to do something right. It's always done right, as far as with the OutSystems platform. So we take that, we look at the intent of your change. So it's like a process where you tell us the intent. When you as a application developer, you're designing an application, you tell us the intent of the application is to look and feel. It could be some business processes this could be some integrations. We determine what's the best way to do that and then once again, from a software disintegration standpoint, we continue to invest in all the right ways to do that the best way possible. And so, I mean, we have customers that have written applications that's 10, 15 years ago. They're still using our platform with those same applications they've added to them, but they have not rewritten those applications. And so if you think about the normal traditional development process, the technical debt incurred over that type of lifetime would be enormous. With us there's no technical debt. They're still using the same application they've simply added capabilities to it. We invest in that platform so they don't have to. >> So big business outcomes down, obviously from a developer productivity perspective, but from the company wide perspective, the ability to eliminate technical debt, some significant opportunities there. Talk to me about the existing OutSystems customers. When are they going to be able to take advantage of this? What is the migration or upgrade path that they can take and when? >> Yeah and so it is very important to me and the team as far as OutSystems to be able to integrate, to innovate as far as for customers, without disrupting customers. And we've probably all been through this path of great new technology is awesome. But then to actually utilize that technology when you're a current customer, it creates pain. And so we've invested heavily in making sure that the process is pain-free. So you can use Project Neo. So we are announcing it as in, it was in public preview as far as now, and then we will release it from GA as far as in the first quarter of next year. So over this timeframe, you'll be able to get in and try it out and all that. Continue to use your current version, which is OutSystems 11. So what we affectionately call O-11, as far as Allo systems. The Allo systems 11 version continue to use, and you can continue to use that today side-by-side and coexistence with the Project Neo. And Project Neo is a code name. So we will have an official product name as for as at launch but it's our affectionate it's kind of a unofficial mascot as Neo. So we call the Project Neo is a little bit of a fun name and you can use it side by side and then in the future, you'll be able to migrate applications over. Or you can just continue to co-exist. I mean, we see a very long lifetime for OutSystems 11, it's a different platform, different technology behind the scenes. Project Neo's Kubernetes-base Linux containers. Based once again, on the ability, we went in with the gist and looked at it and said, re-architect, re-imagine, how would you do this if you had the best and brightest as far as engineers, architects, we have, which we do. Various market and those people and we did that. And so we did that for our customers. And so Neo is that OutSystems 11 still a great choice. If you have applications on it, you can use it. And we have, we anticipate the customers will actually side by side develop on both in which we have some customers in preview today. And that's the process that they have. They will develop on 11, they will develop on Neo and they will continue to do that. And there's no, we are dedicated to making sure that there's no disruption and no pain in that process. And then when customers are ready to migrate over, if that's what they choose, we'll help them migrate over. >> You make it sound easy. And I was wondering if Project Neo had anything to do with the new matrix movie I just saw the trailer for it the other day, I wonder if this is related. >> It was a happy coincidence. It is not easy let me, let me be clear. It is something we have been working on for three years and really this last year really kicked into high gear. And a lot of behind the scenes work, obviously for us, but once again, that's our value proposition. It's we do the hard work. So developers and the customers don't have to do that hard work. But no relations to Neo, I love, I do love the matrix movies. So it's a nice coincidence. (Lisa laughs) >> It is a nice coincidence. Last question, Patrick, for you, as we wrap up the calendar year 2021, we heading into 2022. I think we're all very hopeful that 2022 will be a better year than the last two. What are some of the things that you see as absolutely critical for enterprises? What are they most concerned about right now? >> Yeah, I think it's, look I mean, it's obviously it has been a crazy a couple of years. And if you think about what enterprises want, I mean, they want to provide a great experiences for their customers, a great experience for their employees. Once again, digital transformation, where you don't even kind of talk about digital transformation more because we're in it. And I think that customers need to make sure that the experiences they provide these digital experiences are the best possible experiences. And these are differentiators. These are differentiators for employees. These are differentiators for customers. I believe that software is one of the big differentiators for businesses today and going forward. And that will continue to be so where businesses may be invested in supply chains, invested in certain types of technologies. Business will continue to invest in software because software is that differentiator. And if you look at where we fit, you can go, you can go buy, some great set of software, my software as a service off the shelf. In the end, you're just like every other business you bought the same thing that everybody else had bought. You can go the traditional development route, where you invest a bunch of money, it's a high risk, takes a long time. And once again, you may not get what you want. We believe what is most important to businesses. Get that unique software that fits like a glove that is great for employees, it's great for their customers. And it is a unique differentiator for them. And I really see that in 2022, that's going to be big and going forward. They're the legs for that type of investment that companies make and they return on that is huge. >> I agree with you on that in terms of software as a differentiator. Now we're seeing every company become a software company in every industry these days to be, first to survive in the last 20 months and now to be competitive, it's really kind of a must have. So, Patrick thank you for joining me on the program, talking about Project Neo, GA in the first quarter of calendar year 22. Exciting stuff we appreciate your feedback and your insights and congratulations on Project Neo. >> Thanks, Lisa, appreciate it. >> For Patrick Jean, I'm Lisa Martin, and you're watching theCUBEs continuous coverage of re:Invent 2021. (Outro music)

>>from around the globe. It's the cue with digital coverage of out systems. Next step 2020. Brought to you by Out systems. Welcome back. I'm stupid, man. This is the Cube at Out systems. Next step course. When we gather at the events, one of the big things to talk about is the community and out system show is no exception. Lots of developers, lots of community engagement. A Z they're building and sharing what they have. So to dig into that topic, happen it. Welcome to the program. 10 Lopez. She is the senior director of Community and Advocacy without systems. Again. Thank you so much for joining us. >>Hi. Thanks so much for having me. >>Well, Jen, you know they're there. So much discussion in the industry right now is like OK, what does that the current moment in time with the global pandemic mean for events? What does it mean for communities? The term I've heard used so much is, you know, how do we bring ourselves together even while we're apart? But if you could, you know, give us You know what does the community on help systems look like? You know, you've had this event before. If this was 2019 you know, what did the community activity in the community engagement looked like? >>Yeah, we're definitely in a different world right now, right? So in 2019 gathering the community together, you know, whether it was at at Max step or another in person events that we often had. Um you know that that is such a huge part of building community is getting people together and being able to have those conversations. And, um, sometimes it's just ah, whether it's meeting at, you know, you're getting some coffee and you meet someone. All of those in person things, um, are hard to do online. But we're really working hard this year at, you know, finding those ways to connect in a bunch of different ways with the community. Um, we have our regular technical talks and that sort of thing that we're doing. But we also have a chat where you can come in tow and chat with other community members. We're gonna have ah, 24. You know, we have this 24 hour zoom going on. So you could you know, we're fine trying to find his many ways as possible. Teoh sort of at least get those conversations and have the ability for the community to connect with each other. >>I'm wondering if you can, you know, people look at communities and especially in the developer community There's so many different pieces of that. Uh, when I talkto Gonzalo he was talking about how do we enable the next? You know, 10 million developers? When I talk to help communities, it looks like the app Dev is obviously a big piece of ah of what you're addressing. But you characterize if you could And if you have any staff but loved, understand, you know, the community, the growth of community. You know where the engagement activity is. >>Yeah, thanks. So the community growth of the out systems community has been phenomenal. Um, last year we saw are just for this year with on 90% growth since last year. Uh, we have 22,000 developers on a monthly basis who are actively doing things in the community. Um, that's anywhere from between building APS and asking questions in the forum and, um, using downloading forge components which are reusable APS attending user groups. There's all these things right. We have this activity level that we've seen that has just been through the roof. And, um, Cove in for the community has actually been, You know, we've seen a huge birth specifically march in April, we saw a great increase in new members coming on. And then what happened is our other members jumped in answering way more questions than we've ever had in the forums offering to help in different ways so that between the increase in gross, the growth and increasing activity, uh, the community itself has jumped in to really help out other people. >>Well, if you look at the development community and the tools they use and how they engage there, really, the work from home, you know, movement probably hit them a little bit less that than the average knowledge worker because they're used to being online there. Used to engaging in these environments. Often it is a distributed community, so it sounds like it makes sense. What what else? From a covert standpoint, You know, I've talked to some of the out systems customers and the ability it baked into the five former, something that they're take advantage of. Do you have any interesting stories around. You know how the community is rallying, you know, specifically with Kobe going on? >>Yeah. So, actually, that that Brown was a huge thing for us. We had at both internal and external. We were getting a lot of folks coming to us and saying, you know, everybody wanted to help, right? Especially in the beginning off when it kind of hit globally, everybody wanted to help. So what we did is we launched a program that we called the community the coveted 19 community response. Orban, um and we weren't quite sure exactly how people might react. But what ended up happening is we had thousands of people give ideas. And with those ideas, we had teams of people who were working on building these acts and actually launching the abs to help different communities all around the world with various issues. Whether it waas, you know, um, on an uber like up that was created to help people in a certain community, you know, find somebody who could go to the store for them. Um, there were, you know, these different acts were being created by the community. The ideas were coming from the community and people just really rallied around it because everybody wanted to help and they wanted to participate and be a part of something. And they were able to get these APS out in, you know, record time. Um, I would see other folks. Everyone was was trying to rely on technology at the time. And I would see other folks saying, Oh, you know, we had a team of five people spent 22 weeks building out. You know, our first M v p of this happened at out systems we were seeing people in, you know, two people in one week having, like, awfully blown, flushed out, being created. Um, So we were able to you not just help with the technology simply but help really quickly when it was needed right away. >>You know what? One of the themes I've been hearing a lot at the show is How do we close that? That Helen skill gap? Um, I have to imagine with your community engagements, the advocacy. You've got some visibility in tow. You know what things is out system engaged with when it comes toe educating the next generation, helping people take advantage of some of the new technologies adoption of the new AI features. It gives a little viewpoint as those changing dynamics in the community and specifically for developer. >>Yeah, I think it's it's really interesting. So, um, we have a number of programs with our between our education program and, um, low code schools and various programs where we're getting not just new developers coming in and burning out systems right away. But but actually getting developers who were coming from other programming languages who were ready to learn something new, who are like, Hey, I'm hearing a lot about, you know, uh, these these different ways to be innovative and I, you know, build an act quickly and it's still secure and stable and robust. And all of this. And so we have a lot of people on, you know, coming in in different ways. We're also really excited about a new partnership that we've just launched with women who code. We're you know, we're working really hard at going beyond just sort of those regular ways of people coming in. We want to help bring people from, you know normally, who may be underrepresented intact at the moment because we want to help bring that new generation in and that generations coming from all walks of life. And, um, you know, coming up with working on lots of different ways, Teoh, educate and and bring them in and keep them intact. >>Yeah. You know, Jen, such an important topic. I'm so glad that you brought it up, you know, diversity. Um, you know what? One of the things when when I think about we're lowering the bar. Ah, and you know what necessary skills? You have to get started to be a coder so often it's I have to have this degree. I need to understand these languages. So, you know, do do you feel this general movement is making it more accessible? Are we in a You know, what are we doing? What we doom or to be able to reach out, find some new pools of talent that can help us close this gap. And you know, then, as you said, keep them in tech. >>Yeah, and I think that will be key, is keeping them in tech. But, um, there are right now it's a It's a strange thing to say this is an opportunity, right? But with lots of people. Um, and specifically here in the US, where I am, Obviously we have a lot of folks who have lost jobs, right? People are looking for ways to get into something new. What's great about being able to learn? Ah, out systems is that that you're going to have a a different kind of job, right? You're going to have one of those jobs with an enterprise organization, um, or or or, you know, one of our partnerships. And it's going to level up your career and it totally differently. And there are, um, uh, lots of organizations right now who are also looking to find those those ways online there, like we have all these members in our community. But we're trying to get trained and Intertek and in different ways, and they're reaching out to us as well and saying, Hey, we're hearing a lot about you know, all of these innovative things out systems is doing. How do we work together? And so it's been really exciting to see that it's not just us going out and reaching out its people saying, Oh, I see these really cool things that you're doing and you know, we want to help get our members, um, learning and into this as well. >>All right, let's look a little forward. If we could, Jen, you know, tell us. You know what? What do you What do you see in the future? You know what feedback you're getting from community? What things should we be looking at? Going forward, >>Going forward. I think that, um, development is really going to be focused on on being able to be creative and innovative and finding new ways to do things. We don't have to do things the same old way anymore, right? We can, uh, build a robust application, uh, quickly and likely saw with co bid. Um, you know, we had big issues, and people were able Teoh, uh, figure out a way Thio Thio use technology to actually help fix these issues or solve a problem really quickly. And I see that very much that people it lights something in people's minds of Oh, being a developer doesn't have to just mean sitting and coding all day. It means, you know, doing really but robust things that I can do to help people and use technology in a totally different, innovative way >>Wonderful. Don't want to give you the final word when we talk about out systems bringing the community together. What do you want? People toe understand and connect with on this community? >>Thedc, um, unity itself is very generous and giving and one thing, but I have really, really loved about being a part of out systems is the community itself because they are working really hard to help bring new developers in help train them, give them mentor ship. So there's a There's a big feeling of, you know, it's not just every person out for themselves. They really want to help lift each other up. I think it's really important for, you know, feeding that technology, that new generation and that innovation that that is coming from it. All right, >>well, 10 Lopez thank you. So so much for helping us dig inside the community. Definitely looking at the engagement opportunities this week. And ah, thank you for all of the information that you share. >>Thanks, Dio appreciate it. >>Stay tuned for more cover jumps to minimum. And thank you for watching the Cube

>>from around the globe. It's the cue with digital coverage of out systems. Next Step 2020 brought to you by Out Systems. I am stupid, man. And this is the Cubes coverage of out systems. Next step 2020. And we've love when we get to be able to talk to the practitioners when we come to these events and happy to welcome to the program first time dress. Bruce Bottles. He is the digital channels director at Humana. Give a presentation this year. Also last year. The physical event. Bruce, thank you so much for joining us. Great having on the Cube >>A stew. Thanks so much for having me. It's pleasure to be here. >>All right. So Bruce Humana, a company that most people probably are familiar, you know, health care, of course, super important in general. And even more so in 2020. But if you could just set up for us a little bit, how should we think of Humana these days, your role inside the organisation on? Then we'll get into the discussion from there. >>Yeah, it's a great point, you know, because Humana is and has been going through a pretty significant transformation and one of the big reasons why I joined human about 2.5 years ago was this goal to go from Justin Insurance Company to really a full service healthcare company. So to now, now we're really bridging the gateway where we're almost half of our staff are caregivers are doctors and nurses and clinicians and the other half of us kind of run the business. Eso My role is digital channels, as you would expect, leading up efforts across humana dot com Our mobile APS go through 65 other fitness and, well, this APS as well as pharmacy business. So, uh, good question. >>Awesome. First, I tell you, one of the my favorite conversations over the last few years has been that discussion, you know, digital transformation. It was a buzzword. It gets a little bit overused. But from our standpoint, the companies that are doing it, you know, data is centrally important. We understand what we're doing. We're leveraging modern technologies and worms out there. Can you bring us back a little bit? You know, 2.5 years ago, I'm sure you're rapidly going through a whole bunch of changes, but you know what's the mandate? What are some of the key important pieces along that journey? >>Yeah, that's a great point, because I I do think digital transformation, unfortunately, is a little bit over used and, like most technology waves can be hyped. Um, you know, But the reality is for us, a Humana is that you are heart truly desires to reach our customers, and new and better ways is to meet their needs. Not only that, we know they have today, but the anticipation forecast what those needs are gonna be tomorrow and start building those solutions today for what we know they're gonna need in the near future. So what are the challenges that I saw when I came to the company a couple years ago Was just the quality in the speed and ability to react to new opportunities and unforeseen circumstances and challenges Is that ability to move fairly quickly. To me, that's one of the keys of digital transformation. Is that out? The speed and quality? >>Well, you know, Bruce absolutely. You know, in 2020 hit, the commentary we had is those those companies that have already gone down this journey, as you say, agility. Big able to react really fast, are happy that they done it in anybody that hadn't gone really started down or gone fast were like, Oh, my gosh, I need to get there fast because obviously, 2020 brought a lot of new challenges in place. I want to hold off one minute longer before talking about the specific 2020 challenges because you've got a great story there, but out system says you've been a partner with them. You spoke last year. Conference. What do you bring us back as to how they first got involved? And, you know, what was the plan? Uh, pre 2020? >>Yeah. So it's my journey has been interesting. I've been part of our systems for about six years now, actually. And ah, one of the reasons I came to Humana was the opportunity to introduce the company to a new way to this low code concept. Had used out systems to start a couple of companies prior to Humana and, ah, about 18 months ago, we actually signed the first contract at human of without systems. So you know what? We really are joined Now, is this new opportunity to move quickly to build things differently and to respond Those Like I said, those opportunities that were neighborhood didn't have before. So that's my journey without system that didn't start with Humana. But, ah, I have really enjoyed working with them over the last six years. >>Well, is, you said that ability react fast is something that's been the promise of, but forms clouds and the like wealth. 2020 20. You need to react fast. So, uh, enough set up, I guess. Why don't you tell us how cove in 19 the impact what you you and your team needed to do to kind of move fast and get toe what the internal as well as external customers we're going to need. >>Yeah, thanks for the intro up. You know it really? Let me take it back just a little bit to 2019. So in 2019 we realized that one of our top five interactions that our customers do is they come to our websites and are perhaps looking for a doctor. Uh, we're looking for a hospital or clinic or a pharmacy. And I, doctor, a dentist, etcetera. It's one of the top five interactions on our site. And what we realized is that it was a very disjointed experience. It had been grown up over years. Not uncommon to most. You have Fortune 50 companies. Ah, it was a silo. If you wanted to find a medical doctor, it was different than if you wanted to find a vision, doctor. And it was different if you wanted to find a pharmacy, etcetera s. So not only was it a different experience for customers, but there were different technical solutions. And the cost of maintaining this disparate solutions was really prohibitive to us. Innovating. So I set forth the strategy. Since I was the business owner of one of these, this capability of a dot finding a doctor. I said the roadmap and said we're going to unify them all. So that was our original challenges. To unify all of these fighters into a single provider finder. Well, that was going great. And write about the end of February. We had pharmacy Finder was the first one and then covert hit in March. And thank goodness it did, because hit then because we were ready to respond to one of the most important things our customers asked for And that is help me find a place to get a covert 19 test. We had a giant spreadsheet that the call center was trying to maintain and manage an answer those calls as they would come in and say, Hey, help me find a location to get a test. Well, if you know anything about covert testing, it changes constantly. The testing locations change constantly the type of test they have, the supplies that they have the hours of operations. So it was a daunting task, to say the least. So that's when I stood up and said, Hey, can we volunteer? Can we gather a bunch of volunteers to quickly build some solutions that will help not only the call center, but help our customers serve themselves? So that's really where this Cove in 19 test location came from. It was is out of the genesis of what we had started doing on the provider finder space. >>Yeah, I'm curious. Bruce, I know you gave a presentation here at the event, kind of walk through what you had, Bill. But if you were to look at it, how long did it take to build the covert test? finder, and you've got lots of experience without systems. If you had not already started on this back in 2019 if you had just said OK, I've got a spreadsheet I need to bring in a technology. If I started from scratch, how much longer do you think it would have taken for your team to be able to react? Oh, deploy this new solution. >>That's a great question. In one of the key victories I think we had is you, Honestly, the first challenge that he rented the spreadsheet. We solved that spreadsheet problem in a weekend. So I pulled together some volunteers. I was one of them, and we actually built the replacement for the spread she in a weekend, so that was pretty astounding. That s so the call center was grateful for that, and they quickly had a very unique solution there. But that was really just the touching point where we then took it to is building on top of this unified provider finder. We said, Well, you know, the covert night be testing locations are it s as just in assessments, just another type of provider. So with that perspective, we started building a full back office suite where we had a team of 30 to 40 analysts constant locally, looking across the United States, invalidating testing, location, information, hours of operation, calling them, making sure that they're accurate and then importing all the information into the centralized database that was out systems. Um, and then we quickly were able to build a customer experience where they could. Self search customers could go out there, do a search finding, assessing location themselves, I'd say time wide. We spent about a month building the back office and then deploying out the first version to our customers as well. Very, very rapid, very high quality. On what we've taken it further even since that first month, we're just now actually building it into and integrating it with, ah, health bought that we have developed in parallel separately. Um, but it's just illustrates the agility that we've had the flexibility to be able to take a solution that started out as hey, I want to find a doctor to quickly morphed to help me find a test location for Cove in 19. >>Yeah, it's amazing. Burst. I think back in my career, you know, very early in my career how long it would take to, you know, build the schema, build out a database and populate all the data on how many interference you need to do that to the websites to Now that that that modern app deployment 30 days, you know, that's phenomenal. From kind of full end to end. Obviously you still have some Resource is keeping things up to date. Did you have a rough swag? If you didn't hadn't already been using out systems, would this have been, you know, 23 months or is getting from from the ground up? How long does that take? >>Yeah, good question. You know, overall. Ah, eso Short answer is probably what it took us about four months using traditional methods eso instead of four months, about a month. And that's pretty consistent. What is what we have seen with all of the apse that we've built so far? Without systems, we're seeing about four times the value. I like to say four x value in that being, you know, a quarter of the cost a quarter of the time and we typically will over deliver on scope. It's not too often you can say that, you know, we made it, you know, on budget on time. But we over to alert scope. But but generally speaking, we're seeing about four x value. And I would just say coincidentally, when I was doing the startups I mentioned earlier, I would see up to 10 x value compared to traditional hand coating and large development teams in the start up environment. So smaller companies, I think, should expect to see even better than forex. >>Well, that's great. For since you have such a long history without systems, I'd love to get your take on some of the enhancements is you look at it. It's not just a platform, but they're helping give guidance to build faster. There's really, you know, ai being built in. You know, what have you seen over the years? What's exciting you these days? Anything else that you're kind of asking for, that maybe we should be looking for down on the road map. >>You know, that's one of the greatest things I really enjoy about the ancestors. Partnership is their level of investment in the platform, and they're like like I'm constantly trying to think forward in health care. What of our customers going to need tomorrow. Out Systems is doing the same saying, Hey, what is Bruce going to need to drive his digital business forward in the future? So two big things really come to mind. Number one is mobile. When Version 10 was launched, uh, I started on version nine when versus Ted would launch. It was it was lights out when it came to Mobile. It was an absolute game changer. For the first time, I didn't have to have a large IOS and Android team and a Web team and a back office team so typical I'd have four different teams when the specialties I didn't need that anymore. We could do full stack now with just about any of any developer, so that was huge. The second huge innovation is, I would say the AI you mentioned is that now that the new, um, developer productivity that you see embedded in the app suggestions the it's almost like the platform anticipates what developers need next in their daily tasks. Eso I know that's been a big help. Um, and I think the last thing that I'm looking forward to, that I know they're working on feverishly is really bringing it mawr to even a wider audience of citizen developers. So, designers, we've got a few use cases where our marketing team has worked with us in some of their marketers and designers that aren't developers at all sauce building things. And they said, Hey, you know, after the first couple of APS they designed with us, they said, I I think we can do this ourselves for some basic things. So they did. They started building some basic things. I'm really looking forward to that push out to or, you know, more business folks even further than what they had done before. >>Yeah, but persists. Such a good point. Something I've seen in the serverless community, really enabling, Aziz said. Back in the early days, it was programming you wrote lines of code coding was you pulled pieces. The discussion of low code is trying to make it even simpler and with more modern platform for more about on tools. As you said it tous ip eight things you don't need to You can even have that citizens developer, as you said, go out there. So, uh, first want to give you the final word just you know, Valuev. Seen you've been part of the out systems events in the past. What do you enjoy talking your peers about sharing your story? What? One of the things that you want to make Sure that people, if they're coming to virtual on, maybe it's their first time understand about shows like next step. >>Yeah, next step is just a fantastic event. It's like I always said it. I'm a calendar, never miss it. Disappointed, won't be ableto sit and have a meal with some of the folks in person, but we'll get through it next year. But no, I I'd say you know the sessions. Of course. You know my session. I was excited to share more detail. Ah, on how we went about creating this cove in 19 in this universal finder. So there's tons and tons and tons of sessions just like those great get great insights. Ah, and to make new contacts as well. So I would encourage folks to, you know, pick me up on Twitter, picked up on lengthen ah, and others and, you know, network, because when it comes down to it, we're all innovators, and we're all trying to solve the needs of the communities that we serve. And I believe we're better together. So thanks for having to have me >>Well, person, but we love being able to share those stories. Thank you so much for what you were able to do. Such a valuable, important thing that the community as a whole. And thank you for sharing your story on the Cube. >>Great. Thanks again for having me. Thanks to >>stay with us for watch more coverage from out systems. Next step is to Milliman, and thank you for watching the Cube.

>>from around the globe. It's the cue with digital coverage of out systems. Next step 2020 brought to you by Out systems. Welcome back. I'm stupid, man. In this, the cubes coverage of out systems next step always love when you get to come to the conference is to be able to talk to the practitioners, understand what they're doing. Give us some recommendations that they have for their peers. So happy to welcome to the program. First time guests make eye Spruce es, which is the global vice president of information technology at Colliers International coming to us from Vancouver because I thank you so much for joining us. >>Thank you for having me great to be here. >>All right, if you could just set up for us. First of all, uh, collars. If you know, how should people be thinking about all yours in 2020 and just a little bit briefly about you know, your role there. >>Okay. So, Colliers International, it's a leading commercial real estate for more in 69. 0, sorry. 68 countries. We have over 18,000 professionals around the globe. My role, it's who coordinated the information technology globally. We are very much distributor to decentralized organization. So we have technology groups all around the globe, and, ah, my role is pretty much toe set a direction. And to make sure that we were following on that direction appropriately, >>All right, so the theme we hear it the show, it's about building it. You hear from out system building fast, Build it right, build it for the future. Lots of discussion in the industry about how you need to modernize. And a lot of that is building new applications. So if you could, you know, how does that application portfolio modernization fit in your environment? What are some of the business things that are driving you to build new applications? >>Well, you know, thank you for asking me what of the things that are driving cause that that's one of the teams that I am. Keep bringing up that you have to start with. With that, you have to start with. Why? Why do you even build applications? So what's happening in my mind, or at least I can tell these days is that the environment of business environment, not necessarily the technology environment, but the business environment is changing and it's changing very fast, and we need to adapt to that. We need to adopt a new ways off, engaging with the clients, for example, or providing a service or who you know. Sometimes we just call it and say we digitize and and that's the whole, you know, digital transformation story that. But that's the reason why you need applications because applications are end of the day. What makes us more efficient in What are we doing? That's where the machine interacts with with humans right behind that you have the infrastructure and all that stuff that nobody sees. But what you see it is that application in your phone. It's application that makes you better make you more efficient on the time. That's why it's very important to be able to do it. And if you're just gonna buy everything, you're not gonna derive any competitive advantage out of it. You just gonna, you know, use what's available so more and more these days, we recognize that ability create something specific for the industry or for the organization is extremely important. So I think that's where the ability toe build this application fits for us as well >>All right. Well, you know, we've seen in general that that digitization is something that happened for a lot of years. I have to imagine 2020 has had some specific impacts. Ah, on the marketplace. You know, everybody that interested in real estate, there was probably a brief pause. But now things need to be more online. You mentioned mobile. Um, you know, I would think that being able to react fast is something that it has been driving your activity. Take us inside that. And, you know, how is 2020 impacted your activity specific? >>Well, uh, 2020 has really put Ah, you know, something which is always in my mind to the test and watch in my mind is, well, you know, the business agility that I'm talking about many times. And it's not necessarily that 2020 has forced tested. But in the first half of 2020 I don't think that the ability to create application was tested immediately. First off, all, it was tested. The ability to work in a different environment, work from home or work from somewhere else where you are not, you know, in ah exposed the Corbett 19. Um pandemic eso So that was the first part that was was tested. And, you know, I'm I'm extremely proud of our organization that for us, that was not too much off initial. I mean, our our system has has, uh, um, you know, help us do that. Transmission Very quick and very smooth as we are moving through 2020. Now it's not toe get ah, get our footing again. And you know, people are referring to this as but he would I want the new normal ease. And as we figured out what the new normally is, we start shifting again back to the application. So how are we going to do things in the new normal? What is becoming more important now that it was before, right? It's all of the sudden my ability toe capture, infrared term, our cameras has become a priority. And I'm not saying that the necessary has by just as an example thing that I haven't thought about necessarily before. Now I need to switch very fast in tow that my ability to track my ability toe, you know, Ah, let people in the office, you know, in a certain way, toe to figure out how many people are in the office. Was the density those things before Cove in There were kind of Well, that's very interesting. Yeah, we kind of have a lot off office. We can cram lots of people in their noticeable Ah, well, that's gone. So all of the sudden, those technologies that were called, you know, emerging are becoming very fast mainstream. So our ability to incorporate them it's critical. >>Wonderful, not another. We've laid out some of, ah, you know, the business drivers and some of the urgent needs. They have helped us connect the dots, your usage of out systems. Maybe if you could take us back how long you've been using it on and what that's enabled for your business, >>right? Ah, we They were doctored out systems about three years ago. And wait didn't necessarily take the the most successful pattern did the most successful Parton seems to be or taking a low code platform. All systems on this case and you start with something small and you grow from there. We actually had a mission critical system in our hands, which waas obsolete was the legacy was was a time bomb off technical debt, and we knew that we need to change this, but we didn't want to do it in a traditional way. We didn't want to create another monolithically application. Another stovepipe, um, so So? So that's when we look for something that we call the digital platform. Andi. That's how we got toe out systems. And we adopted it for that particular systems. Now the byproduct, of course we delivered on that, and it's fantastic and it's great, and we had lots of experience, and I can tell you what to do and what not to do under our duty parent, but, um, the by product off that is, that created what we were really after strategically created that capability to do things very fast, right? And it's not only your doing it fast, but your chances to do it passed and well has increased dramatically. You know, from the perspective that chances are that you're gonna be on brand, which it's a struggle, let me tell you that with the developers don't really care about the colors, Ana, and that it's you know it's following your security patterns and whatever it is in there. All this have have increased dramatically by putting, uh, putting it in this platform. >>So when you say, do it fast and do it well, did you have metrics internally as how you measure that? Is it you know, cycle time, time to market, you know, some, you know, percentage of quality of code. How do you kind of measure that it is demonstrably better today faster today? More agile today. Then what you were doing? Four. >>So, first off, all of those are KP eyes that a non technical organization probably has not have in the past. Okay, so that s so this is something that Microsoft will measure or Intuit will measure and stuff like that, which is just a background, because it's interesting. Now we start measuring, but for us, it's very simple. So how many new applications did we develop in the last? I don't know. 10 years. Um, zero way. We had a moment 10 years ago. We developed a lot of things and and and then we were kept going and going. And that's how they get legacy. How many did we developed in the last 18 months? Well, about 19 I think, keep growing, so I think we just start to learn what the KP I should be. I really I don't know any is 19. Good is it is too much. Maybe you should be. Should be only eight eso we learn on the escapee eyes one one KP Eido or one measurement is not the KP I cause I didn't really make it okay. Yeah, yeah, but it's a measurement. In the past, an application was build and then, you know, sporadically. There were efforts to bring it up to date and the business behavior Aziz relates toe that was very specific. They tried to cram everything during the project time because the chances that nothing will ever be reviewed in the next couple of years were close to zero. Well, let me tell you how the new application that we created the core one that mission critical. It's on a release cycle, a weekly release cycle. This is unheard for our organization. And I were gonna I gonna tell you that in my 25 years off managing applications and I d in various industries, non technical industry, that that that's that's critical. I will never achieved with anything that kind off pace. So do I have clear Kate guys to tell you this is way better? No, but I can tell you that there are a whole bunch of things that are emerging, and we will start measuring in in the future that are clear indication that we're in a better sport than we used to be. >>Well, well, well, right it is. It's the measurements that are important to your business That matters. Obviously clear. You've unlocked new capabilities that you didn't have before. It was zero applications on. Now, as you said, you know that why you're delivering value to the business. You understand what it takes to do that. And you know that that general discussion of like, well, everybody is becoming a software business? I think you've laid out some of that, at least in where you are. Right out. You know why that's important for your business. So you mentioned Ah, some things you've learned along the way. I'd love to hear you know, your journey of three years Looking back, certain best practices you've seen out there share a little bit of that wisdom that you've gained, You know? What would you recommend? Piers that are starting down this journey or maybe need to take a new look at how they look at their software development. What would you share with them? >>Yes. Ah, we learned quite a bit. Um, the fact that to get the tool in place, it's ah, it's it's it's really really not. Not enoughto see the benefit way grossly underestimated the resistance to change inside off. I t um so you know, if I would give somebody advice, I would say, Well, if you are, you know, in a typical North American organization where you know, people matters and what they think and what they do matters and you can't just be a, you know, some kind off. Ah, dictatorial. Ah, leader. Then then then you have to give yourself time because people need toe, understand the benefits or off the platform, the local It is not something that is necessarily yeah, immediately embraced by even the brightest devil locals. And unfortunately, we we've seen great people. Um, you know, leaving us eventually because they simply didn't give them by in the concept. So you know anybody who starts this journey, I would say, Do not underestimate the psychological change that needs to happen in order to become efficient at at this. Then it's another the interesting one. There are many, but, you know, the dough. So I think are interesting. You gonna be for the first time, probably way faster than the business respond. And this is something that again I haven't seen in the past in my career where you know you're developing things and you have a question. You come something and you ask the business. So how should this be? It's, you know, it's left or its right and the question is straightforward and the businesses scratching their head is like, Well, like, this is not a simple answer. I really I don't know. I need to think about it. And the business may think a couple of days, and in that time you are You're not putting if you don't plan properly. So So you are so fast that if you're not managing your pipeline and you are not accounting for the fact that the business won't be able to keep up with you and will need to have time to think about certain things, you're gonna have a whole bunch off, uh, off death times now you can fill them with all kinds of things. You can pay technical debt that you build in your previous are jive cycles. It's a whole bunch of things to do, but you need to account for that. And I said, I never seen it before and I always I was trying to catch up to the business. It was for the first time now they actually see this thing reversed and it's uncomfortable. It's It's I saw this becoming uncomfortable actually, for the business. It was perceived that only you are putting a lot of pressure on me right now. Well, yeah, but you know what? If you want me to be fast, we need to respond fast. It's kind of dynamic that is changing in a very interesting way. Eventually, I would say, and it doesn't take a long time, but eventually everybody is generally happier. I was just talking that of day with, uh, you know, our V p off, not American account. It was one of the stakeholders on this product and she said, Well, yeah, this is This is so much. Ah, better than anything we waded before from from an experience perspective, not necessarily from a you know, feature by feature, which is also extremely before rental, and much better, but yes, two things. Psychology off the developer and the velocity that the business can provide. When you're developing on this higher productivity platform, it's something you keep in mind. >>Well, maybe high it makes me It makes me laugh a little bit. I think back to in the early days of cloud computing rolling out there was that discussion that, you know, maybe I t won't matter anymore. It will just become a utility. And the discussion mostly iose that I talk to people in i t is that I t needs to be responsive to the business and actually can reach a point where it is a major driver for that. And so that agility, that speed that you talk about is helping Teoh, you know, really bring things together. Ah, and help. Have you know you need to have that common vision. Um great. What? Want to give you final word? You've worked without systems number year. We're watching them as they you know, Anting, their products Seymour machine learning and ai baked into it for those coming, you know, watching the next step show. You know what final words would you give them from from this event? >>Oh, I wasn't prepared for that. And I guess keep up, the good work would be done. The thing we we made a big bet on ah, on out systems. We want to see them growing. Want to see the community off, local developers growing. I want toe. And I don't know if I want to tell out system, but I want to tell the development community you still need to be a great developer to be able to deliver great applications in low court. It doesn't diminish anybody's value in the in the market is just a different way was gonna make the developer community more productive. We're automating our own tools. That's a normal way to go. And I think out system is doing a fantastic job. Ah, ah dot And I'm looking forward, though, to see growing. I think the next iteration I'm off this thing. I want to see a little bit more off the You know, in our case, I hope to see a little bit more off the citizen developer, uh, coming, coming out and and continue to enhance this. This Ah, Julia tedious. This flexibility, the ability to create and off the day. You know, it's all about competitive advantage for the organization we're in. It's you can part a phrase you and say I t really doesn't matter but creating value out of the technology. It's it's really what matters, and and that's everybody should get the ice on that. >>May I really appreciate you sharing your perspective? Thanks so much >>for joining us. Well, thank you for having me >>stay tuned for more coverage from out systems. Next, step on spew minimum and thank you for watching the Cube.

>> (narrator) From around the globe it's "theCUBE." With digital coverage of OutSystems-NextStep 2020. Brought to you by OutSystems. >> I'm Stu Miniman. And welcome back to thecubes coverage of OutSystems-NextStep. Of course, one of the items that we've been talking a lot in the industry is about how artificial intelligence machine learning are helping people as we go beyond what really human scale can do. And we need to be able to do things more machine scale to help us really dig into this topic. Happy to welcome to the program. First time, guest Antonio Alegria. He is the head of artificial intelligence at OutSystems. Antonio. Thanks so much for joining us. >> Thank you Stu. I'm really happy to be here and really talk a little bit about what we're doing at OutSystems to help our customers and how we're leveraging AI to get to those goals. >> ( Stu) Wonderful. So I saw ahead of the event, a short video that you did and talked about extreme agility with no limits. So, you know, before we dig into the product itself, maybe if you could just, you know, how should we be thinking about AI? You know, there's a broad spectrum, is that, you know, machine learning that there's various components in there, you listened to the big analyst firms, you know, the journey it's big steps and something that is pretty broad. So when we're talking about AI, you know, what does that mean to you? What does that mean to your customers? >> Yeah, so AI OutSystems really speaks to the vision and the core strategy we have for our product. Which is, you know, if you saw the keynote, you know, we talk about, you know, really enabling every company, even those that, you know, that had existed for decades, perhaps have a lot of legacy to become, you know, leading elite cloud software development companies. And really can develop digital solutions at scale really easily. But one thing we see, and then this is a big statistic. One of the things that limits CIO's the most, nowadays is really the lack of talent. Lack of, you know, engineering and software engineering, you know, ability and people that can do that. And there's a statistic that was reported by wall street journal. I saw it recently, perhaps last year. That said that according to federal jobs data in the U S, by the end of 2020 there would be about a million unfilled IET and software development jobs available, right. So there's this big problem. All of these companies really need to scale, really need to invest in digital systems. And so our belief at OutSystems, we've already been abstracting, and we've been focusing on automating as much as possible, the software development tools and applications that we use. We've already seen amazing stories of people coming from different backgrounds, really starting to develop really leading edge applications. And we want to take this to the next level. And we believe that artificial intelligence with machine learning, but also with other AI technologies that we're also taking advantage of can really help us get to a next stage of productivity. So from 10 X productivity to 100 X productivity. And we believe AI plays a role in three ways. We believe AI by learning from all of this data that we now collect in terms of, you know, the projects that are being developed. We are essentially trying to embed a tech lead, so to speak inside a product. And a tech lead that can help developers by guiding them, guiding the most junior ones, by automating some of the boring, repetitive tasks where by validating their work, making sure that they are using the best practices, making sure that it helps them as they scale to refactor their code to automatically design their architectures, things like that. >> (Stu) Wonderful . Antonio. Goncalo, stated it quite clearly in the interview that I had with him, it's really about enabling that next, you know, 10 million developers. We know that there is that skill gap, as you said, and you know, everybody right now, how can I do more? How can I react faster? So that's where, you know, the machine learning artificial intelligence should be able to help. So bring us inside. I know the platform itself has had, you know, guidance and the whole movement. You know, what we used to call low code. Was about simplifying things and allowing people to, you know, build faster. So bring us inside the product, you know, what are the enhancements? What are the new pieces? Some of the key items. >> Yeah. So one interesting thing, and I think one thing that I think OutSystems is really proud of being able to achieve is, if you look at how OutSystems has been using a AI within the platform. We started with introducing AI assistance within the, our software development environment, service studio, right? And so this capability we've been iterating it a lot. We've been evolving it, and now it's really able to accelerate significantly and guide novices, but also help pros dealing through the software development process and coding. By essentially.... and trying to infer and understanding their context and trying to infer their intent, and then automating the steps afterwards. And we do this by suggesting you the most likely, let's say function or code piece that you will need. But then at the next step, which we're introducing this year even better, which is we're trying to autofill most of the, let's say the variables and all of that, and the data flow that you need to collect. And so you get a very delightful, frictionless experience as you are coding. So you're closer to the business value even more than before. Now, this was just the first step. What you're seeing now and what we're announcing, and we're showing at this next step that we showed at the keynote, is that we are trying to fuse.... starting to fuse AI across the OutSystems products and across the software development life cycle. So we took this core technology that we use to guide developers and assist and automate their work. And we use the same capability to help developers, tech leads and architects to analyze the code. Learning from the bad patterns that exist learning from and receiving runtime information about crashes and performance. And inside the product that we call Architecture Dashboard. We're really able to give recommendations to these architects and tech leads, where should they evolve and improve their code. And we're using AI, refusing AI in this product in two very specific ways. Now that we are releasing today. Which is one, is to automatically collect and design and define the architecture. So we call this automated architecture discovery. So if you have a very large factory, you can imagine, you know, have lots of different modules, lots of different applications. And if you need to go and manually have to label everything. So this is a front end, this is a backend. That would take a lot of time. So we use machine learning. Learning from what architects have already done in the past and classifying their architecture. And we can map out your architecture completely, automatically, which is really powerful. Then we also use our AI engine to analyze your factory. And we can detect the best opportunities for refactoring. So refactoring is one of the top problems and the top smells and technical debt problems that large factories have, right? So we can completely identify and pinpoint what are these opportunities for refactoring and we guide you through it. We tell you, okay, all of these hundreds of functions and logic patterns that we see in your code, you could de-refactor this into a single function, and you can save a lots and lots of code. Because you know, the best code, the fastest code, the easiest to maintain is the code you don't write, you don't have. So we're trying to really eliminate crack from these factories, with these capabilities. >> (Stu) Well. >> It's fascinating. You're absolutely right. I'm, curious, you know, I think back to some of the earliest interactions I had with things, they'd give you guidance, you know, spellcheckers, grammar check. How much does the AI that you work on, does it learn what's specific for my organization and my preferences? Is there any community learning over time? Because there are industry best practices out there that are super valuable, but, you know, we saw in the SAS wave. When I can customize things myself, we learn over time. So how does that play into kind of today in the roadmap for AI that you're building? >> That's a great question. So our AI, let's say technology that we used, it actually uses two different, big kinds of AI. So we use machine learning definitely to learn from the community, what are the best practices and what are the most common patterns that people use. So we use that to guide developers, but also to validate and analyze their code. But then we also use automated reasoning. So this is more logic based, reasoning based AI. And we pair these two technologies to really create a system that is able to learn from data, but also be able to reason it at a higher order, about what are good practices and kind of reach conclusions from there and learn new things from there. Now we started by applying these technologies to more of the community data and kind of standard best practices, but our vision is to more and more start learning specifically, and allowing tech leads and architects even, in the future to tailor these engines of AI, perhaps to suggest these are the best practices for my factory. These patterns perhaps are good best practices in general. But in my factory, I do not want to use them because I have some specificities for compliance or something like that. And our vision is that, architects and tech leads can just provide just a few examples of what they like and what they don't like. And the engine just automatically learns and gets tailored to their own environment. >> (Stu) Oh, important that you're able to, you know, have the customers move things forward to the direction that makes sense on their end. I'm also curious, you talk about, you know, what partnerships OutSystems has out there, you know, being able to tie into things like what the public cloud is doing, lots of industry collaboration. So, you know, how does health systems fit into the kind of the broader AI ecosystem? >> Yeah. So one thing I did not mention, and to your point is, so we have kind of two complimentary visions and strategies for AI. So one of them is we really want to improve our own product, improve the automation in the product in the abstraction by using AI, together with great user experience and the best programming language for software automation. Right? So that's one, that's what we generally call AI assistant development and infusing AI across the software development life cycle. The other one is we also believe that, you know, true elite cloud software companies that create frictionless experiences, one of the things that they use to really be super competitive and create these frictionless experiences is that they can themselves use AI and machine learning. To automate processes, create really, really delightful experiences. So we're also investing and we've shown. And we're launching and announcing at NextStep. We've just shown this at the keynote. One tool that we call the machine learning builder, ML builder. So this essentially speaks to the fact that, you know, a lot of companies do not have access to data science talent. They really struggle to adopt machine learning. Like just one out of 10 companies are able to go and put AI in production. So we're essentially abstracting also that. We're also increasing the productivity for customers to implement AI and machine learning. We use partners behind the scenes and cloud providers for the core technology, with automated machine learning and all of that. But we abstract all of the experience. So developers can essentially just pick the data they have already inside the AI systems platform. And they want to just select. I want to train this machine learning model to predict this field, just click, click, click. And it runs dozens of experiments, selects the best algorithms transforms the data for you without you needing to have a lot of data science experience. And then you can just drag and drop into platform. Integrate in your application. And you're good to go. >> (Stu) Well. Sounds, you know, phenomenal. You mentioned data scientists. We talked about that the skill gap. Do you have any statistics, you know, is this helping people, you know, hire faster, lower the bar to entry for people to get on board, you know, increase productivity, what kind of hero numbers do your customers typically, you know, how do they measure success? >> Yeah. So we know that in, for machine learning adoption at companies, we know that, sorry, this is one of the top challenges that they have, right? So companies do not, it's not only that they do not have the expertise to implement machine learning in their products and their applications. They don't even have a good understanding of what are the use cases in order the technology opportunities for them to apply. Right? So this has been listed by lots of different surveys that this is the top problem. These are the two of the top problems that companies have to adopt AI's. Access to skill, data science skill, and understanding of the use case. And that's exactly what we're trying to kind of package up in a very easy to use product, where you can see the use cases you have available. You just select your data. You just click train, you do not need to know the nitty gritty details. And for us, a measure of success is that we've seen customers that are starting to experiment with ML builder. Is that in just a day or a few days, they can iterate over several machine learning models and put them in production. We have customers that have, you know, no machine learning models in production ever. And they just now have two. And they're starting to automate processes. They're starting to innovate with business. And that for us is we've seen as kind of the measure of success for businesses. Initially, what they want to do is they want to do POC's, and they want to experiment and they want to get to production, start getting to field for it and iterate. >> (Stu) From a product standpoint, is the AI just infused in, or there are additional licensing, you know, how do customers, you know, take advantage of it? What's the impact on that from the relationships without systems? >> Yeah. So for AI and machine learning that is fused into our product. And for automation, validation and guidance, there's, you know, no extra charges, just part of the product is what we believe is kind of a core building block in a core service for everything we do in our product. For machine learning services and components that customers can use to.... in their own applications. We allow you to integrate with cloud providers and the billing is done separately. And that's something that we're working towards and building great technical partnerships and exploring other avenues for deeper integration, so that developers and customers do not really have to worry about those things as well. >> (Stu) Yeah. Well, it's such a great way to really democratize the use of this technology platform that they're used to. They start doing it. What's general feedback from your customers? Do they just like, "Oh, it's there." "I started playing with it." "It's super easy, it makes it better." Are there any concerns or pushback? Have we gotten beyond that? What do you hear? Any good customer examples you can share as to general adoption? >> Yeah. So as I said, as we reduce the friction for adopting these technologies. We've seen one thing that's very interesting. So we have a few customers that are, for example, more in the logistics side of industry and vertical. And so they have a more conservative management, like they take time to adopt. They're more of a laggard in adopting these kinds of technologies. The business is more skeptical. Doesn't want to spend a lot of time playing around, right. And once they saw what they could do with a platform, they quickly did a proof of concept. They showed to the business and the business had lots of ideas. So they just started interacting a lot more with IT. Which is something we see with OutSystems platform, not just for AI machine learning, but generally in the digital transformation. Is when the IT can start really being very agile and iterating and innovating. And they start collaborating a lot with the business. And so what we see is, customers asking us for even more. So customers want more use cases to be supported like this. Customers also the ones that are more mature, that already have their centers of excellence, and they have their data scientists for example. They want to understand how they can also bring in perhaps, their use of very specialized tools. How can they integrate that into the platform so that, you know, for certain use cases, developers can very quickly train their own models, but so specialized data science teams can also bring in and developers can integrate their models easily and put them into production. Which is one of the big barriers. We see in a lot of companies, people working on year long projects, they develop the models, but they struggle to get them to production. And so we really want to focus on the whole end to end journey. Either you're building everything within the AI platform, or you're bringing it from a specialized pro tool. We want to make that whole journey frictionless and smooth. >> ( STU) Antonio, final question I have for you. Of course, this space we're seeing maturing, you know, rapid new technologies out there. Give us a little look forward. What should we be expecting to see from OutSystems or things even a little broader as you look at your partner ecosystem over kind of the next six, 12, 18 months? >> Yeah. So what... We're going to continue to see a trend, I think from the cloud service providers of democratization of the AI services. So this is during, just starting to advanced and accelerate. As these providers started packaging. It's like, what our system is also doing, starting to packaging some specific well-defined use cases. And then making the journey for training these models and deploying super simple. That's one thing that's continued to ramp up. And we're going to move from a AI services, more focused on cognitive pre-trained models, right. That, which is kind of the status quo. To custom AI models based on your data. That's kind of the trend we're going to start seeing in that, OutSystems also pushing forward. Generally from the AI and machine learning application and technology side of thing. I think one thing that we are leading on is that you know, machine learning and deep learning is definitely one of the big drivers for the innovation that we're seeing in AI. But you're start seeing more and more what is called hybrid AI. Which is taking machine learning and database artificial intelligence with more logic based automated reasoning techniques in pairing these two to really create systems that are able to operate at a really higher level. A higher cognitive level, which is what OutSystems is investing internally in terms of research and development. And with partnerships with institutions like Carnegie Mellon University, and such as that. >> (Stu) Wonderful. Antonio, who doesn't want, you know, a tech expert sitting next to them, helping get rid of some of the repetitive, boring things or challenges. Thank you so much for sharing the updates. >> (Antonio) Thank you Stu Congratulations on your progress And definitely look forward to hearing more in the future. >> (Antonio) Thank you Stu. Have a good day. >> (Stu) All right. >> Stay tuned for more from OutSystems NextStep. I'm Stu Miniman. And thank you for watching the "theCUBE."

>>from around the globe. It's the Cube with digital coverage of out systems. Next Step 2020 Brought to you by Out Systems Hi and welcome to the Cubes coverage of out systems. Next step. I'm your host stew Minimum and happy toe. Welcome back to the program. He's relatively fresh off the keynote stage. He's also a cube alum. Eso happy to welcome paella risotto. He's the founder and CEO of Out systems. Hello, Thanks so much for joining us. And thanks for having the queue. But your event >>now it's a pleasure to glad to be here. >>So you know your keynote. You know, one of the big themes we've been talking about for quite a while in the industry, of course, is the growth and importance of developers Onda, something that I heard loud and clear from what you and your team are talking about. It's really about helping companies, you know. It's move faster, it's be more agile, and it's really X. It's banding. Uh, you know, we need mawr developers. We need them to be ableto on ramp faster. Uh, on especially here in 2020. As I said, you and I spoke earlier this year at kind of the early stages of the global pandemic. Right now we know it's, you know, we can't have people slow down even when they can't go to the office, even though a lot of developers were dispersed as it is. So if you could see, you know, give us you know it did this your high level, you know, your customers, the developer community that that you're welcoming here to the show? >>No, Absolutely. I mean, we're really excited about this event is a This has gone way beyond our wildest expectations in terms of tendency and all of that. It's bean to an absolutely fantastic e mean what what we've seen. What we've seen is a growing demand from enterprises for solutions that are extremely differentiated. Um, you can actually get software. You can get digital systems out off out of the box, but there's a new increasing number of systems like portals and the work flows and applications that you actually have to infuse with your with your business process with your intellectual property with you as a business and therefore you have to build your own software. And so the the amount of software that's being built inside organizations is increasing its zits, increasing to a point where these these enterprises are facing all sorts of issues related to to to proliferation to skill. Set the fact that they cannot hire enough developers enough architect and offset cops people. I mean, the skill sets that just staggering and they heard, because they are they want to build this software, but they have a lot of difficulties in finding the tools and the skill sets. >>Yeah, it's great to come to an event like this and hear people. They're excited about building applications. They're they're they're getting into code. Um, it's been almost too easy this year, Palo to say, Uh, there's so many challenges, you know, at home everyone's fighting over bandwidth and space. Andi, there's those challenges. So, you know, we need to be able to see kind of that, that joy into what I can win. I can build things and get things done. So, you know, how are you seeing that? You know what? What feedback are you getting? Um and you know, as we said, 2020 we all know is a challenging year. >>Yeah, it's It's been a challenging gear. But it's also, you know, it's also been a near off year off opportunities. And we see that, uh, all over are we stall based on our prospect days and our partners and our community. And in general, these things events. Adult systems have a very different vibe from your typical corporate event, because one of one of the things that Z that's unique about our systems is everyone who comes to this event have built something unique. And so and it it zvehr e gratifying. When you're talking with customers and you're talking with developers, the one thing they want to talk about is how they fixed one particular, very unique problem that they face using our systems and the exchange these war stories, about how fast they were and how quickly they managed to overcome a particular challenge. Or, uh, when they got the change request from the business, that was, we need to do this in in two hours or 24 hours, whatever horrible timeline that they get and they were able to do it. It's these stories that get exchange around the next step floor in this event, and this one has been going on exactly as we've seen. The other ones which were physical events in the past. >>S O Paulo. On the keynote stage, you talked about the fact that you've now got over 1400 customers. You've got 300 partners. Uh, you're not just some, you know, New startup out system's been around for two decades. Now, talk a little bit about, you know, your growth. Some of the innovations that air that air driving customers in increasing, you know where they're coming houses. >>No, absolutely. I mean, the major major innovations that we have been doing is we we we we have been focused a lot on addressing the need for speed. I mean, the cycles of innovation have been compressing in the past years, and every year there is Ah, there is a further compression of the cycle. And so business are coming back to developers are coming back to i. T. They are some of these business. Uh, some of these business folks departments are completely autonomous in terms of what? Of building some digital systems, and all of them have this need for speed for very high productivity. And so we've bean Ah, lot of our investment has bean first and foremost in, how can we make all these folks way more productive? And we've been doing a tremendous amount of research into the anatomy of building these these applications understanding what are the the typical, most common patterns abstracting them, making them really use using a lot of ai and machine learning to create, uh, to create a almost like a a artificial bots that can help developers move quicker and create serious applications with big architectures without making mistakes. But very, very quickly, Um, and therefore, uh, when When we we provide these things extreme speed, we make sure at the same time. And this is where a lot of our innovation also comes along is ah, is this notion of building these applications right? Which is you. You have to be fast, but not at the expense of lack of security, lack of scalability, lack of availability, non observe ability. You know all these things that are that you don't really pay attention when you just want to create a nap and put some functional requirements designed something into either a nap or workflow, whatever. But when you're scaling from 20 users toe one million users. You need to make sure that you can do that. When you're exposing a portal to the external world, you need to make sure that you're not going to be attacked by hackers. Are you going to have the now service attack or at your mobile application is completely shielded and secure and cannot be penetrated. All of these things are things that are all part that cannot be at the expense of speed. And so that's what we try to do. We try to bring together the speed increasing speed, but at same time building fast building it right and making sure that as you evolve that your application is evergreen doesn't create technical debt. So build it for the future. And we focus a lot on this reason. >>Yeah, definitely heard that team loud and clear. Looking forward to actually, I've got so g your head of products toe walk through. Some of the announcement also got your head of a I in that really fascinating stuff as, uh, you know, like emails. Do they kind of, you know, start making suggestions and, you know, it feels like the tech technology is getting better. It's not like it was a few years ago where it was like I just want to turn that off because the suggestions were slowing me down rather than speeding me up, but moved faster. Um, you know, you see what I want to get to You talked about that flexibility of change, Really. One of the big challenges you know right now is that there's always new technologies. There's new opportunity. I need to move fast. So how do I make sure that I could do something today and not be, you know, locked out of that next new thing thing or be able to make a change? So how do you make sure that you, you know, you've got an architect? We said that that's now been around for decades, but, you know, meeting the needs of developers helping to bring on new developers. Um, that you make sure that you can stay, you know, always modern, if you will. >>Now that's that's a That's a fantastic question. It's a really good point. I mean, one of the trade offs of, uh, one of the easy ways of building these these type of products or platforms is you actually your visual modeling your obstructions, Uh, the things that you build so that you increase productivity in a lot of, um in a lot of scenarios. The easiest path is towards linking whatever technology you're going toe power these applications to the way you build the modeling. Um, and one of the things that that out systems as as has always done we design our platform from day one with the perspective that we knew the underlying technology. Name it. Web stacks to kubernetes toe on premise. Virtual machines to containers serverless, uh, technologies, micro application servers. All of these things we knew they were going to dramatically change in the next years. And we've been proven right in the sense that not only take underneath technology or technology that that's used to build these applications have been changing, but they've been changing faster. And the turmoil of technologies that you can build applications is accelerating at creating a huge problem for enterprises that once a certain level of stability. But they don't also want to become whole old. And so the art systems platform allows you to build your applications at the layer where we adult systems we can replace the underlying technology without you having to rewrite the application and because of our technology, you can basically just republish or we upgrade our platforms and automatically your applications will run on the next best of breed technology that's now hot and that is providing you extra scalability, extra security, extra high availability. We take care of that and we show you how we do it because we were following those type of standards. But it's really around the architectures off of the product at the same time, Ato level of the development of the modeling and a lot of these things. We make sure that there is a certain level of stability and we keep on improving it so that we can bring developers into our community. And those sets are constantly relevant as they move from customer to customer as they move from simpler applications toe highly complex ones. All the investment that they've made on our systems gets rewarded in the next 2357 years. We have a community. We have members of argument that have been with us for more than 15 years and we want to keep it that way >>well, that That's impressive. I'm curious. You know, we've We've had this discussion, I guess. How many years ago was it that now that mark injuries and said that software is eating the world? Palo eso So many companies now you're talking about, you know, building software building that application needs to be a key thing. You know, the role of I t. Just servicing the business isn't enough. I t needs to be tightly. I'd with the business and that capability of building software, doing things fast and reacting eyes so important. So what does this kind of these waves coming together? I mean, for out systems the growth of the company. And, you know, I would have to expect that some of your your newer customers look a little bit different than the ones that have been with you for 15 years. >>You know what? It z actually interesting that the problem that we solving is is a very basic, very old problem. And so it's just that what what has changed in in the recent years is that before it was acceptable for a 19 person to go to the business and say this project is going to take three years or this new report that this change that you want to put in your application is going to take a six months or three months to go into production. And today that's an unacceptable answer. Um, and so today, with these type of platforms, like out systems, this provides it provides a tremendous, uh, pleasant life for the guys who are actually developing and delivering thes digital systems. These applications, because the relationship with the business is a much more constructive one. Instead of you saying no Oh, I want this. I want this new mobile app and, uh, and someone coming back to you. Okay, give me two million and give me 12 months or 14 months to build this this app. Now you can go back and say, OK, well, that that's going to take me one week and I have off a guy ready to build that for you. That first version and they can work together with you so that we get those requirements right, because we know that the model application is going to be it. The first version we're going to produce is not going to be the one that you want And so we want to reiterate that conversation is the holy Grail of what we always wanted in the relationship between 90 and the business and now way have it with without systems. And that's the That's the alert. Now, if you look into the tens of industries, this particular type of characteristic is this dynamic between business I t and building. These things exist in every industry, and that's why our target addressable market is so huge. And that's why we're growing so fast at this point, because it's a it's a capability that everyone wants and before it just looks magic now, before it was considered impossible. And that's why people didn't ask for >>it. Paolo talking about that, that growth in that potential? What's your commentary on? You know the skill gaps out there, You know, how do we onboard Mawr developers, You know what's what's the opportunity and the challenge that you see out there just really when you talk about the future of jobs in this space? >>Well, um, what what we've seen is that, for instance, we measured we're very scientific. Adult systems about looking had the anatomy of skills and the what are the skill sets needed to build what type of systems. And it's not all or nothing thing. A lot off. People try to sometimes simplify and say there is this notion of the professional developer on the business developer or or even the cities and developer, which is a term we don't really enjoy it out systems that much. Um, but it's this very binary separation, and what we've seen in reality is that there is, ah, continuum. A spectrum of skill sets that we can pile up. And we can create and develop tools and capabilities, for instance, in the out systems platform that allow us to take an increasingly larger number of backgrounds and people to build an increasingly larger number of more complex applications. And so it z kind of a moving target. But the potential is that the shortage of computer science grads that exist today in the world on its not Onley in the Western world is it's all over Asia Latin America places where you'd consider that you have enough talent to fulfill the demand. Demand is huge compared with that supply of developers and so being able to, for instance, happening on on the stem, Um, the science majors being able to tap on social grads like architectural, uh, architect's and normal civil architects and the, uh, social engineers and and and all of that, all of those profiles we have found that we can bring them into the out systems community, and then they have them complement the sum of their natural skills with some technical skills and being able to actually produce these systems. And so we by doing that, we multiplied by 10 the pool of available resources to our to our customers and to to the enterprises want to build software. But they're facing this issue of the skills shortened. >>Oh, Paula, we We've got a great lineup for our coverage with the Cube. I've got a couple of your customers. I mentioned some of the executives. I've got your head of developer and community on there, but want to give you the final word. You know, takeaways you want. You know that the the audience out there toe have to understand about out systems today in the strategy going forward. >>Well, I think what what I wanted to say is that we've we've proven that we've been around for some time. And the reason for this is because it takes a while to build a product that's truly comprehensive and powerful enough that you can build complex, serious applications very quickly, but that are also that do not, uh, that you don't have to be facing a wall of security, of scalability and all of that. So this is a platform that takes a long time to get right. It takes a lot of input from our from our install base. Takes a lot off. Ah, lot of learnings from all the, uh, hundreds of thousands of applications and projects we've seen. But today our customers can take that benefits and move forward very, very quickly. Andi, we're going to stay around for many years to come because it's such a pleasurable job to be able to help all of these enterprises become as innovative as they can and as fast as they can. So I'm really excited about being in this position as we have today. >>Well, Paulo, really pleasure for us toe Be part of this event. Thanks so much and definitely looking forward to talking to the rest of your your team's your customer in the ecosystem. >>Thank you too. >>Stay with us for more coverage. Jumps to minimum. And thanks. As always, for watching. Thank you.

>> Narrator: From around the globe it's theCube with digital coverage of OutSystems NextStep 2020. Brought to you by OutSystems. >> Hi, I'm Jen. I'm the Head of Community and Advocacy here at OutSystems. I'm really excited about NextStep especially for our development community. We have a ton of technical talks specifically to help you learn and grow in building OutSystems applications but we also have booths to learn more about the community, to get to know various training and what certifications you can get. There may even be some free stuff. There's also a booth specifically around career. If you're looking for a job, if you're an organization that's looking to hire someone. We have a lot of information and stuff going on there. You can chat with the teams in all of these places and we will also have an ongoing 24 hour Zoom with the community to come in and have those conversations. So, join us for tons of knowledge, technical talks, and also to get to know the community a little bit. We look forward to seeing you there. (calm background music)

>> From around the globe. It's "theCUBE" with digital coverage of OutSystems NexStep 2020 brought to you by OutSystems. >> Welcome back, I'm Stu Miniman and this is CUBE's coverage of OutSystems NextStep. Really excited to have "theCUBE" at this event. Lots of excitement in the Keynote and happy to bring back one of the Keynote speakers Goncalo Gaiolas. He is the vice president of product at OutSystems. Understand the team calls him GG, Goncalo, thanks so much for joining us and good to talk you. >> Good to talk to you too, still welcome. Again thank you for having me. >> Yeah, so we're glad to be here. One of the big themes of course we've heard of the show it's built it fast, build it right, build it for the future and a theme that you hit on, something we've been hearing for years and only more so, is building new applications that software development, we need to do it faster. I hear from Microsoft Satya Nadella talks about just the massive amount of new applications that will be out there, so it's one of those things, do we have enough people? Do we have the right skillset? So take us into that developing software first, what that really means and how you're helping your customers there. >> Of course, so absolutely. So when we talk to customers, I talk to customers all the time and when they speak to me, they're essentially telling me about their business problems. And today the ability to get custom, highly customized, specific applications that fit their business have great experiences and get go to market quickly is one of the top business issues that they have. Now we know that this has historically been something that companies have tried to do but now more than so with a fast pace of change and sort of response cycles that companies have to go through this has now become an imperative. We've actually built most of our platform to accelerate the entire cycle of application, not only development, but also deployment, delivery, all the pieces that can, where we can increase productivity. That's really what we've been doing for the past 20 years or so. And we believe that we can help customers with a bunch of things from getting access to more wide diverse talent pools, to deliver great experiences that are best in class and rival the consumer experiences that people are used to. And that helped them modernize their existing legacy systems, all of that with unbeatable speed. >> It's interesting. The discussion of modernization has been around for decades, but in the old world it was like, okay, I'm going to do some migration, it's going to take a couple of years and then when I get there just don't breathe on it 'cause we think we got it mostly where we need whereas today, I mean the conversation we have, what I want to hear you and your team talking about is we know that, you know what we stand up, there's constantly going to be change, it's not the monolist anymore. Maybe speak a little bit more to that. That it's not just modernizing, it's not a destination, but it's that ongoing journey that we need to continue to have. >> Absolutely, absolutely. So we spoke about speed. Speed is often thought of as how fast you get to market but we think of speed also on how do you keep that change going at the right pace so that you react quickly and you react without sacrificing important things like the scalability or the security or the performance of the things you're trying to do. So really think about it in three ways. The speed that we do, that we provide our customers for their initial deployments without ever sacrificing the building it right part. But then above all, building for change, building for the future, allowing customers to experience new technologies but to react very, very quickly. So we obsess over having giving customers the ability to really iterate on top of what they deliver because even with a great platform such as ours, you'll never get it right the first time around in the world as we know and this year was best prove ever. The world is always constantly changing around you. So it doesn't matter if you're the first to market, if you can't change and adapt that's, in due time competitors will catch up with you and you got to really build it for it to change, build it for the future. >> Yeah, you bring up a really great point there and you kind of hinted that your platform is going through a lot of changes. Help us, bring us inside a little bit. When you know, we're going to talk more, I know about the AIP some of the new enhancements there. OutSystems now almost 20 years old, you go back a decade ago, we weren't talking digital transformation, it was like a mobile enablement, some of those other pieces there. So it's very different challenges to get day, very different skillsets, generational changes going on. So help us understand a little bit the kind of the underpinnings the evolution of OutSystems to be able to still be competitive and quite relevant today. >> Excellent. That's a great questions too, and I mean we spent the last 20 years just completely obsessing this problem of how do we help customers go fast and change fast. And to do that, we had to tackle the entire life cycle of software. So in the past few years, we've actually focused very much on helping customers deliver best in class front-end experiences. How do you build the great user interface? How do you do that in the world? What is a low-code quote unquote "Environment"? How do you do that to compete with the best and class front-end developers? That's really where we've focused very much of our last few years. At the same time, the backend of that and how those things get delivered and how more mission-critical you can get, that's been quite an investment for us. So really investing in security and scalability of the generated apps on the ability to have multiple global deployments has been a great investment. We were big on clouds. Most of our customers run on our own clouds and we invest tremendously there to give them all the options that they need. So if you think about these two things, you get very much massive mission-critical enterprise applications built on top of our platform. And if I go back, you were talking about our 20 years. If I go back five or six years ago, I remember talking to customers and customers telling us, wake me up when you can build a mobile banking system or my customer portal and today that's every week, we're building things like that in our technology. So it's really a testament of the maturity of the platform and how customers have embraced it. I would say also Stu, just to compliment and to finalize the answer. I'd say, we see a lot of focus as well on making it more accessible to people from different backgrounds. Just in the next couple of years there are going to be the world's going to need 10 million more developers. We're not going to get to those 10 million more developers if we keep using the same methodologies we've used in the past. So we invest very much on things that accelerate transitions from people coming into the professional development cycle, obviously because our platform is visual and we invest very, very much on the user experience, the developer experience, we make it easy for people to be onboarded into the platform without really ever sacrificing the power of what they can deliver. So those are the things that we've really been focusing on for the past few years. >> Yeah, we've looked back the last 10 years, there's so much discussion about starting to take advantage of the cloud, companies that were born in the cloud starting off there, we know that we are just in the early days and what we hear customers they're accelerating even more, so there needs to be that wholesale change and therefore they need tools that can help them culturally and building those new pieces. So I guess it makes sense maybe, let's bring the same inside, the platform itself, you did a great job in the Keynote walking through so maybe we can encapsulate for our audience a bit, some of the key capabilities that you have inside the platform. >> Absolutely. So you should think of our platform as divided in three great pieces to three major components. First of all, there's the entire development capabilities or we call the our development studio. That's been something we've been developing for years and adding capabilities. It's a visual environment, where developers can assemble really quickly applications that they can consume services from their enterprise or from cloud services which to your point is something we see customers struggling with. How do I make sense of the 240 services available to me on the cloud? How do I connect these together so they make business sense to me? So that's one part, and that's really the part that is typically associated with Low-code. At the same time, we're also very much focused on this deployment and runtime capability because an app is not really so useful if you've just developed it. Now, the biggest or a big part of it is actually to get it in front of customers delivered with zero downtime with many, many cycles and many, many deployments per day. So we focused very much on the delivery pipeline, if you will and our own cloud. So we've invested, we have a great partnership with AWS, we run our cost on top of AWS. We've invested on abstracting all of the complexity that you need to run an enterprise grade mission-critical system on top of such a platform and that's really the second component is all of our runtime capabilities of all of our cloud and our deployment capabilities. And finally, I would be remissive. I wouldn't talk about our entire community and our forge. Our forge is of repository in our marketplace or of applications and components. We have a very active developer community it actually grew more than 90% last year and they contribute to law to disforge. So think of the forge as a capability that allows unlimited extension of the platform powered by a very, very active community. So it's that full stack development environment that has not only our surface studio, but also our builders I forgot to mention our great builders that accelerates typical use cases like building workflows or building prototyping applications we've seen that in the keynotes. Also our cloud and our deployment capabilities very much focused that would be the second part, very much focused on that building, it writes in that mission-critical deployment. And finally our community, our ecosystem of partners and our forge that really extend the platform to do many, many different use cases that are valuable for our customers. >> I'm looking forward. I know we have Jen Lopez coming on "theCUBE" to talk more about the community. One of the things >> Excellent that we would love to see at shows like this and that builder message of course resonates quite well with the AWS Community since that was one of their big ad campaigns for a couple of years. Maybe if you could, it often helps to illustrate if you've got a customer example that speed of response. 2020 is at no shortage of companies having to move fast in directions that weren't necessarily what they were planning to at the beginning of the year. >> Absolutely, absolutely. So we've had many examples, one that we addressed in the Keynote stem cell technologies is that we're really able to respond super quickly to the COVID challenge by deploying a series of applications that serve many, many different stakeholders, and that speed was off of the essence. We've also had a campaign, some of you might've seen around LU Crowe, LU Crowe is a financial services provider in the United States. And LU Crowe has been able to respond to government stimulus packages and capabilities by creating applications specifically target for this use case, this is a highly optimized for the workflow that they were going after. So great example as well. And we've had customers all around the world, but only building their now mission-critical new completely new applications to get people back to work safely in multiple different workflows, from field services to people going to the office, but also customers have seen more ramp in their own businesses and really took those digital channels to the next level. We see telcos, large telcos using our systems to deploy new digital systems that are now very, very important because people more they have an extra incentive to use digital channels if you will. So it's been quite intensive to see all of these customers respond and react to their own specific needs in the middle of everything that's going on. That's good, I think quite good. >> Yeah, one of the trends that we've been seeing in the last couple of years is the pulling together of developers not just being, a group off on the side that you threw something over the wall to, but there was not just the DevOps Movement, but a lot of the tooling now is helping to give visibility to multiple groups. When I talked to some of your customers, when I look at the servers community, oftentime it's not a hardcore developer but that citizen developer that's getting involved or sometimes even somebody from marketing that can leverage some of those tools. Can you talk a little bit about how your platform is maybe now hitting different skills and different people inside organizations as we see this trend progress? >> Absolutely, absolutely. So we have a, I think we have a quite unique take on the entire approach to what's normally called the citizen developer. So the first thing I would tell too, is that we think we obsess over productivity of these teams. And like to your point, the composition of these teams, how they're put together is radically changing, you don't have the business team and the IT team and the operations team, everything is being managed and fusing together. So we think very much about how do we optimize for that entire life cycle and for the entire collaboration set of collaboration activities and workflows for those teams. In the keynote, you've seen it talk about the thing we called the experience builder. The experience builder is a great tool for business users, business stakeholders, and professional developers to collaborate on assembling in a very visual, very fast way, what a beautiful mobile application would look like. So we think that is the future, is to design specific capabilities that allow for these interactions for these jobs to be done, to happen in the natural context of the collaboration. We don't think, we think the citizen Developer Movement is an offshoot of not actually addressing the root cause of giving developers and IT teams the capabilities to interact, collaborate, and respond fast enough. Now at the same time, we're very much fans of a diverse set of people or backgrounds coming into the professional developer community. We have examples all around the world with people with chemical engineering backgrounds, history majors that have transitioned through training and through usage of the product that transitioned to full time developer careers. And we don't think of those as citizen developers. We just think of those as the new developers coming into the workplace, enabled by platforms and tools such as ours. That really democratized the ability for people to make application creation their own trade, their own job. So that's a very, very exciting trend. >> Well, and what I hear you saying reminds me of the conversations we've been having in general about automation, AI, and even robots worthed, today there's a lot of software robots out there. So just as most people are familiar now, if you go into email and it starts trying to complete your sentences for you or give you responses, I heard you talking your AI capabilities are starting to help along that way and therefore that could open up the window for a broader class of people to be able to lower that bar to entry to become a full developer. Is that where you see it going? I'd love to hear just a little bit more directionally the AI piece today for the next year and how comfortable are your customers with that as to partnering with the software? >> Absolutely, that's a great, great question. And we think of AI. So it's two layers actually. One is what would happen if you had a colleague that would be sitting next to you? There will be a phenomenal technical lead, somebody that would guide, automate and validate all the work that you've done, and to really remove anything that's completely boring and repetitive out of your way. That's really how we think about AI. We started this journey about two years ago if I will, about three years ago, two to three years ago at our other next step. And what we've seen over the last few years, we started very small with a thing we call the next-best-action prediction and now more and more, we've been fusing AI across the product so that every poor part of the development cycle from what developers code and the logic, the UI to how architecture is measured and technical depth is measured. Everything is infused in with the AI and we see great reception from customers because it just makes their life easier. It's removing the boring, that's the expression they use. Its just removes the boring, repetitive things and frees human creativity. It's not going to take the developer's job, it's essentially going to make them super humans if you will, so that's one way. The other thing that you've seen us talk about the Keynote is how do we make this happen for our customers customer? How we help developers deploy AI capabilities within their own applications build like a platform so that what they do and what they deliver has the same property, the same characteristics of the thing that I'm talking about. So with deploying AI, it's still hard. It's still very, very hard these days. We talk a lot about it, but it's not for the, it's hard. It has many, many problems. You've got to treat the data, create the models, deploying the models, integrate them into the applications. And we're really making sure that that cycle is compressed as well and accessible to more people. So to answer your question, we see customers, absolutely. It's probably the part of the product they get most excited when they see it in action like they saw it on the Keynote because it's just going to make their lives easier, more easier to respond faster to change. Easier to build new things and easier to have people that are just happy with their lives and with their careers. >> I know, such a good point there. I've run through people over time that sometimes they're hesitant or they're not sure, they're worried it's going to take away their job. And it's like, well, don't you think if you had an extra hour or an extra day that you would do other things that would be more fulfilling and more valuable for the business. So important that we can do that and top priority we've been hearing for the last year or so >> Absolutely >> Is to really embrace that automation, AI. All right Goncalo, I want to get, let you have the final word a lot going on at the show here. We're really excited to have a broad coverage here. Going to dig more into the AI, the community, a couple of your customers and the like so. Give us a final word from OutSystems NextStep. >> Well, I think the final word here is what we're building is just starting. We're looking forward as you said, OutSystems is about 20 years old now, we're looking forward to the next 20 years into really embracing the next 10 million developers that are coming into the workplace and to make their lives and their contributions to society, to organizations, to businesses really, really fantastic. And to have people going to meetings and say yes, and to feel like they're the roadrunner and not that to feel that they're pushing their businesses back, and we're going to be here to support our community in this transition. And to make them super human, super productive, so a hundred times more productive than with any alternative, that's our vision. And so I hope that if you're still on the fence about is this the right thing for me, that you take a closer look and really understand what it is that we can offer to actually most businesses. We're applicable to most businesses at this point in time. >> Goncalo, thank you so much. Alan Cohen, a great friend of "theCUBE" said many years ago, right we need to get people going from saying no or being slow and the answer needs to be go. >> So Goncalo. >> Yeah, absolutely. >> Thank you so much for joining us pleasure talking to you. >> Thank you too, pleasure. >> Okay, stay tuned for more coverage from OutSystems Nextstep, I'm Stu Miniman and thank you for watching "theCUBE". (bright upbeat music)

>>from around the globe. It's the cue with digital coverage of out systems. Next Step 2020 Brought to you by out systems. I'm stupid, man. And welcome back to the cubes Coverage of out systems Next step course. One of the items that we've been talking a lot in the industry is about how artificial intelligence, machine learning or helping people is. We go beyond what really human scale can do and we need to be ableto do things more machine scale. Help us really dig into this topic. Happy to welcome to the program First time guest Antonio Alegria. He is the head of artificial intelligence at out systems. Tonio, thanks so much for joining us. >>Thank you. So I'm really happy to be here and and really talk a little bit about what? We're doing it out systems to help our customers and our leverage eai to get to those goals. >>Wonderful. So I I saw ahead of the event a short video that you did and talked about extreme agility with no limits. So, you know, before we drink, dig into the product itself. Maybe if you could just how should we be thinking about a I you know, there's broad spectrum. Is that machine learning that there's various components in there? Listen to the big analyst firms. You know, the journey. It's big steps and something that that is pretty broad. So when we're talking about A I, you know, what does that mean to you? What does that mean to your customers? >>Eso So AI out systems really speaks to division and the core strategy we have for our product, which is, you know, if you saw the keynote, no, we talk about no, really enabling every company, even those that you know, that existed for decades, perhaps have a lot of legacy to become. You know, leading elite cloud software development companies and really can develop digital solutions at scale really easily. But one thing we see and then this is a big statistic. One of the things that limits limits CEOs the most nowadays is really the lack of town lack of engineering, a softer engineering, you know, ability and people that that that could do that. And there's a statistic that was reported by The Wall Street Journal. I saw it recently, perhaps last year, that said that according to federal jobs dating the U. S. By the end of 2. 2020 there would be about a million unfilled I E. T s after development jobs available. Right? So there's this big problem All of these companies really need to scale, really need to invest in digital systems and so horribly fed out systems. We've already been abstracting and we've been focusing automating as much as possible the softer development tools and applications that use. We've already seen amazing stories of people coming from different backgrounds really starting to develop, really leading edge applications. And we want to take this to the next level. And we believe that artificial intelligence with machine learning but also with other AI technologies that were also taking advantage of can really help us get to a next stage of productivity. So from 10 x productivity to 100 x productivity and we believe AI plays a rolling three ways. We believe II by learning from all of this data that we not collect in terms of, you know, projects are being developed. We're essentially trying to embed a tech lead, so to speak, inside a product and attack Lee that can help developers by guiding them got in the most junior ones by automating some of the boring, repetitive tasks were by validating their work. Making sure that they're using the best practice is making sure that it helps them as they scale to re factor on their code to automatically designed architectures. Things like that >>Wonderful. Antonio Gonzalo stated it quite clearly in the interview that I had with him. It's really about enabling that next you know, 10 million developers. We know that there is that skill gap, as you said, and you know everybody right now how can I do more? How can I react faster? Eso that's where you know, the machine learning artificial intelligence should be able to help. So bring us inside. I know the platform itself has had, you know, guidance and and the whole movement. You know, what we used to call low code was about simplifying things and allowing people to, you know, build faster. So bring us inside the product. You know what? The enhancements? One of the new pieces. Some of the key key items, >>Yes, So 11 interesting thing. And I think one thing that I think out system is really proud of being able to achieve is if you look at how out system has been using a AI within the platform. We started with introducing AI assistance within the Our Software Development Environment Service studio. Right? And so this capability, we've been generating it a lot. We've been evolving it, and now it's really able to accelerate significantly and guide novices, but also help pros dealing through software development process and coding by essentially trying to infer understanding their context and trying to infer their intent and then automating the steps afterwards. And we do this by suggesting you the most likely let's say function or or code p sexual one you need. But then, at the next step, which we're introducing this year, even better, which is we're trying to auto fill most of them. Let's see the variables and all of that in the data flow that you need to collect. And so you get a very delightful frictionless experience as you are coating, so you're closer to the business value even more than before. Now this is the This was just the first step, what you're seeing now and what we're announcing, and we're showing up at this next step that we show that the keynote is that we're trying to fuse starting to fuse AI across the out systems products and across this after development life cycle. So he took this core technology that we used to guide developers and assistant automate their work. Um, and we use the same capability to help developers. Tech leads an architect's to analyze the code, learning from the bad patterns that exist, learning from and receiving runtime information about crashes and performance and inside the product recall architecture, dashboard were really able to give recommendations to these architects and tech leads. Where should they evolve and improve their code? And we're using AI refusing AI in this product into very specific ways. Now that we're releasing today, which is one is to automatically collect and design and defined the architecture. So we call this automated architecture discovery. So if you have a very large factory, you can imagine, you know have lots of different modules, lots of different applications, and if you need to go and manually have to label everything so this is ah, front, and this is the back end. That would take a lot of time. So we use machine learning, learning from what architects have already done in the past, classifying their architecture. And we can map out your architecture completely automatically, which is really powerful. Then we also use our AI engine to analyze your factory and weaken detect the best opportunities for re factoring. Sorry. Factoring is one of the top problems in the top smells and technical depth problems that large factories have. Right, So we can completely identify and pinpoint. What are these opportunities for re factory and we guide you through it, which held you okay, all of these hundreds of functions and logic patterns that we see in your code Could you re factor this into a single function and you can save a lots and lots of code because, as you know, the best code the fastest coast easiest to maintain is the Cody. Don't ride. You don't have. So we're trying to really eliminate Kurt from these factories with these kids ability. >>Well, it's fascinating. You're absolutely right. I'm curious. You know, I think back to some of the earliest interactions I had with things that give you guys spell checkers. Grammar check. How much does the AI that you work on. Does it learn what specific for my organization in my preferences? Is there any community learning over time? Because there are industry breast pack that best practices out there that are super valuable. But, you know, we saw in the SAS wave when I can customize things myself were learned over time. So how does that play into kind of today in the road map for a I that you're building >>that? That's a good question. So our AI let's say technology that we use it actually uses to two different big kinds of AI. So we use machine learning definitely to learn from the community. What are the best practices and what are the most common pattern that people use? So we use that to guide developers, but also to validate and analyze their code. But then we also use automated reasoning. So this is more logic based reasoning based AI and repair these two technologies to really create a system that is able to learn from data but also be able to reason at a higher order about what are good practices and kind of reach conclusions from there and learn new things from there now. We started by applying these technologies to more of the community data and kind of standard best practices. But our vision is to more and more start learning specifically and allowing tech leads an architect even in the future. To Taylor. These engines of AI, perhaps to suggest these are the best practices for my factory. These patterns perhaps, are good best practices in general. But in my factory, I do not want to use them because I have some specificities for compliance or something like that. And our vision is that architects and techniques can just provide just a few examples of what they like and what they don't like in the engine just automatically learns and gets tailor to their own environment. >>So important that you're able to, uh, you know, have the customers move things forward in the direction that makes sense on their end. I'm also curious. You talk about, um, you know what what partnerships out systems has out there, you know, being able to tie into things like what the public cloud is doing. Lots of industry collaboration. So how does health system fit into the kind of the broader ai ecosystem. >>Yes. So one thing I did not mention and to your point is eso were have kind of to, um Teoh Complementary visions and strategies for a I. So one of them is we really want to improve our own product, improve the automation in the product in the abstraction by using AI together with great user experience and the best programming language for software on automation. Right, So that's one. That's what we generally call AI assisted development. And if using AI across this software development life cycle, the other one is We also believe that you know, true elite cloud software companies that create frictionless experiences. One of the things that they used to really be super competitive and create this frictionless experiences is that they can themselves use AI and machine learning to to automate processes created really, really delightful experiences. So we're also investing and we've shown and we're launching, announcing that next step we just showed this at at the keynote one tool that we call the machine learning builder ml builder. So this essentially speaks to the fact that you know, a lot of companies do not have access to data science talent. They really struggle to adopt machine learning. Like just one out of 10 companies are able to go and put a I in production. So we're essentially abstracting also that were also increasing the productivity for you for customers to implement an AI and machine learning we use. We use partners behind the scenes and cloud providers for the core technology with automated machine learning and all of that. But we abstract all of the experience so developers can essentially just pick of the data they have already in the inside the all systems platform, and they want to just select. I want to trade this machine learning model to predict this field, just quickly click and it runs dozens of experiments, selects the best algorithms, transforms that the data for you without you needing to have a lot of data science experience. And then you can just drag and drop in the platform integrating your application. And you're good to go. >>Well, it sounds comes Ah, you know, phenomenal. You mentioned data scientists. We talked about that. The skill gap. Do you have any statistics? You know? Is this helping people you know? Higher, Faster. Lower the bar the entry for people to get on board, you know, increased productivity. What kind of hero numbers do your customers typically, you know, how do they measure success? >>Yes, So we know that in for machine learning adoption at cos we know that. Sorry, This is one of the top challenges that they have, right? So companies do not. It's not only that they do not have the expertise to implement machine learning at in their products in their applications. They don't even have a good understanding of what are the use cases in or out of the technology opportunities for them to apply. Right? So this has been listed by lots of different surveys that this is the top problem. These other 22 of the top problems that companies have to adopt a ice has access to skilled. They decided skill, understanding of the use case. And that's exactly what we're trying to kind of package up in a very easy to use product where you can see the use cases you have available, we just select your data, you just click train. You do not need to know that many greedy details and for us, a measure of success is that we've seen customers that are starting to experiment with ML Builder is that in just a day or a few days that can iterating over several machine learning models and put them in production. We have customers that have, you know, no machine learning models and production ever, and they just now have to, and they're starting to automate processes. They're starting to innovate with business. And that, for us, is we've seen it's kind of the measure of success for businesses initially, what they want to do is they want to do. POC is and they want to experiment and they want to get to production stopped. Getting to field for it and generate from >>a product standpoint, is the A. I just infused in or there's there additional licensing, how to customers, you know to take advantage of it. What's the impact on that from the relationship without systems? >>Yes. So for for for a I in machine learning that is fused into our product and for automation, validation and guidance, there's no extra charge is just part of the product. It's what we believe is kind of a core building block in a course service for everything we do in our product for machine learning services and components that customers can use to in their own applications. We allow you to integrate with cloud providers, and the building is is done separately on. That's something that that we're working towards and building great technical partnerships and exploring other avenues for deeper integration so that developers and customers do not really have to worry about those things. Well, >>it's it's It's such a great way to really democratize the use of this technology platform that they're used to. They start doing it. What's general feedback from your customers? Did they just like, Oh, it's there. I start playing with it. It's super easy. It makes it better there any concerns or push back. Have we gotten beyond that? What? What? What do you hear any any good customer examples you can share us toe general adoption? >>Yes. So, as I said, as we re reduce the friction for adopting these technologies, we've seen one thing that's very interesting. So we have a few customers that are present more in the logistics site of industry and vertical, and so they they have a more conservative management, like take time to adopt and more of a laggard in adopting these kinds of technologies, the businesses more skeptical. But I want to spend a lot of time playing around right and whence they saw. Once they saw what they could do with a platform, they quickly did a proof of concept. They show to the business and the business had lots of ideas. So they just started interacting a lot more with I t, which is something we see without systems platform not just for a I machine learning, but generally in the jib. Digital transformation is when the I teak and can start really being very agile in iterating and innovating, and they start collaborating a lot with the business. And so what we see is customers asking us for even more so customers want more use cases to be supported like this. Customers also the ones that are more mature than already, have their centers of excellence and they have their data scientists, for example. They want to understand how they can also bring in perhaps their use of very specialized tool talking in it. Integrate that into the platform so that you know, for certain use cases. Developer scan very quickly trained their own models. But so specialized data science teams can also bring in. And developers can integrate their models easily and put them into production, which is one of the big barriers we see in a lot of companies people working on yearlong projects. They develop the models that they struggle to get them to production. And so we really want to focus on the whole into in journey. Either you're building everything within the octopus platform or you're bringing it from a specialized pro tool. We want to make that whole journey frictionless in school. >>And Tony a final question I have for you. Of course, this space we're seeing maturing, you know, rapid Ah, new technologies out there gives a little look forward. What should we be expecting to see from out systems or things even a little broader? If you look at your your partner ecosystem over kind of the next 6, 12 18 months, >>Yes. So, um, what you're going to continues to see a trend, I think, from from the closer providers of democratization of the AI services. So this is during that just starting to advanced and accelerate as these providers started packaging. It's like what out systems also doing, starting to packaging Cem some specific, well defined use cases and then making the journey for training these models and deploying Super super simple. That's one thing that's continued to ramp up, and we're going to move from A I services more focused on cognitive, pre trained models, right, that which is kind of the status quo to custom ai models based on your data. That's kind of the train we're going to start seeing in that out systems also pushing forward generally from the AI and machine learning application and technology side of thing. I think one thing that we're leading leading on is that you know, machine learning and deep learning is definitely one of the big drivers for the innovation that we're seeing in a I. But you're start seeing more and more what is called hybrid I, which is taking machine learning and data based artificial intelligence with more logic based automated reasoning techniques, impairing these two to really create systems that are able to operate at a really higher level, higher cognitive level of which is what out systems investing internally in terms of research and development and with partnerships with institutions like Carnegie Mellon University and >>rely Antonio, who doesn't want, you know, a tech experts sitting next to them helping get rid of some of the repetitive, boring things or challenges. Thank you so much for sharing the update. Congratulations. Definitely Look forward to hearing war in the future. >>Thank you. Do have a good day >>Stay tuned for more from out systems. Next step is to minimum and thank you for watching.

>> Announcer: From theCUBE studios in Palo Alto and Boston, connecting with thought leaders all around the world, this is a CUBE Conversation. >> Hi, I'm Stu Miniman, and welcome to a CUBE Conversation. We always love talking to founders of companies. We love supporting the Boston-area community, but even more, right now, we're of course talking to leaders in the industry about some of the challenges facing with the global pandemic, so, happy to welcome to the program first-time guest Paulo Rosado, who is the founder and CEO of OutSystems. You are based in Boston, your company is global. Paulo, thanks so much for joining us, and let's start out talking about kind of the age we are in right now and how you are supporting your customers, your employees, and the developer community that you engage with. >> Absolutely, and it's a pleasure to be here, Stu. Actually, since the 23rd of March, that our 1,100 employees are all working remote, so we've had more than 1,000 Zoom calls logged, at least among the people that I know. And we have dogs and kids everywhere, and we have to adjust, 'cause we have a lot of new parents, so the kids are all over them and whatever. But actually, productivity and morale is really at a high rate. The business is going really, really well. However, as in a very OutSystems type of way, actually, because we're so fast building these digital solutions that we've launched a program with our partners. We asked them for ideas, we got more than 200 ideas coming in, and we're sponsoring 20 of those ideas. One of them is with Deloitte, for instance, where we fundamentally, in one week, they've created a full logistics system to manage all the supplies within 16 municipalities, including ventilators, masks, PPEs and the like. >> Well, that's great to hear, right. So if people want to find out more on the OutSystems website, it's the COVID-19 Community Response Program, and love to see, Paulo, we're going to talk a bit about OutSystems and what you're doing for customers. Of course, the speed of development of new applications is what your company's been doing for a long time, and it kind of becomes a little bit bromide that we talk about, "Oh, well, software's eating the world." Well, in challenging times, how is software hoping to meet the challenges that communities, municipalities, employees, companies need to survive in these challenging situations? So anything else you want to talk about, kind of the community program? >> Yeah, well, so what we did is we opened up the community, worldwide community, actually, because today we serve about 60 countries, and so we wanted to have projects that really add impact. We had a couple from Germany, and some from Asia, and it's amazing. Today, we have sponsored 14, so we have 14 scalable installations already running. Some of these projects have gone live, some are still in development. But what's interesting is that the 200,000-plus communities, that they're getting together. We have all these virtual teams, subject matter experts, relationships with house officers and house offices, and developers, and we're just churning away. And the innovation of the people when they have, actually, something that they can build real solutions fast, they can iterate on top, it's absolutely amazing. And it's our contribution, also, to the world here, really. >> Yeah, very important, Paulo, thank you for doing that. Boy, I think, Paulo, you started the company back in 2001. The discussion around software and developers was rather nascent back in those days. So bring us a little bit through the journey of the company, if you would, and some of the major things that are different now in, really, you're entering the third decade of the company, so bring us back to some of the early days, as well as, what is significantly different today? >> Actually, the idea that we had initially was very much the one that has become truth. We were just about 14 years ahead of the market. So the company's called OutSystems because at the time, we believed that a large percent of systems would migrate out of the data center. That is what today is called the cloud. We believed, at the time, based on all the evidence, that a lot of software that companies were going to be building needed to be done in a very agile way, which is, you need to build fast, but not only build fast, but change very, very fast. And it took us a while until we reached about three to four years ago, when suddenly, everything became agile. Suddenly, everything that you build, all the software that you build, you no longer had one year or 18 months to build this project. Now, you had weeks, and those times have been compressing. And so, what's happening now is we encounter ourselves in a world where companies increasingly want to build more software because they want to be differentiated, they want to compete, but the talent available and the speed they have to build these pieces of software are becoming more and more challenging, and we help a lot in doing that. We are the most mature, the most advanced no-code/low-code platform in the market. And so, it's a great time for us now. >> Yeah, Paulo, I'd like to help understand software development, application modernization are very important topics for a number of years now. I think back to last year, Satya Nadella on stage at Microsoft Ignite, and he was talking about just the massive amounts of new applications that would be built over the next few years. And it's interesting, a company like Microsoft that, you go back 10 years ago, it'd be like, "Well, you'll be using all of our software, "not thinking about building your own software." So you've got partnerships with the public cloud providers, there's all sorts of new partners as well as competitors entering the space. So help us understand kind of where OutSystems fits in this ecosystem and differentiates itself from some of the other noise that's out there. >> No, absolutely. Well, we've woken up a lot of giants, definitely, with this approach. One of the differentiators is that these platforms are actually pretty hard to build, and so, if you look into what Satya said in that particular conference, he was mentioning the fact that fundamentally, every company needs to become a cloud software company. But in order for you to become a cloud software company, you need a very large number of talent skills. You need good web developers, front-end developers, back-end developers. You need to have people who understand DevOps, you need to understand scalability, security, all of these things. You can do that with the tens and even hundreds of tools that are in the market, but what the platform like OutSystems stands up by doing is ends up abstracting a lot of debt and just gives you a very fast capacity for you to build your mobile applications, your pricing engines, your workflows, your portals, in a very fast way. So leveraging the people that you have, leveraging the unique knowledge of the business that you have, and letting you catch up to disruptors that really have all those technical skill sets that today are so rare. >> Yeah, and I'd love to hear, tell us a little bit about your customer base. So you've been around for many years, so I'm sure it is quite diverse, but how many customers does OutSystems have? If you've got a key use case or two that might help us understand where this low-code/no-code solution is helping them through their journey. >> Oh, absolutely, we have companies like Safeway, Chevron, T-Mobile. All of them have somehow different use cases, because we are in the business of innovation, and so, whatever you want to innovate with, you innovate typically with OutSystems. We have a particular company which is the largest oil and gas terminal management company in the world. They have 73 terminals. And one of the things they built was a full ERP, a full platform, digital platform, to manage all logistics of the tankers that come into the ports, deploy the oil in the reservoirs, and then having trucks that come and take the oil away. It's a very complex business, and they were looking at, fundamentally, a four- to six-year project to build this, and they did it in seven months. And so, these type of compressions of time for these very large systems is a huge, huge differentiator. Then we have, on the other hand, companies that have built their front ends, typically mobile applications integrated with web applications, and those applications change, fundamentally, almost every day or every week. We have a bank, for instance, that's releasing a version per day in their applications. That speed of development gives them a huge competitive advantage but puts a lot of pressure on the stack and all the IT that's needed, and we help there because of the platform. >> Yeah, Paulo, we've been talking for years about some of the transformations that companies are going through, and that application transformation really is one of the bigger challenges that they face along those lines. In some of the events I go to, the communities I look at, there's a lot of talk about how containerization in Kubernetes is helping to move the infrastructure team to get ready for this. Of course, we've talked a bit already about how public cloud's changing things. Serverless is a different paradigm for how application developers should think about the platforms they're living on. How does OutSystems kind of plug in to these trends which have come along in the time since you've been out there? >> Oh, very well. The way these platforms work, at least, the way the OutSystems platform works, is that we have an automation layer who's responsible, fundamentally, for compressing time and making things increasingly easier. Basically, just give an IT department or company the capacity to build things 100 times faster. But underneath, we actually use the newest architectures that give us high scalability, also scaling resilience, 99.999% of uptime. And in those cases, for instance, for that, we use containers, Linux, Docker, all of those type of technologies. We run standard on AWS, we also run on Azure. And so, we can provide automation, but underneath, we're fundamentally using the same tools that all enterprise-grade architects are using. >> Okay, great, Paulo. Last question I have for you, give us a little bit your outlook on the future of software development, what we should be looking at when it comes to OutSystems and your community. >> Well, actually, it's not only about OutSystems, it's all about development of the software. We believe, and we see evidence of it, that while software development used to be done by some elites about 10, 15 years ago, today, every company needs to build their own software. And more than 65% of new software that's going to be built in the next three to five years is going to be done with a no-code or low-code platform. That's just too much, you just need that speed, you don't have enough talent. And actually, what we see, and we're doing a lot of research there, is that complementing the developers, we're seeing more and more AI bots that actually assist development in a lot of the boring tasks that are part of the development and deployment cycle, like validation of code, automatic testing, creating the right patterns of architecture for high scalability and maintainability. We're introducing a lot of those things in the platform. So in the next years, we believe we'll see more and more developers being helped by artificial intelligence bots, therefore progressing in that 100X to 1,000X automation productivity enhancement. >> Well, I tell you, you're hitting on one of our favorite topics to talk about. (Paulo chuckles) We did an event years ago with Andy McAfee and Erik Brynjolfsson from MIT, talking about how it really is about racing with the machines. So I've seen things that said, "Oh, computer programmers, you're the next things "that are going to be replaced by robots." And what I'm hearing from you is, of course, what we know is that really it is the combination of people plus this software that are really going to supercharge things going forward. And you're nodding, so you would agree. >> That's exactly it. And we already have evidence of that because we have a lot of our AI is already deployed inside the platform, and so we're measuring, we're learning with it. And we can see tremendous, almost exponential improvements. It's almost as if a developer, as they're creating these functional requirements, gets augmented with an extra brain. So it really works, and it's time now, it's reaching time for AI to be used to help the software development cycle. >> Right, well, Paulo, thank you so much for the conversation. Absolutely we hope that these kind of technologies are the ones that are going to help the global economy as we hopefully move forward from the results of the current global situation here. So thank you so much for joining us, and definitely look forward to keeping track of the company in the future. All right-- >> Thank you, Stu, it was a pleasure. Thank you very much. >> Thanks, I'm Stu Miniman, and as always, check out theCUBE.net for all of the digital events, as well as the archives of interviews that we've done, reach out to us if you have any questions, and as always, thank you for watching theCUBE. (upbeat electronic music)

>> Narrator: From around the globe, it's theCUBE! With digital coverage of DevOps Virtual Forum, brought to you by Broadcom. >> Hi, Lisa Martin here covering the Broadcom DevOps Virtual Forum. I'm very pleased to be joined today by a CUBE alumni, Jeffrey Hammond, the Vice President and Principal Analyst serving CIOs at Forrester. Jeffrey, nice to talk with you today. >> Good morning, it's good to be here. >> So, a virtual forum, a great opportunity to engage with our audiences. So much has changed in the last, it's an understatement, right? Or it's an overstated thing, but it's obvious. So much has changed. When we think of DevOps, one of the things that we think of is speed, enabling organizations to be able to better serve customers or adapt to changing markets like we're in now. Speaking of the need to adapt, talk to us about what you're seeing with respect to DevOps and Agile in the age of COVID. What are things looking like? >> Yeah, I think that for most organizations, we're in a period of adjustment. When we initially started, it was essentially a sprint. You run as hard as you can for as fast as you can for as long as you can and you just kind of power through it. And that's actually what the folks at GitHub saw in May, when they run an analysis of how developers commit times and level of work that they were committing and how they were working. In the first couple months of COVID, was progressing, they found that developers, at least in the Pacific Time Zone, were actually increasing their work volume, maybe 'cause they didn't have two hour commutes, or maybe because they work stuck away in their homes, but for whatever reason, they were doing more work. And it's almost like, if you've ever run a marathon, the first mile or two in the marathon, you feel great, you just want to run and you want to power through it, you want to go hard. And if you do that, by the time you get to mile 18 or 19, you're going to be gassed, sucking for wind. And that's I think where we're starting to hit. So as we start to gear our development shops up for the reality that most of us won't be returning into an office until 2021 at the earliest. And many organizations will be fundamentally changing their remote work policies, we have to make sure that the agile processes that we use, and the DevOps processes and tools that we use to support these teams are essentially aligned to help developers run that marathon, instead of just kind of power through. So, let me give you a couple specifics. For many organizations, they have been in an environment where they will tolerate remote work and what I would call remote work around the edges, like developers can be remote, but product managers and essentially scrum masters and all the administrators that are running the SCM repositories and the DevOps pipelines are all in the office. And it's essentially centralized work. That's not where we are anymore. We're moving from remote workers at the edge to remote workers at the center of what we do. And so, one of the implications of that is that we have to think about all the activities that you need to do from a DevOps perspective, or from an agile perspective. They have to be remotable. One of the things I found with some of the organizations I talked to early on was, there were things that administrators had to do that required them to go into the office, to reboot the SCM server as an example, or to make sure that the final approvals for production were made. And so, the code could be moved into the production environment. And so, it actually was a little bit difficult because they had to get specific approval from the HR organizations to actually be allowed to go into the office in some states. And so, one of the the results of that is that, while we've traditionally said tools are important, but they're not as important as culture, as structure, as organization, as process, I think we have to rethink that a little bit. Because to the extent that tools enable us to be more digitally organized and to achieve higher levels of digitization in our processes, and be able to support the idea of remote workers in the center. They're now on an equal footing with so many of the other levers that organizations have at their disposal. I'll give you another example. For years, we've said that the key to success with Agile at the team level is cross functional, co-located teams that are working together. Physically co-located. It's the easiest way to show agile success. We can't do that anymore. We can't be physically located at least for the foreseeable future. So, how do you take the low hanging fruits of an agile transformation and apply it in the time of COVID? Well, I think what you have to do is you have to look at what physical co-location has enabled in the past and understand that it's not so much the fact that we're together looking at each other across the table, it's the fact that we're able to get into a shared mind space. From a measurement perspective, we can have shared purpose, we can engage in high bandwidth communications. It's the spiritual aspect of that physical co-location that is actually important. So, one of the biggest things that organizations need to start to ask themselves is, how do we achieve spiritual co-location with our Agile teams, because we don't have the ease of physical co-location available to us anymore. >> Well, spiritual co-location is such an interesting kind of provocative phrase there, but something that probably was a challenge. Here we are seven, eight months in, for many organizations as you say, going from physical workspaces, co-location, being able to collaborate face to face to a light switch flip overnight, and this undefined indeterminate period of time where all we were living with was uncertainty. How does spiritual... When you talk about spiritual co-location in terms of collaboration and processes and technology. Help us unpack that and how are you seeing organizations adopt it? >> Yeah, it's a great question. And I think it goes to the very root of how organizations are trying to transform themselves to be more agile and to embrace DevOps. If you go all the way back to the original Agile Manifesto. There were four principles that were espoused. Individuals and interactions over processes and tools. That's still important, individuals and interactions are at the core of software development. Processes and tools that support those individuals in those interactions are more important than ever. Working software over comprehensive documentation. Working software is still more important. But when you are trying to onboard employees, and they can't come into the office, and they can't do the two day training session, and kind of understand how things work, and they can't just holler over theCUBE, to ask a question, you may need to invest a little bit more in documentation to help that onboarding process be successful in a remote context. Customer collaboration over contract negotiation. Absolutely still important. But employee collaboration is equally as important if you want to be spiritually co-located and if you want to have a shared purpose. And then, responding to change over following a plan. I think one of the things that's happened in a lot of organizations is we have focused so much of our DevOps effort around velocity. Getting faster, we need to run as fast as we can. Like that sprinter, okay? Trying to just power through it as quickly as possible. But as we shift to the marathon way of thinking, velocity is still important but agility becomes even more important. So when you have to create an application in three weeks to do track and trace for your employees, agility is more important than just flat out velocity. And so, changing some of the ways that we think about DevOps practices is important to make sure that that agility is there. For one thing, you have to defer decisions as far down the chain to the team level as possible. So those teams have to be empowered to make decisions. Because you can't have a program level meeting of six or seven teams in one large hall and say, here's the lay of the land, here's what we're going to do, here are our processes, and here are our guardrails. Those teams have to make decisions much more quickly. The developers are actually developing code in smaller chunks of flow. They have to be able to take two hours here, or 50 minutes there and do something useful. And so, the tools that support us have to become tolerant of the reality of how we're working. So, if they work in a way that it allows the team together to take as much autonomy as they can handle, to allow them to communicate in a way that delivers shared purpose, and allows them to adapt and master new technologies, then they're in the zone, they'll get spiritually connected. I hope that makes sense (chuckles). >> It does, I think we all could use some of that. But you talked about in the beginning and I've talked to numerous companies during the pandemic on theCUBE about the productivity or rather the number of hours worked has gone way up for many roles, and times that they normally at late at night on the weekends. So, but it's a cultural, it's a mind shift. To your point about DevOps focused on velocity, sprint, sprint, sprint, and now we have to. So that cultural shift is not an easy one for developers and even the biz folks to flip so quickly. What have you seen in terms of the velocity at which businesses are able to get more of that balance between the velocity, the sprint and the agility? >> I think at the core, this really comes down to management sensitivity. When everybody was in the office, you could kind of see the mental health of development teams by watching how they work, you can call it management by walking around, right? We can't do that, managers have to be more aware of what their teams are doing, because they're not going to see that developer doing a check in at 9:00 p.m. on a Friday, because that's what they had to do to meet the objectives. And they're going to have to find new ways to measure engagement and also potential burnout. A friend of mine once had a great metric that he called the Parking Lot Metric. It was helpful as the parking lot at nine and helpful was it at five. And that gives you an indication of how engaged your developers are. What's the digital equivalent of the Parking Lot Metric in the time of COVID, it's commit stats, it's commit rates, it's the turn rate that we have in our code. So we have this information, we may not be collecting it, but then the next question becomes how do we use that information? Do we use that information to say, well, this team isn't delivering at the same level of productivity as another team? Do we weaponize that data? Or do we use that data to identify impedances in the process? Why isn't a team working effectively? Is it because they have higher levels of family obligations, and they've got kids that are at home? Is it because they're working with hardware technology, and guess what, it's not easy to get the hardware technology into their home office, because it's in the lab, at the corporate office. Or they're trying to communicate halfway around the world. And they're communicating with an office lab that is also shut down. And the bandwidth just doesn't enable the level of high bandwidth communications. So, from a DevOps perspective, managers have to get much more sensitive to the exhaust that the DevOps tools are throwing off, but also how they're going to use that in a constructive way to prevent burnout. And then they also need to, if they're not already managing, or monitoring or measuring the level of developer engagement they have, they really need to start. Whether that's surveys around developer satisfaction, whether it's more regular social events where developers can kind of just get together and drink a beer and talk about what's going on in the project and monitoring who checks in and who doesn't. They have to work harder, I think than they ever have before. >> Well, and you mentioned burnout. And that's something that I think we've all faced in this time at varying levels, and it changes and it's a real, there's a tension in the air regardless of where you are. There's a challenge, as you mentioned, people having their kids as co-workers and fighting for bandwidth, because everyone is forced in this situation. I'd love to get your perspective on some businesses that have done this, well, this adaptation. What can you share in terms of some real world examples that might inspire the audience? >> Yeah, I'll start with Stack Overflow. They recently published a piece in the Journal of the ACM around some of the things that they had discovered. First of all, just a cultural philosophy. If one person is remote, everybody is remote. And you just think that way from the executive level. Social spaces, one of the things that they talk about doing is leaving the video conference room open at the team level all day long. And the team members will go on mute, so that they don't have to, that they don't necessarily have to be there with somebody else listening to them. But if they have a question, they can just pop off mute really quickly and ask the question and if anybody else knows the answer, it's kind of like being in that virtual pod, if you will. Even here at Forrester, one of the things that we've done is we've invested in social ceremonies. We've actually moved our team meetings on my analyst team from once every two weeks to weekly. And we have built more time in for socialization, just so we can see how we're doing. I think Microsoft has really made some good information available in how they've managed things like the onboarding process. I think Amanda Silver over there mentioned that a couple of weeks ago, a presentation they did that Microsoft's onboarded over 150,000 people since the start of COVID. If you don't have good remote onboarding processes, that's going to be a disaster. Now, they're not all developers, but if you think about it, everything from how you do the interviewing process, to how you get people their badges, to how they get their equipment. Security is another issue that they called out. Typically, IT security, security of developers machines, ends at the corporate desktop. But now since we're increasingly using our own machines, our own hardware, security organization's going to have to extend their security policies to cover employee devices. And that's caused them to scramble a little bit. So, the examples are out there. It's not a lot of like, we have to do everything completely differently. But it's a lot of subtle changes that have to be made. I'll give you another example. One of the things that we are seeing is that more and more organizations to deal with the challenges around agility with respect to delivering software and embracing low code tools. In fact, we see about 50% of firms are using low code tools right now, we predict it's going to be 75% by the end of next year. So, figuring out how your DevOps processes support an organization that might be using Mendix or OutSystems, or the Power Platform, building the front end of an application, like a track and trace application really, really quickly. But then hooking it up to your back end infrastructure. Does that happen completely outside the DevOps investments that you're making? And the agile processes that you're making? Or do you adapt your organization. Are hybrid teams now, teams that not just have professional developers, but also have business users that are doing some development with a low code tool. Those are the kinds of things that we have to be willing to entertain in order to shift the focus a little bit more toward the agility side, I think. >> A lot of obstacles but also a lot of opportunities for businesses to really learn, pay attention here, pivot and grow and hopefully some good opportunities for the developers and the business folks to just get better at what they're doing and learning to embrace spiritual co-location. Jeffrey, thank you so much for joining us on the program today, very insightful conversation. >> It's my pleasure, it's an important thing. Just remember, if you're going to run that marathon, break it into 26, 10 minute runs, take a walk break in between each, and you'll find that you'll get there. >> Digestible components, wise advice. Jeffrey Hammond, thank you so much for joining. For Jeffrey, I'm Lisa Martin. You're watching Broadcom's DevOps Virtual Forum. (bright upbeat music)

>>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi, Lisa Martin here covering the Broadcom dev ops virtual forum. I'm very pleased to be joined today by a cube alumni, Jeffrey Hammond, the vice president and principal analyst serving CIO is at Forester. Jeffrey. Nice to talk with you today. >>Good morning. It's good to be here. Yeah. >>So a virtual forum, great opportunity to engage with our audiences so much has changed in the last it's an understatement, right? Or it's an overstated thing, but it's an obvious, so much has changed when we think of dev ops. One of the things that we think of is speed, you know, enabling organizations to be able to better serve customers or adapt to changing markets like we're in now, speaking of the need to adapt, talk to us about what you're seeing with respect to dev ops and agile in the age of COVID, what are things looking like? >>Yeah, I think that, um, for most organizations, we're in a, uh, a period of adjustment, uh, when we initially started, it was essentially a sprint, you know, you run as hard as you can for as fast as you can for as long as you can and you just kind of power through it. And, and that's actually what, um, the folks that get hub saw in may when they ran an analysis of how developers, uh, commit times and a level of work that they were committing and how they were working, uh, in the first couple of months of COVID was, was progressing. They found that developers, at least in the Pacific time zone were actually increasing their work volume, maybe because they didn't have two hour commutes or maybe because they were stuck away in their homes, but for whatever reason, they were doing more work. >>And it's almost like, you know, if you've ever run a marathon the first mile or two in the marathon, you feel great and you just want to run and you want to power through it and you want to go hard. And if you do that by the time you get to mile 18 or 19, you're going to be gassed. It's sucking for wind. Uh, and, and that's, I think where we're starting to hit. So as we start to, um, gear our development chops out for the reality that most of us won't be returning into an office until 2021 at the earliest and many organizations will, will be fundamentally changing, uh, their remote workforce, uh, policies. We have to make sure that the agile processes that we use and the dev ops processes and tools that we use to support these teams are essentially aligned to help developers run that marathon instead of just kind of power through. >>So, um, let me give you a couple of specifics for many organizations, they have been in an environment where they will, um, tolerate Rover remote work and what I would call remote work around the edges like developers can be remote, but product managers and, um, you know, essentially scrum masters and all the administrators that are running the, uh, uh, the SCM repositories and, and the dev ops pipelines are all in the office. And it's essentially centralized work. That's not, we are anymore. We're moving from remote workers at the edge to remote workers at the center of what we do. And so one of the implications of that is that, um, we have to think about all the activities that you need to do from a dev ops perspective or from an agile perspective, they have to be remote people. One of the things I found with some of the organizations I talked to early on was there were things that administrators had to do that required them to go into the office to reboot the SCM server as an example, or to make sure that the final approvals for production, uh, were made. >>And so the code could be moved into the production environment. And so it actually was a little bit difficult because they had to get specific approval from the HR organizations to actually be allowed to go into the office in some States. And so one of the, the results of that is that while we've traditionally said, you know, tools are important, but they're not as important as culture as structure as organization as process. I think we have to rethink that a little bit because to the extent that tools enable us to be more digitally organized and to hiring, you know, achieve higher levels of digitization in our processes and be able to support the idea of remote workers in the center. They're now on an equal footing with so many of the other levers, uh, that, that, um, uh, that organizations have at their disposal. Um, I'll give you another example for years. >>We've said that the key to success with agile at the team level is cross-functional co located teams that are working together physically co located. It's the easiest way to show agile success. We can't do that anymore. We can't be physically located at least for the foreseeable future. So, you know, how do you take the low hanging fruits of an agile transformation and apply it in, in, in, in the time of COVID? Well, I think what you have to do is that you have to look at what physical co-location has enabled in the past and understand that it's not so much the fact that we're together looking at each other across the table. It's the fact that we're able to get into a shared mindspace, uh, from, um, uh, from a measurement perspective, we can have shared purpose. We can engage in high bandwidth communications. It's the spiritual aspect of that physical co-location that is actually important. So one of the biggest things that organizations need to start to ask themselves is how do we achieve spiritual colocation with our agile teams? Because we don't have the, the ease of physical co-location available to us anymore? >>Well, the spiritual co-location is such an interesting kind of provocative phrase there, but something that probably was a challenge here, we are seven, eight months in for many organizations, as you say, going from, you know, physical workspaces, co-location being able to collaborate face to face to a, a light switch flip overnight. And this undefined period of time where all we were living with with was uncertainty, how does spiritual, what do you, when you talk about spiritual co-location in terms of collaboration and processes and technology help us unpack that, and how are you seeing organizations adopted? >>Yeah, it's, it's, um, it's a great question. And, and I think it goes to the very root of how organizations are trying to transform themselves to be more agile and to embrace dev ops. Um, if you go all the way back to the, to the original, uh, agile manifesto, you know, there were four principles that were espoused individuals and interactions over processes and tools. That's still important. Individuals and interactions are at the core of software development, processes and tools that support those individual and interact. Uh, those individuals in those interactions are more important than ever working software over comprehensive documentation. Working software is still more important, but when you are trying to onboard employees and they can't come into the office and they can't do the two day training session and kind of understand how things work and they can't just holler over the cube, uh, to ask a question, you may need to invest a little bit more in documentation to help that onboarding process be successful in a remote context, uh, customer collaboration over contract negotiation. >>Absolutely still important, but employee collaboration is equally as important if you want to be spiritually, spiritually co-located. And if you want to have a shared purpose and then, um, responding to change over following a plan. I think one of the things that's happened in a lot of organizations is we have focused so much of our dev ops effort around velocity getting faster. We need to run as fast as we can like that sprinter. Okay. You know, trying to just power through it as quickly as possible. But as we shift to, to the, to the marathon way of thinking, um, velocity is still important, but agility becomes even more important. So when you have to create an application in three weeks to do track and trace for your employees, agility is more important. Um, and then just flat out velocity. Um, and so changing some of the ways that we think about dev ops practices, um, is, is important to make sure that that agility is there for one thing, you have to defer decisions as far down the chain to the team level as possible. >>So those teams have to be empowered to make decisions because you can't have a program level meeting of six or seven teams and one large hall and say, here's the lay of the land. Here's what we're going to do here are our processes. And here are our guardrails. Those teams have to make decisions much more quickly that developers are actually developing code in smaller chunks of flow. They have to be able to take two hours here or 50 minutes there and do something useful. And so the tools that support us have to become tolerant of the reality of, of, of, of how we're working. So if they work in a way that it allows the team together to take as much autonomy as they can handle, um, to, uh, allow them to communicate in a way that, that, that delivers shared purpose and allows them to adapt and master new technologies, then they're in the zone in their spiritual, they'll get spiritually connected. I hope that makes sense. >>It does. I think we all could use some of that, but, you know, you talked about in the beginning and I've, I've talked to numerous companies during the pandemic on the cube about the productivity, or rather the number of hours of work has gone way up for many roles, you know, and, and, and times that they normally late at night on the weekends. So, but it's a cultural, it's a mind shift to your point about dev ops focused on velocity, sprints, sprints, sprints, and now we have to, so that cultural shift is not an easy one for developers. And even at this folks to flip so quickly, what have you seen in terms of the velocity at which businesses are able to get more of that balance between the velocity, the sprint and the agility? >>I think, I think at the core, this really comes down to management sensitivity. Um, when everybody was in the office, you could kind of see the mental health of development teams by, by watching how they work. You know, you call it management by walking around, right. We can't do that. Managers have to, um, to, to be more aware of what their teams are doing, because they're not going to see that, that developer doing a check-in at 9:00 PM on a Friday, uh, because that's what they had to do, uh, to meet the objectives. And, um, and, and they're going to have to, to, um, to find new ways to measure engagement and also potential burnout. Um, friend of mine once had, uh, had a great metric that he called the parking lot metric. It was helpful as the parking lot at nine. And how full was it at five? >>And that gives you an indication of how engaged your developers are. Um, what's the digital equivalent equivalent to the parking lot metric in the time of COVID it's commit stats, it's commit rates. It's, um, you know, the, uh, the turn rate, uh, that we have in our code. So we have this information, we may not be collecting it, but then the next question becomes, how do we use that information? Do we use that information to say, well, this team isn't delivering as at the same level of productivity as another team, do we weaponize that data or do we use that data to identify impedances in the process? Um, why isn't a team working effectively? Is it because they have higher levels of family obligations and they've got kids that, that are at home? Um, is it because they're working with, um, you know, hardware technology, and guess what, they, it's not easy to get the hardware technology into their home office because it's in the lab at the, uh, at the corporate office, uh, or they're trying to communicate, uh, you know, halfway around the world. >>And, uh, they're communicating with a, with an office lab that is also shut down and, and, and the bandwidth just doesn't enable the, the level of high bandwidth communications. So from a dev ops perspective, managers have to get much more sensitive to the, the exhaust that the dev ops tools are throwing off, but also how they're going to use that in a constructive way to, to prevent burnout. And then they also need to, if they're not already managing or monitoring or measuring the level of developer engagement, they have, they really need to start whether that's surveys around developer satisfaction, um, whether it's, you know, more regular social events, uh, where developers can kind of just get together and drink a beer and talk about what's going on in the project, uh, and monitoring who checks in and who doesn't, uh, they have to, to, um, work harder, I think, than they ever have before. >>Well, and you mentioned burnout, and that's something that I think we've all faced in this time at varying levels and it changes. And it's a real, there's a tension in the air, regardless of where you are. There's a challenge, as you mentioned, people having, you know, coworker, their kids as coworkers and fighting for bandwidth, because everyone is forced in this situation. I'd love to get your perspective on some businesses that are, that have done this well, this adaptation, what can you share in terms of some real-world examples that might inspire the audience? >>Yeah. Uh, I'll start with, uh, stack overflow. Uh, they recently published a piece in the journal of the ACM around some of the things that they had discovered. Um, you know, first of all, just a cultural philosophy. If one person is remote, everybody is remote. And you just think that way from an executive level, um, social spaces. One of the things that they talk about doing is leaving a video conference room open at a team level all day long, and the team members, you know, we'll go on mute, you know, so that they don't have to, that they don't necessarily have to be there with somebody else listening to them. But if they have a question, they can just pop off mute really quickly and ask the question. And if anybody else knows the answer, it's kind of like being in that virtual pod. Uh, if you, uh, if you will, um, even here at Forrester, one of the things that we've done is we've invested in social ceremonies. >>We've actually moved our to our team meetings on, on my analyst team from, from once every two weeks to weekly. And we have built more time in for social Ajay socialization, just so we can see, uh, how, how, how we're doing. Um, I think Microsoft has really made some good, uh, information available in how they've managed things like the onboarding process. I think I'm Amanda silver over there mentioned that a couple of weeks ago when, uh, uh, a presentation they did that, uh, uh, Microsoft onboarded over 150,000 people since the start of COVID, if you don't have good remote onboarding processes, that's going to be a disaster. Now they're not all developers, but if you think about it, um, everything from how you do the interviewing process, uh, to how you get people, their badges, to how they get their equipment. Um, security is a, is another issue that they called out typically, uh, it security, um, the security of, of developers machines ends at, at, at the corporate desktop. >>But, you know, since we're increasingly using our own machines, our own hardware, um, security organizations kind of have to extend their security policies to cover, uh, employee devices, and that's caused them to scramble a little bit. Uh, so, so the examples are out there. It's not a lot of, like, we have to do everything completely differently, but it's a lot of subtle changes that, that have to be made. Um, I'll give you another example. Um, one of the things that, that we are seeing is that, um, more and more organizations to deal with the challenges around agility, with respect to delivering software, embracing low-code tools. In fact, uh, we see about 50% of firms are using low-code tools right now. We predict it's going to be 75% by the end of next year. So figuring out how your dev ops processes support an organization that might be using Mendix or OutSystems, or, you know, the power platform building the front end of an application, like a track and trace application really, really quickly, but then hooking it up to your backend infrastructure. Does that happen completely outside the dev ops investments that you're making and the agile processes that you're making, or do you adapt your organization? Um, our hybrid teams now teams that not just have professional developers, but also have business users that are doing some development with a low-code tool. Those are the kinds of things that we have to be, um, willing to, um, to entertain in order to shift the focus a little bit more toward the agility side, I think >>Lot of obstacles, but also a lot of opportunities for businesses to really learn, pay attention here, pivot and grow, and hopefully some good opportunities for the developers and the business folks to just get better at what they're doing and learning to embrace spiritual co-location Jeffrey, thank you so much for joining us on the program today. Very insightful conversation. >>My pleasure. It's it's, it's an important thing. Just remember if you're going to run that marathon, break it into 26, 10 minute runs, take a walk break in between each and you'll find that you'll get there. >>Digestible components, wise advice. Jeffery Hammond. Thank you so much for joining for Jeffrey I'm Lisa Martin, you're watching Broadcom's dev ops virtual forum >>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom, >>Continuing our conversations here at Broadcom's dev ops virtual forum. Lisa Martin here, please. To welcome back to the program, Serge Lucio, the general manager of the enterprise software division at Broadcom. Hey, Serge. Welcome. Thank you. Good to be here. So I know you were just, uh, participating with the biz ops manifesto that just happened recently. I just had the chance to talk with Jeffrey Hammond and he unlocked this really interesting concept, but I wanted to get your thoughts on spiritual co-location as really a necessity for biz ops to succeed in this unusual time in which we're living. What are your thoughts on spiritual colocation in terms of cultural change versus adoption of technologies? >>Yeah, it's a, it's, it's quite interesting, right? When we, when we think about the major impediments for, uh, for dev ops implementation, it's all about culture, right? And swore over the last 20 years, we've been talking about silos. We'd be talking about the paradox for these teams to when it went to align in many ways, it's not so much about these teams aligning, but about being in the same car in the same books, right? It's really about fusing those teams around kind of the common purpose, a common objective. So to me, the, this, this is really about kind of changing this culture where people start to look at a kind of OKR is instead of the key objective, um, that, that drives the entire team. Now, what it means in practice is really that's, uh, we need to change a lot of behaviors, right? It's not about the Yarki, it's not about roles. It's about, you know, who can do what and when, and, uh, you know, driving a bias towards action. It also means that we need, I mean, especially in this school times, it becomes very difficult, right? To drive kind of a kind of collaboration between these teams. And so I think there there's a significant role that especially tools can play in terms of providing this complex feedback from teams to, uh, to be in that preface spiritual qualification. >>Well, and it talked about culture being, it's something that, you know, we're so used to talking about dev ops with respect to velocity, all about speed here. But of course this time everything changed so quickly, but going from the physical spaces to everybody being remote really does take it. It's very different than you can't replicate it digitally, but there are collaboration tools that can kind of really be essential to help that cultural shift. Right? >>Yeah. So 2020, we, we touch to talk about collaboration in a very mundane way. Like, of course we can use zoom. We can all get into, into the same room. But the point when I think when Jeff says spiritual, co-location, it's really about, we all share the same objective. Do we, do we have a niece who, for instance, our pipeline, right? When you talk about dev ops, probably we all started thinking about this continuous delivery pipeline that basically drives the automation, the orchestration across the team, but just thinking about a pipeline, right, at the end of the day, it's all about what is the meantime to beat back to these teams. If I'm a developer and a commit code, I don't, does it take where, you know, that code to be processed through pipeline pushy? Can I get feedback if I am a finance person who is funding a product or a project, what is my meantime to beat back? >>And so a lot of, kind of a, when we think about the pipeline, I think what's been really inspiring to me in the last year or so is that there is much more of an adoption of the Dora metrics. There is way more of a focus around value stream management. And to me, this is really when we talk about collaboration, it's really a balance. How do you provide the feedback to the different stakeholders across the life cycle in a very timely matter? And that's what we would need to get to in terms of kind of this, this notion of collaboration. It's not so much about people being in the same physical space. It's about, you know, when I checked in code, you know, to do I guess the system to automatically identify what I'm going to break. If I'm about to release some allegation, how can the system help me reduce my change pillar rates? Because it's, it's able to predict that some issue was introduced in the outpatient or work product. Um, so I think there's, there's a great role of technology and AI candidate Lynch to, to actually provide that new level of collaboration. >>So we'll get to AI in a second, but I'm curious, what are some of the, of the metrics you think that really matter right now is organizations are still in some form of transformation to this new almost 100% remote workforce. >>So I'll just say first, I'm not a big fan of metrics. Um, and the reason being that, you know, you can look at a change killer rate, right, or a lead time or cycle time. And those are, those are interesting metrics, right? The trend on metric is absolutely critical, but what's more important is you get to the root cause what is taught to you lean to that metric to degrade or improve or time. And so I'm much more interested and we, you know, fruit for Broadcom. Are we more interested in understanding what are the patterns that contribute to this? So I'll give you a very mundane example. You know, we know that cycle time is heavily influenced by, um, organizational boundaries. So, you know, we talk a lot about silos, but, uh, we we've worked with many of our customers doing value stream mapping. And oftentimes what you see is that really the boundaries of your organization creates a lot of idle time, right? So to me, it's less about the metrics. I think the door metrics are a pretty, you know, valid set metrics, but what's way more important is to understand what are the antiperspirants, what are the things that we can detect through the data that actually are affecting those metrics. And, uh, I mean, over the last 10, 20 years, we've learned a lot about kind of what are, what are the antiperspirants within our large enterprise customers. And there are plenty of them. >>What are some of the things that you're seeing now with respect to patterns that have developed over the last seven to eight months? >>So I think the two areas which clearly are evolving very quickly are on kind of the front end of the life cycle, where DevOps is more and more embracing value stream management value stream mapping. Um, and I think what's interesting is that in many ways the product is becoming the new silo. Uh, the notion of a product is very difficult by itself to actually define people are starting to recognize that a value stream is not its own little kind of Island. That in reality, when I define a product, this product, oftentimes as dependencies on our products and that in fact, you're looking at kind of a network of value streams, if you will. So, so even on that, and there is clearly kind of a new sets, if you will, of anti-patterns where products are being defined as a set of OTRs, they have interdependencies and you have have a new set of silos on the operands, uh, the Abra key movement to Israel and the SRE space where, um, I think there is a cultural clash while the dev ops side is very much embracing this notion of OTRs and value stream mapping and Belgium management. >>On the other end, you have the it operations teams. We still think business services, right? For them, they think about configure items, think about infrastructure. And so, you know, it's not uncommon to see, you know, teams where, you know, the operations team is still thinking about hundreds of thousands, tens of thousands of business services. And so the, the, there is there's this boundary where, um, I think, well, SRE is being put in place. And there's lots of thinking about what kind of metrics can be fined. I think, you know, going back to culture, I think there's a lot of cultural evolution that's still required for true operations team. >>And that's a hard thing. Cultural transformation in any industry pandemic or not is a challenging thing. You talked about, uh, AI and automation of minutes ago. How do you think those technologies can be leveraged by DevOps leaders to influence their successes and their ability to collaborate, maybe see eye to eye with the SRS? >>Yeah. Um, so th you're kind of too. So even for myself, as a leader of a, you know, 1500 people organization, there's a number of things I don't see right. On a daily basis. And, um, I think the, the, the, the technologies that we have at our disposal today from the AI are able to mind a lot of data and expose a lot of, uh, issues that's as leaders we may not be aware of. And some of the, some of these are pretty kind of easy to understand, right? We all think we're agile. And yet when you, when you start to understand, for instance, uh, what is the, what is the working progress right to during the sprint? Um, when you start to analyze the data you can detect, for instance, that maybe the teams are over committed, that there is too much work in progress. >>You can start to identify kind of, interdepencies either from a technology, from a people point of view, which were hidden, uh, you can start to understand maybe the change filler rates he's he is dragging. So I believe that there is a, there's a fundamental role to be played by the tools to, to expose again, these anti parents, to, to make these things visible to the teams, to be able to even compare teams. Right. One of the things that's, that's, uh, that's amazing is now we have access to tons of data, not just from a given customer, but across a large number of customers. And so we start to compare all of these teams kind of operate, and what's working, what's not working >>Thoughts on AI and automation as, as a facilitator of spiritual co-location. >>Yeah, absolutely. Absolutely. It's um, you know, th there's, uh, the problem we all face is the unknown, right? The, the law city, but volume variety of the data, uh, everyday we don't really necessarily completely appreciate what is the impact of our actions, right? And so, um, AI can really act as a safety net that enables us to, to understand what is the impact of our actions. Um, and so, yeah, in many ways, the ability to be informed in a timely matter to be able to interact with people on the basis of data, um, and collaborate on the data. And the actual matter, I think is, is a, is a very powerful enabler, uh, on, in that respect. I mean, I, I've seen, um, I've seen countless of times that, uh, for instance, at the SRE boundary, um, to basically show that we'll turn the quality attributes, so an incoming release, right. And exposing that to, uh, an operations person and a sorry person, and enabling that collaboration dialogue through data is a very, very powerful tool. >>Do you have any recommendations for how teams can use, you know, the SRE folks, the dev ops says can use AI and automation in the right ways to be successful rather than some ways that aren't going to be nonproductive. >>Yeah. So to me, the th there, there's a part of the question really is when, when we talk about data, there are there different ways you can use data, right? Um, so you can, you can do a lot of an analytics, predictive analytics. So I think there is a, there's a tendency, uh, to look at, let's say a, um, a specific KPI, like a, an availability KPI, or change filler rate, and to basically do a regression analysis and projecting all these things, going to happen in the future. To me, that that's, that's a, that's a bad approach. The reason why I fundamentally think it's a better approach is because we are systems. The way we develop software is, is a, is a non-leader kind of system, right? Software development is not linear nature. And so I think there's a D this is probably the worst approach is to actually focus on metrics on the other end. >>Um, if you, if you start to actually understand at a more granular level, what har, uh, which are the things which are contributing to this, right? So if you start to understand, for instance, that whenever maybe, you know, you affect a specific part of the application that translates into production issues. So we, we have, I've actually, uh, a customer who, uh, identified that, uh, over 50% of their unplanned outages were related to specific components in your architecture. And whenever these components were changed, this resulted in these plant outages. So if you start to be able to basically establish causality, right, cause an effect between kind of data across the last cycle. I think, I think this is the right way to, uh, to, to use AI. And so pharma to be, I think it's way more God could have a classification problem. What are the classes of problems that do exist and affect things as opposed to analytics, predictive, which I don't think is as powerful. >>So I mentioned in the beginning of our conversation, that just came off the biz ops manifesto. You're one of the authors of that. I want to get your thoughts on dev ops and biz ops overlapping, complimenting each other, what, from a, the biz ops perspective, what does it mean to the future of dev ops? >>Yeah, so, so it's interesting, right? If you think about DevOps, um, there's no felony document, right? Can we, we can refer to the Phoenix project. I mean, there are a set of documents which have been written, but in many ways, there's no clear definition of what dev ops is. Uh, if you go to the dev ops Institute today, you'll see that they are specific, um, trainings for instance, on value management on SRE. And so in many ways, the problem we have as an industry is that, um, there are set practices between agile dev ops, SRE Valley should management. I told, right. And we all basically talk about the same things, right. We all talk about essentially, um, accelerating in the meantime fee to feedback, but yet we don't have the common framework to talk about that. The other key thing is that we add to wait, uh, for, uh, for jeans, Jean Kim's Lascaux, um, to, uh, to really start to get into the business aspect, right? >>And for value stream mapping to start to emerge for us to start as an industry, right. It, to start to think about what is our connection with the business aspect, what's our purpose, right? And ultimately it's all about driving these business outcomes. And so to me, these ops is really about kind of, uh, putting a lens on this critical element that it's not business and it, that we in fact need to fuse business 19 that I need needs to transform itself to recognize that it's, it's this value generator, right. It's not a cost center. And so the relationship to me, it's more than BizOps provides kind of this Oliver or kind of framework, if you will. That set the context for what is the reason, uh, for it to exist. What's part of the core values and principles that it needs to embrace to, again, change from a cost center to a value center. And then we need to start to use this as a way to start to unify some of the, again, the core practices, whether it's agile, DevOps value, stream mapping SRE. Um, so, so I think over time, my hope is that we start to optimize a lot of our practices, language, um, and, uh, and cultural elements. >>Last question surgeon, the last few seconds we have here talking about this, the relation between biz ops and dev ops, um, what do you think as DevOps evolves? And as you talked to circle some of your insights, what should our audience keep their eyes on in the next six to 12 months? >>So to me, the key, the key, um, challenge for, for the industry is really around. So we were seeing a very rapid shift towards kind of, uh, product to product, right. Which we don't want to do is to recreate kind of these new silos, these hard silos. Um, so that, that's one of the big changes, uh, that I think we need to be, uh, to be really careful about, um, because it is ultimately, it is about culture. It's not about, uh, it's not about, um, kind of how we segment the work, right. And, uh, any true culture that we can overcome kind of silos. So back to, I guess, with Jeffrey's concept of, um, kind of the spiritual co-location, I think it's, it's really about that too. It's really about kind of, uh, uh, focusing on the business outcomes on kind of aligning on driving engagement across the teams, but, but not for create a, kind of a new set of silos, which instead of being vertical are going to be these horizontal products >>Crazy by surge that looking at culture as kind of a way of really, uh, uh, addressing and helping to, uh, re re reduce, replace challenges. We thank you so much for sharing your insights and your time at today's DevOps virtual forum. >>Thank you. Thanks for your time. >>I'll be right back >>From around the globe it's the cube with digital coverage of devops virtual forum brought to you by Broadcom. >>Welcome to Broadcom's DevOps virtual forum, I'm Lisa Martin, and I'm joined by another Martin, very socially distanced from me all the way coming from Birmingham, England is Glynn Martin, the head of QA transformation at BT. Glynn, it's great to have you on the program. Thank you, Lisa. I'm looking forward to it. As we said before, we went live to Martins for the person one in one segment. So this is going to be an interesting segment guys, what we're going to do is Glynn's going to give us a really kind of deep inside out view of devops from an evolution perspective. So Glynn, let's start. Transformation is at the heart of what you do. It's obviously been a very transformative year. How have the events of this year affected the >> transformation that you are still responsible for driving? Yeah. Thank you, Lisa. I mean, yeah, it has been a difficult year. >>Um, and although working for BT, which is a global telecommunications company, um, I'm relatively resilient, I suppose, as a, an industry, um, through COVID obviously still has been affected and has got its challenges. And if anything, it's actually caused us to accelerate our transformation journey. Um, you know, we had to do some great things during this time around, um, you know, in the UK for our emergency and, um, health workers give them unlimited data and for vulnerable people to support them. And that's spent that we've had to deliver changes quickly. Um, but what we want to be able to do is deliver those kinds of changes quickly, but sustainably for everything that we do, not just because there's an emergency. Um, so we were already on the kind of journey to agile, but ever more important now that we are, we are able to do those, that kind of work, do it more quickly. >>Um, and that it works because the, the implications of it not working is, can be terrible in terms of you know, we've been supporting testing centers,  new hospitals to treat COVID patients. So we need to get it right. And then therefore the coverage of what we do, the quality of what we do and how quickly we do it really has taken on a new scale and what was already a very competitive market within the telco industry within the UK. Um, you know, what I would say is that, you know, we are under pressure to deliver more value, but we have small cost challenges. We have to obviously, um, deal with the fact that, you know, COVID 19 has hit most industries kind of revenues and profits. So we've got this kind of paradox between having less costs, but having to deliver more value quicker and  to higher quality. So yeah, certainly the finances is, um, on our minds and that's why we need flexible models, cost models that allow us to kind of do growth, but we get that growth by showing that we're delivering value. Um, especially in these times when there are financial challenges on companies. So one of the things that I want to ask you about, I'm again, looking at DevOps from the inside >>Out and the evolution that you've seen, you talked about the speed of things really accelerating in this last nine months or so. When we think dev ops, we think speed. But one of the things I'd love to get your perspective on is we've talked about in a number of the segments that we've done for this event is cultural change. What are some of the things that you've seen there as, as needing to get, as you said, get things right, but done so quickly to support essential businesses, essential workers. How have you seen that cultural shift? >>Yeah, I think, you know, before test teams for themselves at this part of the software delivery cycle, um, and actually now really our customers are expecting that quality and to deliver for our customers what they want, quality has to be ingrained throughout the life cycle. Obviously, you know, there's lots of buzzwords like shift left. Um, how do we do shift left testing? Um, but for me, that's really instilling quality and given capabilities shared capabilities throughout the life cycle that drive automation, drive improvements. I always say that, you know, you're only as good as your lowest common denominator. And one thing that we were finding on our dev ops journey was that we  would be trying to do certain things quick, we had automated build, automated tests. But if we were taking a weeks to create test scripts, or we were taking weeks to manually craft data, and even then when we had taken so long to do it, that the coverage was quite poor and that led to lots of defects later on in the life cycle, or even in our production environment, we just couldn't afford to do that. >>And actually, focusing on continuous testing over the last nine to 12 months has really given us the ability to deliver quickly across the whole life cycle. And therefore actually go from doing a kind of semi agile kind of thing, where we did the user stories, we did a few of the kind of agile ceremonies, but we weren't really deploying any quicker into production because our stakeholders were scared that we didn't have the same control that we had when we had more waterfall releases. And, you know, when we didn't think of ourselves. So we've done a lot of work on every aspect, um, especially from a testing point of view, every aspect of every activity, rather than just looking at automated tests, you know, whether it is actually creating the test in the first place, whether it's doing security testing earlier in the lot and performance testing in the life cycle, et cetera. So, yeah,  it's been a real key thing that for CT, for us to drive DevOps, >>Talk to me a little bit about your team. What are some of the shifts in terms of expectations that you're experiencing and how your team interacts with the internal folks from pipeline through life cycle? >>Yeah, we've done a lot of work on this. Um, you know, there's a thing that I think people will probably call it a customer experience gap, and it reminds me of a Gilbert cartoon, where we start with the requirements here and you're almost like a Chinese whisper effects and what we deliver is completely different. So we think the testing team or the delivery teams, um, know in our teeth has done a great job. This is what it said in the acceptance criteria, but then our customers are saying, well, actually that's not working this isn't working and there's this kind of gap. Um, we had a great launch this year of agile requirements, it's one of the Broadcom tools. And that was the first time in, ever since I remember actually working within BT, I had customers saying to me, wow, you know, we want more of this. >>We want more projects to have extra requirements design on it because it allowed us to actually work with the business collaboratively. I mean, we talk about collaboration, but how do we actually, you know, do that and have something that both the business and technical people can understand. And we've actually been working with the business , using agile requirements designer to really look at what the requirements are, tease out requirements we hadn't even thought of and making sure that we've got high levels of test coverage. And what we actually deliver at the end of it, not only have we been able to generate tests more quickly, but we've got much higher test coverage and also can more smartly, using the kind of AI within the tool and then some of the other kinds of pipeline tools, actually deliver to choose the right tasks, and actually doing a risk based testing approach. So that's been a great launch this year, but just the start of many kinds of things that we're doing >>Well, what I hear in that, Glynn is a lot of positives that have come out of a very challenging situation. Talk to me about it. And I liked that perspective. This is a very challenging time for everybody in the world, but it sounds like from a collaboration perspective you're right, we talk about that a lot critical with devops. But those challenges there, you guys were able to overcome those pretty quickly. What other challenges did you face and figure out quickly enough to be able to pivot so fast? >>I mean, you talked about culture. You know, BT is like most companies  So it's very siloed. You know we're still trying to work to become closer as a company. So I think there's a lot of challenges around how would you integrate with other tools? How would you integrate with the various different technologies. And BT, we have 58 different IT stacks. That's not systems, that's stacks, all of those stacks can have hundreds of systems. And we're trying to, we've got a drive at the moment, a simplified program where we're trying to you know, reduce that number to 14 stacks. And even then there'll be complexity behind the scenes that we will be challenged more and more as we go forward. How do we actually highlight that to our users? And as an it organization, how do we make ourselves leaner, so that even when we've still got some of that legacy, and we'll never fully get rid of it and that's the kind of trade off that we have to make, how do we actually deal with that and hide that from our users and drive those programs, so we can, as I say, accelerate change,  reduce that kind of waste and that kind of legacy costs out of our business. You know, the other thing as well, I'm sure telecoms is probably no different to insurance or finance. When you take the number of products that we do, and then you combine them, the permutations are tens and hundreds of thousands of products. So we, as a business are trying to simplify, we are trying to do that in an agile way. >>And haven't tried to do agile in the proper way and really actually work at pace, really deliver value. So I think what we're looking more and more at the moment is actually  more value focused. Before we used to deliver changes sometimes into production. Someone had a great idea, or it was a great idea nine months ago or 12 months ago, but actually then we ended up deploying it and then we'd look at the users, the usage of that product or that application or whatever it is, and it's not being used for six months. So we haven't got, you know, the cost of the last 12 months. We certainly haven't gotten room for that kind of waste and, you know, for not really understanding the value of changes that we are doing. So I think that's the most important thing of the moment, it's really taking that waste out. You know, there's lots of focus on things like flow management, what bits of our process are actually taking too long. And we've started on that journey, but we've got a hell of a long way to go. But that involves looking at every aspect of the software delivery cycle. >> Going from, what 58 IT stacks down to 14 or whatever it's going to be, simplifying sounds magical to everybody. It's a big challenge. What are some of the core technology capabilities that you see really as kind of essential for enabling that with this new way that you're working? >>Yeah. I mean, I think we were started on a continuous testing journey, and I think that's just the start. I mean as I say, looking at every aspect of, you know, from a QA point of view is every aspect of what we do. And it's also looking at, you know, we've started to branch into more like AI, uh, AI ops and, you know, really the full life cycle. Um, and you know, that's just a stepping stone to, you know, I think autonomics is the way forward, right. You know, all of this kind of stuff that happens, um, you know, monitoring, uh, you know, watching the systems what's happening in production, how do we feed that back? How'd you get to a point where actually we think about change and then suddenly it's in production safely, or if it's not going to safety, it's automatically backing out. So, you know, it's a very, very long journey, but if we want to, you know, in a world where the pace is in ever-increasing and the demands for the team, and, you know, with the pressures on, at the moment where we're being asked to do things, uh, you know, more efficiently and as lean as possible, we need to be thinking about every part of the process and how we put the kind of stepping stones in place to lead us to a more automated kind of, um, you know, um, the future. >>Do you feel that that planned outcomes are starting to align with what's delivered, given this massive shift that you're experiencing? >>I think it's starting to, and I think, you know, as I say, as we look at more of a value based approach, um, and, um, you know, as I say, print, this was a kind of flow management. I think that that will become ever, uh, ever more important. So, um, I think it starting to people certainly realize that, you know, teams need to work together, you know, the kind of the cousin between business and it, especially as we go to more kind of SAS based solutions, low code solutions, you know, there's not such a gap anymore, actually, some of our business partners that expense to be much more tech savvy. Um, so I think, you know, this is what we have to kind of appreciate what is its role, how do we give the capabilities, um, become more of a centers of excellence rather than actually doing mounds amounts of work. And for me, and from a testing point of view, you know, mounds and mounds of testing, actually, how do we automate that? How do we actually generate that instead of, um, create it? I think that's the kind of challenge going forward. >>What are some, as we look forward, what are some of the things that you would like to see implemented or deployed in the next, say six to 12 months as we hopefully round a corner with this pandemic? >>Yeah, I think, um, you know, certainly for, for where we are as a company from a QA perspective, we are, um, you let's start in bits that we do well, you know, we've started creating, um, continuous delivery and DevOps pipelines. Um, there's still manual aspects of that. So, you know, certainly for me, I I've challenged my team with saying how do we do an automated journey? So if I put a requirement in JIRA or rally or wherever it is and why then click a button and, you know, with either zero touch for one such, then put that into production and have confidence that, that has been done safely and that it works and what happens if it doesn't work. So, you know, that's, that's the next, um, the next few months, that's what our concentration, um, is, is about. But it's also about decision-making, you know, how do you actually understand those value judgments? >>And I think there's lots of the things dev ops, AI ops, kind of that always ask aspects of business operations. I think it's about having the information in one place to make those kinds of decisions. How does it all try and tie it together? As I say, even still with kind of dev ops, we've still got elements within my company where we've got lots of different organizations doing some, doing similar kinds of things, but they're all kind of working in silos. So I think having AI ops as it comes more and more to the fore as we go to cloud, and that's what we need to, you know, we're still very early on in our cloud journey, you know, so we need to make sure the technologies work with cloud as well as you can have, um, legacy systems, but it's about bringing that all together and having a full, visible pipeline, um, that everybody can see and make decisions. >>You said the word confidence, which jumped out at me right away, because absolutely you've got to have be able to have confidence in what your team is delivering and how it's impacting the business and those customers. Last question then for you is how would you advise your peers in a similar situation to leverage technology automation, for example, dev ops, to be able to gain the confidence that they're making the right decisions for their business? >>I think the, the, the, the, the approach that we've taken actually is not started with technology. Um, we've actually taken a human centered design, uh, as a core principle of what we do, um, within the it part of BT. So by using human centered design, that means we talk to our customers, we understand their pain points, we map out their current processes. Um, and then when we mapped out what this process does, it also understand their aspirations as well, you know? Um, and where do they want to be in six months? You know, do they want it to be, um, more agile and, you know, or do they want to, you know, is, is this a part of their business that they want to do one better? We actually then looked at why that's not running well, and then see what, what solutions are out there. >>We've been lucky that, you know, with our partnership, with Broadcom within the payer line, lots of the tools and the PLA have directly answered some of the business's problems. But I think by having those conversations and actually engaging with the business, um, you know, especially if the business hold the purse strings, which in, in, uh, you know, in some companies include not as they do there is that kind of, you know, almost by understanding their, their pain points and then starting, this is how we can solve your problem. Um, is we've, we've tended to be much more successful than trying to impose something and say, well, here's the technology that they don't quite understand. It doesn't really understand how it kind of resonates with their problems. So I think that's the heart of it. It's really about, you know, getting, looking at the data, looking at the processes, looking at where the kind of waste is. >>And then actually then looking at the right solutions. Then, as I say, continuous testing is massive for us. We've also got a good relationship with Apple towards looking at visual AI. And actually there's a common theme through that. And I mean, AI is becoming more and more prevalent. And I know, you know, sometimes what is AI and people have kind of this semantics of, is it true AI or not, but it's certainly, you know, AI machine learning is becoming more and more prevalent in the way that we work. And it's allowing us to be much more effective, be quicker in what we do and be more accurate. And, you know, whether it's finding defects running the right tests or, um, you know, being able to anticipate problems before they're happening in a production environment. >>Well, thank you so much for giving us this sort of insight outlook at dev ops sharing the successes that you're having, taking those challenges, converting them to opportunities and forgiving folks who might be in your shoes, or maybe slightly behind advice enter. They appreciate it. We appreciate your time. >>Well, it's been an absolute pleasure, really. Thank you for inviting me. I have a extremely enjoyed it. So thank you ever so much. >>Excellent. Me too. I've learned a lot for Glenn Martin. I'm Lisa Martin. You're watching the cube >>Driving revenue today means getting better, more valuable software features into the hands of your customers. If you don't do it quickly, your competitors as well, but going faster without quality creates risks that can damage your brand destroy customer loyalty and cost millions to fix dev ops from Broadcom is a complete solution for balancing speed and risk, allowing you to accelerate the flow of value while minimizing the risk and severity of critical issues with Broadcom quality becomes integrated across the entire DevOps pipeline from planning to production, actionable insights, including our unique readiness score, provide a three 60 degree view of software quality giving you visibility into potential issues before they become disasters. Dev ops leaders can manage these risks with tools like Canary deployments tested on a small subset of users, or immediately roll back to limit the impact of defects for subsequent cycles. Dev ops from Broadcom makes innovation improvement easier with integrated planning and continuous testing tools that accelerate the flow of value product requirements are used to automatically generate tests to ensure complete quality coverage and tests are easily updated. >>As requirements change developers can perform unit testing without ever leaving their preferred environment, improving efficiency and productivity for the ultimate in shift left testing the platform also integrates virtual services and test data on demand. Eliminating two common roadblocks to fast and complete continuous testing. When software is ready for the CIC CD pipeline, only DevOps from Broadcom uses AI to prioritize the most critical and relevant tests dramatically improving feedback speed with no decrease in quality. This release is ready to go wherever you are in your DevOps journey. Broadcom helps maximize innovation velocity while managing risk. So you can deploy ideas into production faster and release with more confidence from around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi guys. Welcome back. So we have discussed the current state and the near future state of dev ops and how it's going to evolve from three unique perspectives. In this last segment, we're going to open up the floor and see if we can come to a shared understanding of where dev ops needs to go in order to be successful next year. So our guests today are, you've seen them all before Jeffrey Hammond is here. The VP and principal analyst serving CIO is at Forester. We've also Serge Lucio, the GM of Broadcom's enterprise software division and Glenn Martin, the head of QA transformation at BT guys. Welcome back. Great to have you all three together >>To be here. >>All right. So we're very, we're all very socially distanced as we've talked about before. Great to have this conversation. So let's, let's start with one of the topics that we kicked off the forum with Jeff. We're going to start with you spiritual co-location that's a really interesting topic that we've we've uncovered, but how much of the challenge is truly cultural and what can we solve through technology? Jeff, we'll start with you then search then Glen Jeff, take it away. >>Yeah, I think fundamentally you can have all the technology in the world and if you don't make the right investments in the cultural practices in your development organization, you still won't be effective. Um, almost 10 years ago, I wrote a piece, um, where I did a bunch of research around what made high-performance teams, software delivery teams, high performance. And one of the things that came out as part of that was that these teams have a high level of autonomy. And that's one of the things that you see coming out of the agile manifesto. Let's take that to today where developers are on their own in their own offices. If you've got teams where the team itself had a high level of autonomy, um, and they know how to work, they can make decisions. They can move forward. They're not waiting for management to tell them what to do. >>And so what we have seen is that organizations that embraced autonomy, uh, and got their teams in the right place and their teams had the information that they needed to make the right decisions have actually been able to operate pretty well, even as they've been remote. And it's turned out to be things like, well, how do we actually push the software that we've created into production that would become the challenge is not, are we writing the right software? And that's why I think the term spiritual co-location is so important because even though we may be physically distant, we're on the same plane, we're connected from a, from, from a, a shared purpose. Um, you know, surgeon, I worked together a long, long time ago. So it's been what almost 15, 16 years since we were at the same place. And yet I would say there's probably still a certain level of spiritual co-location between us, uh, because of the shared purposes that we've had in the past and what we've seen in the industry. And that's a really powerful tool, uh, to build on. So what do tools play as part of that, to the extent that tools make information available, to build shared purpose on to the extent that they enable communication so that we can build that spiritual co-location to the extent that they reinforce the culture that we want to put in place, they can be incredibly valuable, especially when, when we don't have the luxury of physical locate physical co-location. Okay. That makes sense. >>It does. I shouldn't have introduced us. This last segment is we're all spiritually co-located or it's a surge, clearly you're still spiritually co located with jump. Talk to me about what your thoughts are about spiritual of co-location the cultural impact and how technology can move it forward. >>Yeah. So I think, well, I'm going to sound very similar to Jeff in that respect. I think, you know, it starts with kind of a shared purpose and the other understanding, Oh, individuals teams, uh, contributed to kind of a business outcome, what is our shared goal or shared vision? What's what is it we're trying to achieve collectively and keeping it kind of aligned to that? Um, and so, so it's really starts with that now, now the big challenge, always these over the last 20 years, especially in large organization, there's been specialization of roles and functions. And so we, we all that started to basically measure which we do, uh, on a daily basis using metrics, which oftentimes are completely disconnected from kind of a business outcome or purpose. We, we kind of reverted back to, okay, what is my database all the time? What is my cycle time? >>Right. And, and I think, you know, which we can do or where we really should be focused as an industry is to start to basically provide a lens or these different stakeholders to look at what they're doing in the context of kind of these business outcomes. So, um, you know, probably one of my, um, favorites experience was to actually weakness at one of a large financial institution. Um, you know, Tuesday Golder's unquote development and operations staring at the same data, right. Which was related to, you know, in calming changes, um, test execution results, you know, Coverity coverage, um, official liabilities and all the all ran. It could have a direction level links. And that's when you start to put these things in context and represent that to you in a way that these different stakeholders can, can look at from their different lens. And, uh, and it can start to basically communicate and, and understand have they joined our company to, uh, to, to that kind of common view or objective. >>And Glen, we talked a lot about transformation with you last time. What are your thoughts on spiritual colocation and the cultural part, the technology impact? >>Yeah, I mean, I agree with Jeffrey that, you know, um, the people and culture, the most important thing, actually, that's why it's really important when you're transforming to have partners who have the same vision as you, um, who, who you can work with, have the same end goal in mind. And w I've certainly found that with our, um, you know, continuing relationship with Broadcom, what it also does though, is although, you know, tools can accelerate what you're doing and can join consistency. You know, we've seen within simplify, which is BTS flagship transformation program, where we're trying to, as it can, it says simplify the number of systems stacks that we have, the number of products that we have actually at the moment, we've got different value streams within that program who have got organizational silos. We were trying to rewrite, rewrite the wheel, um, who are still doing things manually. >>So in order to try and bring that consistency, we need the right tools that actually are at an enterprise grade, which can be flexible to work with in BT, which is such a complex and very dev, uh, different environments, depending on what area of BT you're in, whether it's a consumer, whether it's a mobile area, whether it's large global or government organizations, you know, we found that we need tools that can, um, drive that consistency, but also flex to Greenfield brownfield kind of technologies as well. So it's really important that as I say, for a number of different aspects, that you have the right partner, um, to drive the right culture, I've got the same vision, but also who have the tool sets to help you accelerate. They can't do that on their own, but they can help accelerate what it is you're trying to do in it. >>And a really good example of that is we're trying to shift left, which is probably a, quite a bit of a buzz phrase in their kind of testing world at the moment. But, you know, I could talk about things like continuous delivery direct to when a ball comes tools and it has many different features to it, but very simply on its own, it allows us to give the visibility of what the teams are doing. And once we have that visibility, then we can talk to the teams, um, around, you know, could they be doing better component testing? Could they be using some virtualized services here or there? And that's not even the main purpose of continuous delivery director, but it's just a reason that tools themselves can just give greater visibility of have much more intuitive and insightful conversations with other teams and reduce those organizational silos. >>Thanks, Ben. So we'd kind of sum it up, autonomy collaboration tools that facilitate that. So let's talk now about metrics from your perspectives. What are the metrics that matter? Jeff, >>I'm going to go right back to what Glenn said about data that provides visibility that enables us to, to make decisions, um, with shared purpose. And so business value has to be one of the first things that we look at. Um, how do we assess whether we have built something that is valuable, you know, that could be sales revenue, it could be net promoter score. Uh, if you're not selling what you've built, it could even be what the level of reuse is within your organization or other teams picking up the services, uh, that you've created. Um, one of the things that I've begun to see organizations do is to align value streams with customer journeys and then to align teams with those value streams. So that's one of the ways that you get to a shared purpose, cause we're all trying to deliver around that customer journey, the value with it. >>And we're all measured on that. Um, there are flow metrics which are really important. How long does it take us to get a new feature out from the time that we conceive it to the time that we can run our first experiments with it? There are quality metrics, um, you know, some of the classics or maybe things like defect, density, or meantime to response. Um, one of my favorites came from a, um, a company called ultimate software where they looked at the ratio of defects found in production to defects found in pre production and their developers were in fact measured on that ratio. It told them that guess what quality is your job to not just the test, uh, departments, a group, the fourth level that I think is really important, uh, in, in the current, uh, situation that we're in is the level of engagement in your development organization. >>We used to joke that we measured this with the parking lot metric helpful was the parking lot at nine. And how full was it at five o'clock. I can't do that anymore since we're not physically co-located, but what you can do is you can look at how folks are delivering. You can look at your metrics in your SCM environment. You can look at, uh, the relative rates of churn. Uh, you can look at things like, well, are our developers delivering, uh, during longer periods earlier in the morning, later in the evening, are they delivering, uh, you know, on the weekends as well? Are those signs that we might be heading toward a burnout because folks are still running at sprint levels instead of marathon levels. Uh, so all of those in combination, uh, business value, uh, flow engagement in quality, I think form the backbone of any sort of, of metrics, uh, a program. >>The second thing that I think you need to look at is what are we going to do with the data and the philosophy behind the data is critical. Um, unfortunately I see organizations where they weaponize the data and that's completely the wrong way to look at it. What you need to do is you need to say, you need to say, how is this data helping us to identify the blockers? The things that aren't allowing us to provide the right context for people to do the right thing. And then what do we do to remove those blockers, uh, to make sure that we're giving these autonomous teams the context that they need to do their job, uh, in a way that creates the most value for the customers. >>Great advice stuff, Glenn, over to your metrics that matter to you that really make a big impact. And, and, and also how do you measure quality kind of following onto the advice that Jeff provided? >>That's some great advice. Actually, he talks about value. He talks about flow. Both of those things are very much on my mind at the moment. Um, but there was this, I listened to a speaker, uh, called me Kirsten a couple of months ago. It taught very much around how important flow management is and removing, you know, and using that to remove waste, to understand in terms of, you know, making software changes, um, what is it that's causing us to do it longer than we need to. So where are those areas where it takes long? So I think that's a very important thing for us. It's even more basic than that at the moment, we're on a journey from moving from kind of a waterfall to agile. Um, and the problem with moving from waterfall to agile is with waterfall, the, the business had a kind of comfort that, you know, everything was tested together and therefore it's safer. >>Um, and with agile, there's that kind of, you know, how do we make sure that, you know, if we're doing things quick and we're getting stuff out the door that we give that confidence, um, that that's ready to go, or if there's a risk that we're able to truly articulate what that risk is. So there's a bit about release confidence, um, and some of the metrics around that and how, how healthy those releases are, and actually saying, you know, we spend a lot of money, um, um, an investment setting up our teams, training our teams, are we actually seeing them deliver more quickly and are we actually seeing them deliver more value quickly? So yeah, those are the two main things for me at the moment, but I think it's also about, you know, generally bringing it all together, the dev ops, you know, we've got the kind of value ops AI ops, how do we actually bring that together to so we can make quick decisions and making sure that we are, um, delivering the biggest bang for our buck, absolutely biggest bang for the buck, surge, your thoughts. >>Yeah. So I think we all agree, right? It starts with business metrics, flow metrics. Um, these are kind of the most important metrics. And ultimately, I mean, one of the things that's very common across a highly functional teams is engagements, right? When, when you see a team that's highly functioning, that's agile, that practices DevOps every day, they are highly engaged. Um, that that's, that's definitely true. Now the, you know, back to, I think, uh, Jeff's point on weaponization of metrics. One of the key challenges we see is that, um, organizations traditionally have been kind of, uh, you know, setting up benchmarks, right? So what is a good cycle time? What is a good lead time? What is a good meantime to repair? The, the problem is that this is very contextual, right? It varies. It's going to vary quite a bit, depending on the nature of application and system. >>And so one of the things that we really need to evolve, um, as an industry is to understand that it's not so much about those flow metrics is about our, these four metrics ultimately contribute to the business metric to the business outcome. So that's one thing. The second aspect, I think that's oftentimes misunderstood is that, you know, when you have a bad cycle time or, or, or what you perceive as being a buy cycle time or better quality, the problem is oftentimes like all, do you go and explore why, right. What is the root cause of this? And I think one of the key challenges is that we tend to focus a lot of time on metrics and not on the eye type patterns, which are pretty common across the industry. Um, you know, if you look at, for instance, things like lead time, for instance, it's very common that, uh, organizational boundaries are going to be a key contributor to badly time. >>And so I think that there is, you know, the only the metrics there is, I think a lot of work that we need to do in terms of classifying, descend type patterns, um, you know, back to you, Jeff, I think you're one of the cool offers of waterscrumfall as a, as, as a key pattern, the industry or anti-spatter. Um, but waterscrumfall right is a key one, right? And you will detect that through kind of a defect arrival rates. That's where that looks like an S-curve. And so I think it's beyond kind of the, the metrics is what do you do with those metrics? >>Right? I'll tell you a search. One of the things that is really interesting to me in that space is I think those of us had been in industry for a long time. We know the anti-patterns cause we've seen them in our career maybe in multiple times. And one of the things that I think you could see tooling do is perhaps provide some notification of anti-patterns based on the telemetry that comes in. I think it would be a really interesting place to apply, uh, machine learning and reinforcement learning techniques. Um, so hopefully something that we'd see in the future with dev ops tools, because, you know, as a manager that, that, you know, may be only a 10 year veteran or 15 year veteran, you may be seeing these anti-patterns for the first time. And it would sure be nice to know what to do, uh, when they start to pop up, >>That would right. Insight, always helpful. All right, guys, I would like to get your final thoughts on this. The one thing that you believe our audience really needs to be on the lookout for and to put on our agendas for the next 12 months, Jeff will go back to you. Okay. >>I would say look for the opportunities that this disruption presents. And there are a couple that I see, first of all, uh, as we shift to remote central working, uh, we're unlocking new pools of talent, uh, we're, it's possible to implement, uh, more geographic diversity. So, so look to that as part of your strategy. Number two, look for new types of tools. We've seen a lot of interest in usage of low-code tools to very quickly develop applications. That's potentially part of a mainstream strategy as we go into 2021. Finally, make sure that you embrace this idea that you are supporting creative workers that agile and dev ops are the peanut butter and chocolate to support creative, uh, workers with algorithmic capabilities, >>Peanut butter and chocolate Glen, where do we go from there? What are, what's the one silver bullet that you think folks to be on the lookout for now? I, I certainly agree that, um, low, low code is, uh, next year. We'll see much more low code we'd already started going, moving towards a more of a SAS based world, but low code also. Um, I think as well for me, um, we've still got one foot in the kind of cow camp. Um, you know, we'll be fully trying to explore what that means going into the next year and exploiting the capabilities of cloud. But I think the last, um, the last thing for me is how do you really instill quality throughout the kind of, um, the, the life cycle, um, where, when I heard the word scrum fall, it kind of made me shut it because I know that's a problem. That's where we're at with some of our things at the moment we need to get beyond that. We need >>To be releasing, um, changes more frequently into production and actually being a bit more brave and having the confidence to actually do more testing in production and go straight to production itself. So expect to see much more of that next year. Um, yeah. Thank you. I haven't got any food analogies. Unfortunately we all need some peanut butter and chocolate. All right. It starts to take us home. That's what's that nugget you think everyone needs to have on their agendas? >>That's interesting. Right. So a couple of days ago we had kind of a latest state of the DevOps report, right? And if you read through the report, it's all about the lost city, but it's all about sweet. We still are receiving DevOps as being all about speed. And so to me, the key advice is in order to create kind of a spiritual collocation in order to foster engagement, we have to go back to what is it we're trying to do collectively. We have to go back to tie everything to the business outcome. And so for me, it's absolutely imperative for organizations to start to plot their value streams, to understand how they're delivering value into aligning everything they do from a metrics to deliver it, to flow to those metrics. And only with that, I think, are we going to be able to actually start to really start to align kind of all these roles across the organizations and drive, not just speed, but business outcomes, >>All about business outcomes. I think you guys, the three of you could write a book together. So I'll give you that as food for thought. Thank you all so much for joining me today and our guests. I think this was an incredibly valuable fruitful conversation, and we appreciate all of you taking the time to spiritually co-located with us today, guys. Thank you. Thank you, Lisa. Thank you. Thank you for Jeff Hammond serves Lucio and Glen Martin. I'm Lisa Martin. Thank you for watching the broad cops Broadcom dev ops virtual forum.