>> Hey, welcome back here everybody. It's Jeff Frick here with theCUBE. We're at PagerDuty Summit. It's in San Francisco at Pier 27. It's a new facility, we've never been here. It's pretty unique. It's right between the Bay bridge and Pier 39. Beautiful day out on the water and it's all about DevOps here at PagerDuty. And I'm going to tease Jen later if people even know what a pager is at this town. So we are excited to have Nicole Forsgren She's a founder at CEO and chief scientist of DevOps research and assessment. I had to read it, it's a big mouthful but it goes buy DORA for sure. Nicole, welcome to see you. Good to see you. >> Thanks so much. It's good to be here. >> Alright so you are the DevOps expert. You got a really interesting past. Did some research on the LinkedIn profile industry. Academe industry, Academe and now you're out helping people. >> Yes, bounce around a bit. It's all about the pivot right? >> Absolutely. >> Out here doing DevOps. >> Absolutely, absolutely so you do an annual report on the state of DevOps. So where are we? DevOps has been being talked about for a long, long time. How much is reality? How far are we on this journey? What are you seeing? >> Right so it's really interesting you point that out right, because for years everyone's been like DevOps. What is it? Does it matter? And so DORA and by the way, DORA is myself. Jess Humble, Jame Kim. We just brought on Sue Chow. But the core founders, we've partnered up with the team at Puppet, and for the last several years. We've put out the state of DevOps report. To kind of help define at least from a research standpoint and from our standpoint. What it is? What are the key contributors to really drive value and does it drive value? It's for years and I'll talk about this later this afternoon on my closing keynote. For years and when I say years, I mean decades of academic rigorous, pure review research. Technology didn't matter. Like it didn't matter at all. It just never delivered value to organizations. But then we started seeing patterns and really interesting patterns and companies saying no. We're seeing results, we're delivering value. We're delivering outcomes. Core essential outcomes for end users and customers in the business. And so we got together and say okay, let's really take a look at this in a really important way. >> Right, now how far we've come right. 'Cause now most companies are technology companies. They just happen to warp their technology around a particular product or a particular service. >> Yeah, exactly. >> And now most leading the technology in terms of a vehicle to drive value and to drive transformation. So DevOps is also very wrapped up in this whole concept of digital transformation. That's all anybody wants to talk about. It's in every earnings call, so how closely are the two related and how do you see, 'cause DevOps got a little bit more history in terms of the buzz of transformation. Are people applying DevOps concept beyond strictly development and operations? >> So, there's a lot to unpack there. So like you said, it's really, really involved. Although it has some kind of a buzz word, right? Some people love it, some people embrace it, some people never want to hear it. So it's really all about what's important to the company in delivering value. But it's core is really about taking important methodologies and practices to deliver value and it's about using technology and automation, in conjunction with core values and practices and processes that we've adopted from the lane and agile movements. >> Jeff: Right, right. And having a really good healthy culture that's about more than just DevOps. Right like you said. DevOps, QA, Info Sec. The business marrying all of that, pulling all of it together, working in conjunction in the right kind of ways to deliver value. To deliver key outcomes to help us pivot, move fast, learn, have fast feedback. So that we can do what we need to do for the company, for the business, because like you said, it's so many companies right now, really are technology organizations that happened to be wrapped around in some particular industry. >> Jeff: Right, right. >> Capital One is a financial institution. Really they are a technology organization that happens to do finance and deliver finance really, really well for their customers. So many other companies are doing retail but it's driven by technology. Right or they do insurance and it's driven by technology or they're a healthcare organizations that really can't do what they do unless they have technology to really drive it. >> Right, right. The financials institutions are interesting because if you talk to like my kids. If they've ever been inside of an actual bank and then and how often do they go to the atm? So not even atm, so the way that people more and more interact with the company is through digital mediums. >> Right. >> But I'm curious to get you're input on the big question that we always ask people is how do I get started. Right, what is the easy paths to success? How do I get some early success so I can build on that success? What's interesting is you have a very unique approach to solve that question as oppose to what I think or based on what I'm really good at, I think we should start here. >> Yes, we really do-- >> Do you guys have different-- >> And this is really why DORA exist and this is what we do. So myself Jess Humble, Jean Kim. This explains the genesis of DORA. So we have a couple different things so the mission of DORA is to help companies get better through science and proven methods. Ans so we have a couple of different things we do. The first is that state of DevOps report that we put together at Puppet. And those are all open sourced and so if you want some ideas of what really statistically drives improvement, go find those. They're open source, they're totally free. We've tried so many resources because we don't want companies to fail. We've all lived through that awful dot com mess. We've seen companies fail. Go find those resources. Now your question though, where should I start? If I'm a company, what should I do? We've all go into conferences myself, Jean, Jess and we've had companies come up and say well where should I start? And the answer is always, it depends. The answer is always it depends because I can't tell you absent context, absent data, absent information. If I don't know about someone's detail information. I can't tell you and so what we also have is we offer an assessment where I can collect data from the doers. Right there's this fantastic report from Forester. It's called the dangerous disconnect and that's such a great title because if you ask executives. They drastically over estimate technology and DevOps maturity in organizations. So you shouldn't be, I mean I love-- >> Over estimate. >> Of course they do. I mean because we need to be really, really optimistic about where our organizations are going. >> Right, right. >> Those are our roles as executives. And so that's appropriate but in certain conditions that's appropriate. But where it's not appropriate is when you're setting detail strategy for your organizations. And so what we do is we offer an assessment where using these strong scientifically based measure that we have prepared and refined over now, four years of rigorous academic research. We can go with a 15 minute survey, collect data from everyone in organization that like I said are the doers. DevOps, TestOps, QA, InfoSec including vendors, contractors, consultants to people that are in the weeds every single day. I can measure you. I can benchmark you against the industry. I've got over 23,000 data points around the world. All industries, all company sizes. And then, where should they start? I can algorithmically tell you what your bottle neck is, what your constraint is. Where you should start to accelerate your performance. >> Based on my data? >> Based on your data. >> Based on your algorithms and based on your population data from this huge data set >> Yes, and with the companies that we're working with right now, they're seeing amazing results. They're calling it out-sized results. So a really great example we have was with Capital One. They did the assessment across over a dozen lines of business. And by focusing on two core capabilities out of over 20. We focus them on the right two capabilities. They saw a 20X improvement in deploy frequency in only two months with zero increase in internet. >> 20% improvement-- >> 20X >> 20X? >> 20X >> In two months. >> 20 times. >> Wow. >> So it's that ability to measure consistently see visibility throughout that software engineering life cycle. So we also had feedback from customer like Verizon. That that visibility, that consistency of measurement was also a really huge value add. >> Jeff: Right, right. >> Measurements hard. >> Well it's interesting, I saw some of your videos and some of your prior key notes and stuff and talking about, everyone says data is in the world. But the data without context, the data without the right algorithms, and you talk about a bunch data dirty things and data problems. Data itself is not the new oil. So I wanted to get to your report 'cause that's kind of your bench mark. That's your big stake in the ground. So how are we've been doing it? What do you do different than other things that are out there? Besides the fact that it's open source which I'll ask you about as a follow up. What makes your research special? >> So why is our report different from any other reports out there? I think there's a couple things. The piece that makes me the proudest is that, the state of DevOps report is so different because it's academically rigorous. It's a true research report and I love that the team has been so loving and so patient with me. Because when I started working with the rest of the group four years ago, I stepped in and I said. This is what I want to do. These are my ideas. I was still a professor at the time, so as you mentioned, I was industry and then academia and I'm now in industry again. But I stepped in and I said, I think there's this really, really fantastic opportunity to take a look of what's going on but we have to measure this in really rigorous ways. And by doing that, it allows us to look at predictive relationships, which is interesting because it let's us say. If we focus on core capabilities, they will predict organization's ability to develop and deliver quality software with speed and stability. Which will in turn drive improvements in organizational performance. Profitability, productivity, market share. Effectiveness, efficiency delivering mission and organizational goals. Notice I'm saying predict and drive. I'm not saying correlate, which is really interesting. And so in these years of research, we've been able to identify core capabilities that drive improvement. So it allows organizations to understand what's important to invest in. It's not just this worked for my team. This worked for that team. Hey, I think this is what I'm going to try because as someone fond of joking. Anecdote is nice but the plural of anecdote isn't anecdata. (laughing) Right, and that was my frustration when I was in tech and before and when I was in consulting. If you want to try a thing and you want to apply it but it's really hard if I only have one or two or three or five maybe even 10 stories. We need so much data to really understand what will likely work for teams and for industries as a whole. And like I said, God bless the team, because I came in and I was really rigorous and I would say that doesn't work, we can't measure that. That doesn't work here and sometimes I'd come back and I'd say that doesn't hold. The stats don't hold and they say, "But it has to." "I know it worked here and I know it worked here." And I'm like, but it's not, we have no evidence to support that. The stats don't hold. This doesn't work. We can't say that and we're like hey, we'll have to try it again next year. Not try it again next year but we have to find a different way to measure it. We have to have a different hypothesis to test. But then we also find really amazing things like I said a couple times, it predicts a team's ability to develop and deliver code with speed and stability. Speed and stability. We found four years ago speed and stability go together. For years, we didn't know that was the case or we thought that in order to get stability, you had to slow down. It doesn't show up anywhere in the data. No where, high performers get both. >> So do the executives, do they realize the leader that having better internal thought for development has an impact on their business relative to saving a few bucks on parts or spending a few more bucks on marketing? As a real driver of value as oppose to it's just always internal apps that we have to build for whatever reason. >> They're starting to get there. And so what we're starting to do is we're really focusing heavily on delivering code with speed and stability. And then, we're saying okay, imagine if you could deliver with speed and stability here. What could you do with delivering features? How does that help you get to market faster? How does that help you beat your competitors? How does it allow you to respond to complaints and regulatory changes? And so that's really what helps us drive and then another way that we are a little different from other reports that are out there. Other industry reports are also very helpful but they are very different. So I don't say things like 27% of the industry is using configuration management. Other report say that and that is interesting. I don't report on percentage of the industry that's doing something. >> Right, right. >> But those other reports can not say what is predictive of improvement. So we are the prediction. Occasionally, I'll report correlations if I don't have the statistics to go as strong as-- >> And what moves it from correlation to prediction is the strength of the algorithms? >> No, it's the strength of the research design. >> The strength of the research design upfront? >> Yep, up front. >> Before you feed it in. >> Upfront and-- >> 'Cause really, you're knocking them at research. >> Yes. >> Rigor. >> Yep. >> That's the underpinning of the whole thing. >> And much more data has been published in academic periodicals, so we are still actively doing research. >> And I would imagine that the annual report is really an ongoing, longitudinal study across a whole lot of the same companies over and over and over, year in, year out. So you get them-- >> So it's open every year. >> As well. >> Yep. >> Awesome, alright Nicole. Well that is fascinating and everyone should go to DORA and get the free research. And then if they want to bring you guys in, and you offer custom services to help the particular company execute and do better. >> Yes, absolutely. So you can go to DevOps-research.com to find all of our research and anything else you want to find out about engaging with us or anything like that. >> Nicole Forsgren. She's DORA the explorer. She'll help you out with your DevOps. I'm Jeff Frick, you're watching theCUBE from PagerDuty Summit. Thanks for watching. (uptempo techno music)