>> Narrator: Live from Stanford University in Palo Alto California, it's theCUBE, covering Women in Data Science Conference 2018. Brought to you by Stanford. >> Back to the cube, our continuing coverage of Women in Data Science 2018 continues. I am Lisa Martin, live from Stanford University. We have had a great array of guests this morning, from speakers, panelists, as well as attendees. This is an incredible one day technical event, and we're very excited to be joined by one of the panelists on the career panel this afternoon, Dr. Jennifer Prendki, the Head of Data Science at Atlassian. Welcome to theCUBE. >> Hi, it's my pleasure to be here. >> It's exciting to have you here. >> So you lead all search and machine learning initiatives at Atlassian, but you were telling me something interesting about your team, tell us about that. >> The interesting thing about my team is even though I'm the Head of Data Science, my team is not 100% data scientists. The belief of the company is that we really wanted to be in charge of our own destiny and be able to deploy our models ourselves and not be depending on other people to make deployment faster. >> Was that one of the interesting kind of culture elements that attracted you last year to Atlassian? >> What is really interesting about Atlassian, it's definitely a company that create products that I would say virtually every single software company in the world is using. They have a very strong software engineering culture, and so last year they decided to embrace data science. I thought it was a very interesting challenge for me to try and infuse a little bit of my passion for data and data-driven est to the company. >> You had quite a fast ramp at Atlassian. You joined last summer, and in less than six months, you grew your team of data scientists and engineers from three people to fifteen, and it gets better, in less than six months, across three locations, Mountain View, San Francisco, and Sydney. What were some of the key things for you that led you to make that impact so quickly? >> I think most data scientists on the world are interested in making an impact, and this is a company that obviously does a lot of impact, and a lot of people talk about this company, and there is obviously a lot of interesting data, and so I think one of the amazing things is that we have a very important role to play, because we are in a position where we have data related to the way people work with each other, collaborate with each other, and this is a very unique data set, so it's usually pretty easy to attract people to Atlassian. >> You mentioned collaboration, and that's certainly an undertone here at WiDS. In its third year, you were here last year as an attendee, now you're here this year as a speaker. They've grown this event dramatically in a couple of years alone. The opportunity to reach, they're expecting, a hundred thousand, to engage. It's a hundred and seventy-seven regional events, Margot Gerritsen gave us that number about an hour ago, in fifty-three countries. What is it about WiDS that attracted you, not only back, this year, but to welcome the opportunity to be on this career panel? >> I'll actually tell you something, so, we talk about diversity, and I think people usually think of diversity as meeting some kind of racial bar, to have, equality between male and female, or specific minorities. I think people tend to forget that the real diversity is diversity of thought, and so I actually found out that the very data science job I actually got, I was actually the only person who had a background in applied math, and everybody else was coming from a background in computer science. I quickly realized that I'm the only person who is really trained to push for, let's validate our models really properly, etc., and so that made realize how important that is to have a lot of diversity. I think WiDS is definitely a place where you see lots of women interested in the same thing, but coming from different perspective, different horizons, at different levels, and this is really something unique in the industry. >> Diversity of thought, I love that. I've not heard that before, I'm going to use that, but I'll give you credit for it. That is one of the things that is so, the more people we speak to, not just at WiDS, but at events like this on theCUBE, you hear, there's still such a need, obviously, the scale of which that WiDS has grown, shows clear demand for, we need more awareness that this diversity is missing, but in the fact that data science is so horizontal, across every industry, and it sort of is blurring the boundaries between rigid job roles, doctor, lawyer, attorney, teacher, whatever. This is quite pervasive and it provides the opportunity for data scientists globally to be able to make massive impact, but also, it still, as Margot Gerritsen was sharing earlier, it still requires what you said is that diversity in thought because having a particular small set of perspectives evaluating data, you think about it from an enterprise perspective, the types of companies that Atlassian deals with, and they are looking to grow and expand and launch new business models, but if the thought diversity is narrow, there's probably a lot of opportunity that is never going to be discovered. One of the things also I found interesting in your background, was that you found yourself sort of at this interesting juxtaposition of being a mentor, and going, wait a minute, this now gives you a great opportunity, but it also comes with some overhead. You've got it from a management perspective. What is that sort of crossroads that you've found yourself reaching and what have you done with that? >> I think it's true of probably every single technical role, but maybe data science more than others, you have to be technical to be part of the story. I think people need to have a leader that they can relate to and I think it's very important that you're still part of this. It's particularly interesting for data science, because data science is a field that moves so quickly. Usually you have people moving on to data science manager positions after being in IC and so if you don't make a conscious effort to remain that technical point of contact person, that people trust and people go to, then, when I think back of the technologies that were trendy when I was still in IC compared to now, it's really important for the managers to be still aware of that, to do a good job as a mentor and as a leader. >> You also said something I think before we went live, that is an important element for the women that WiDS is aiming to inspire and educate, today. Those that are new to the field or thinking about it, as well as those who've been it for a while. There is not just getting there, and going yes I'm interested, this is my passion, I want to have a career in this, it's also having to learn how to be a female leader, and you mentioned from a management perspective, you got to learn, you have to know how to be assertive. Tell us a little bit about the trials and tribulations that you have encountered in that respect. >> That's a very interesting question, because I'm actually very happy to see that nowadays, it's becoming easier and easier for women to step into individual contributor positions, because I think that people realize now that a woman can do just as good a job as men for a defined position, but when you're actually in a leadership position, you have to step into like a thought leadership role. Basically, you sometimes have to be in a meeting where you only have all the male engineers or male data scientists over there and say, you know what, I disagree with you, right? This as a woman becomes a little bit challenging because following the processes that are already in place, I believe that people have realized that it's okay for a woman to do that, but then being the assertive person that goes against the flow and says you are not thinking about it the right way, might sometimes be a problem, because women are not being perceived as creatures that are naturally assertive. It's typical for people, like a Head of Data Science, female data scientists, to be in a situation where they are perceived as being maybe a little bit aggressive or a little bit pushy, and you sometimes fall into this old saying, "he's the boss, she's bossy," kind of thing, and that is a challenge. >> I had someone once tell me a couple years ago, and I'm in tech as well, that I was pushy, and I think this was a language barrier thing, I think he meant to say persistent, but on that front, tell me a little bit more about your team of data scientists and engineers, and the females on your team, how do you help coach them to embrace, it's okay to speak your mind? What's that been like for you? >> I would say I was actually pretty soft-spoken myself. At some point I realized that public speaking actually helped me out there. Somebody at some point told me like, you should go, you're a brilliant, technical like go speak at a conference, and then I realized people are listening to me. You always have a little bit of like imposter syndrome kind of problem as a woman, so it helped me overcome this. Now I'm kind of trained to stimulate the ladies on my group to do the same thing, because that has worked really well for me I think. You have to get outside your comfort zone, and try to, things that help you have the self-confidence for you to get to the level of assertiveness you need to become successful. >> Exactly right, we've had a number of women on the show, today alone, talk about getting outside of your comfort zone, and one of my mentors always says, get comfortably uncomfortable. That's not an easy thing to achieve, but I think you walk in the door at WiDS, and you instantly feel inspired, and empowered. I think a number of the women that we've had on today, already, have talked about having, sort of being charged as a mentor with the responsibility like you just said, of helping those that are following your footsteps, to maybe understand how to have that confidence, and then have that right balance, so that there's professionalism there, there's respect, but it's not just about getting them into the field. It's about teaching them how to, once you're there, how to navigate a career path that is successful. >> That's an interesting thought, because I actually believe that getting comfortable with the uncomfortable is definitely something that data science is about, because you have new technologies, you have new models, you have lateral moves, like I actually was in the advertising industry as a data scientist, before switching to e-commerce and then eventually to the software industry, so I think that people who are trained to be data scientists are like that, and they should also be comfortable with the uncomfortable in their daily lives. >> Yeah, so you were mentioning before we went on that some of the people that you work with are like, it's my hope and dream to be at WiDS next year. What are some of the things that you've heard as we're at the halfway mark of WiDS today, that you're going to go back and share with your team, as well as maybe your friends, other females that are working in STEM fields as well? >> I would say, last year I was here just listening to all the people and whatever. This year, I'm on the panel, so I mean, I'm just like, nothing is impossible, I think. We've proven that over and over again in data science, I mean, who would have thought that ten years ago, we would be at the level of understanding of artificial intelligence and the entire field, right? It's all about waiting and seeing what the future has to bring to you, and we have all these amazing women today, to actually show us that, it's possible to get there, and it's exciting to be here. >> It is possible, and it's exciting. Well, Jennifer, thanks so much for carving out some of your time today to speak with us. We wish you continued success at Atlassian and we look forward to seeing you back at WiDS next year. >> Thank you. >> We want to thank you for watching theCUBE, we're live at Stanford University at the third annual Women in Data Science Conference, hashtag WiDS2018, join the conversation. I'll be right back with my next guest after a short break. (upbeat music)

>> Narrator: Live from Orlando, Florida, it's The Cube covering Grace Hopper's celebration of women in computing. Brought to you by SiliconANGLE Media. >> Welcome back to the Cube's coverage of the Grace Hopper conference here in Orlanda, Florida. I'm your host, Rebecca Knight. We're joined by Aubrey Blanch. She is the Head of Diversity at Atlassian. >> Yeah, thank you so much for having me. >> Well, thank you for coming on the program. >> Absolutely, it's great to be here. >> So, tell me a little bit more about what you do as the Head of Diversity at Atlassian. >> Yeah, so I was always tell people that my job is to make people really happy and to give them an equal opportunity to succeed? But what that actually means day-to-day is that I spend a lot of time looking at the data that tells me are we hiring the right people, are we hiring people equitably, do they love coming to work and are they having an impact? So, I, that means sometimes designing programs, sometimes doing focus groups, but always trying to think about how do we make sure that everyone has the thing that they need to be really successful at Atlassian and sort of fulfill our company mission which is to help unleash the potential of every teams and for us, you know, we, we unleash the potential in every team and we know that every team is diverse and so we know that it's just an imperative for us to look like the customers that we're serving because it means that we understand them and it means that we can help them do better work. And I know that you are really dedicated to the idea of including empirical science-- >> Yes. >> In, in what you do. >> Aubrey: Yes. >> Talk to me about some the, the most powerful studies, the most powerful research that you try to bring to your thought process in terms of hiring. >> Yeah, absolutely, so I'm a recovering social scientist by training so I get really excited about the idea that you can use research to make little tweaks to the way that you do things that changes outcomes in really big ways. So, one example. We know that women, on average, when they have the same contributions as their male colleagues, actually tend to rate themselves lower. Right? Same work and then they say, "No, that's not quite as good." And so, last year we made a change to our performance review process that helps get rid of problems that might be introduced by that. So, if you're a manager and you're reading two people's work and one person has given themselves a three and one's given them a four that might affect your rating. So, we actually changed it so that now managers right the review without seeing their direct reports review. Turns out it removes bias, it shortens the process, and it helps identify whether people have an agreement about what people's work is. And we found that that meant that everyone was getting a more equitable set of ratings and we could say, "Eh, we removed bias "and it made it easier for the business." And it meant that people were getting rewarded for the value that they were creating. >> And you're also, you're also big on data. >> Aubrey: Yes. >> And so you, you first of all have to collect the data. >> So what's kinds of-- >> Yeah. >> How are you collecting data and polling employees about whether or not they are happy? Absolutely, so first, you have to collect data about who people are and how they identify. So, things like gender, race, disability status. We collect that data. And then we survey people, right? Asking them not, are you happy, but have you grown in the last six months? You know, does your manager support you in doing those things? And you can sort of triangulate what a person's experience looks like that way. But you also look at bigger things. You look at things like promotion velocity. Or what is your attrition and retention rates? And those tell you a lot of things. You dig into exit surveys and you say, "What's the number one reason that people are leaving?" Let's fix it. >> Right. >> And the other piece of data that I get really excited about and something that's sort of Atlassian's thing, I guess, is that we actually report on the diversity of our work force at the team level. So, you can check it out. It's atlassian.com/diversity. But in addition to those corporate level statistics, we really think that the diversity on your teams matters because your teams are who you're engaging with day-to-day. And you get the value out of diversity because two different people come together. And so it doesn't actually matter if you have 30% women in your company if all the women are in HR and marketing and all the men are in engineering. What matters is each of those teams is diverse because it helps them build better. And so we think it's important to measure it that way. >> That is such a great point because I think that a lot of companies can bolster their diversity numbers. >> Aubrey: Yeah. >> And with women in the more traditionally female-oriented parts of the company. >> Absolutely. But that cut of data also helps drive bigger impact. So, I'll give you an example. When we cut our data at the team level, what we saw, and this was about a year ago, that about 13.5% of our technical employees were women but when we looked at all of our teams that were developing software, two thirds of them had a woman team member. And so from that insight we were able to say, well those women are probably isolated on their teams. And so they're likely lacking a sense of community and belonging and so instead of just investing in recruiting, we created a variety of programs that helped women collaborate across their teams. So, things as simple as a coffee dates program where women opt in and are assigned to another woman in their office to have coffee with every other week. Or something more structured like a peer-mentoring ring that's cross-functional. And what we found is that that actually helped drive retention for women in those rolls. So, while we're investing in recruiting, we're also making sure that we're keeping and growing the women that are already on our teams. >> So this is, this is incredible. These small tweaks as you started off saying-- >> Yeah. >> That are really changing the way you do business. >> Absolutely. >> What is you're, you're best advice to the rest of the tech industry where Atlassian, feels like you've figured out something here? >> Yeah, I think it's trust the data and know that there are no best practices or silver bullets. So, we've made incredible progress over the last few years so-- >> And you do, and you publish your numbers. >> Yes we do. >> As you said. >> Yeah, every year. We've improved our hiring of women in technical roles by 80% over the last two years and it's, we've honestly just adopted the same approach that our software teams use. Which is we test something, we see whether it works and then we iterate and improve it. >> Agile, right. >> Right. And so it's not about one training or one program, it's about re-thinking about how you engage with your people and how you respond to their experiences. Because they'll tell you what they want and need and it's about providing that. And I always tell people best practices are a starting point but they may or may not work for you. So, you need to be open minded to the idea that the first thing you try just might not work because your culture might be different or something like that. For us, we also like to think about diversity in a really broad way. So, my other piece of advice is think intersectionally, right? So when we say-- >> What does that mean? >> Yeah. >> How do you define that? >> So, it's a big, complicated word but it just means that we all have layers. So, I, for example, identify as a woman but I also identify as American and Hispanic and five feet tall and an HR person and all of us carry all of those identities around and what you, so you need to understand that women is a diverse group. But, when you do that, when you start talking about axes of diversity that are past gender, it turns out it turns what could be an us-versus-them conversation into something that's about we. Because maybe someone says, "Well, I don't identify as female "but this is the unique thing that I bring in." And suddenly you've created it where everyone has an incentive and has skin in the game to create inclusion and you will get greater gender equity out of that. So, it's a little bit counter-intuitive to start backwards in a way, or start complex and work towards simple but that's something that we've found has been incredibly helpful in galvanizing people to get involved and really changing the culture in a way that it's not a top down initiative or a bottom-up initiative, it's everyone moving in the same direction. >> Well, Aubrey, it sounds so common-sensical, of course, yes, yes. >> Yeah. >> But it's only obvious after you say it. >> Right, yeah, yes. >> And after you've tried it and tested and iterated on it. (laughs) >> So that would be my thing is, is whatever diversity matters to you because at Atlassian, for example, we're an Australian company and so international diversity is incredibly important, right? Where you come from. You know we, I always joke, you're more likely to hear three languages walking across the office than anything else and that's a really cool place to be but it means we've already gotten used to working in a diverse environment and now it's how do we just add additional aspects of diversity to our culture and to our teams? >> Right, and let's not fight that. >> Absolutely. >> 'Cause it's working. >> Right, and the other thing that I've found which is really exciting is as I've seen teams start to change their composition, you don't just hear really great things from those folks who come from under-represented groups. People from those majority groups say, "Wow, it's actually improving my experience at work," because they have access to more perspectives and people who have different experiences than them. >> So, it's firing different parts of their brains to-- >> Yeah. >> To-- >> It's just more interesting to do your job that way. >> Have better ideas, yeah. >> So, that's the other thing that's real important is this is a win-win-win solution, it's not a zero-sum game. >> Right. Well Aubrey, thanks so much for joining us. It's been a lot of fun talking to you. >> Absolutely. Thank you so much for having me. >> We will have more from the Cube's coverage of the Grace Hopper Conference just after this.

>> Announcer: Live, from Washington DC, it's the CUBE. Covering .conf2017. Brought to you by Splunk. >> And welcome back here on theCUBE. John Walls and Dave Vellante, we're in Washington DC for .conf2017, Splunk's annual get together coming up to the nation's capital for the first time. This is the eighth year for the show, and 7,000 plus attendees, 65 countries, quite a wide menu of activities going on here. We'll get into that a little bit later on. We're joined now by a couple of gentlemen, Michael Arahuleta who is the Vice President of Engineering at Atlassian, Michael, thank you for being with us. >> Thank you, actually it's Director of Business Development. >> John: Oh, Director of Business Development, my apologies >> He's doin' a great job >> My apologies. >> I don't need that. >> Oh very good. And Brooke Gravitt, who I believe is the VP of Engineering, >> There ya go. >> And the Chief Software Architect at Forty8Fifty. >> Yep, how ya doin'? >> No promotions or job assignments, I've gotcha on the right path there? >> Yeah, yeah. >> Good deal, alright. Thank you for joining us, both of you. First off, let's just set the stage a little bit for the folks watching at home, tell us a little bit about your company, descriptions, core competencies, and your responsibilities, and then we'll get into the intersection, of why the two of you are here. So Michael, why don't you lead off. >> So Atlassian, we, in our simplest form, right, we make team collaboration software. So our goal as a company is to really help make the tools that companies use to collaborate and communicate internally. Our primary focus, and kind of our bread and butter has always been making the tools that software companies use to turn around and make their software. Which is a great position to be in, and an increasingly we're seeing ourselves expand into providing that team collaboration software products like Jira, Confluence, BitBucket, and now, the new introduction of a product called Stride, which is a real time team collaboration product, not just for technical teams, but we're really seeing a great opportunity to empower all teams 'cause every team in every organization needs a better way to communicate and get things done. That's really what Atlassian core focus is all about. >> John: Gotcha. Brooke, if you would. >> Yeah, so Forty8Fifty Labs, we're the software development and DevOps focused subsidiary of Veristor Systems based out of Atlanta. We focus primarily on four key partners, which would be Atlassian, Splunk, QA Symphony, and Red Hat, and primarily, we do integrations and extensibility around products that these guys provide as well as hosting, training, and consulting on DevOps and Atlassian products. >> So the ideal state in your worlds is you've got -- true DevOps, Agile, infrastructure as code, I'll throw all the buzzwords out at ya, but essentially you're not tossing code from the development team into the operations team who them hacks the code, messes it up, points fingers, all that stuff is in part anyway what you're about eliminating, >> Right. >> And getting to value sooner. Okay, so that's the sort of end state Nirvana. Many companies struggle with that obviously, You got, what, Gartner has this term, bimodal IT, which everybody, you know, everybody criticizes but it's sort of true. You've got hybrid clouds, you've got, you know, different skillsets, what is the state of, Agile development, DevOps, where are we in terms of organizational maturity? Wonder if you guys could comment. >> I'll start with that right, I think -- Even though we've been talking about DevOps for a while and companies like Atlassian and Splunk, we live and breathe it. I still think when you look at the vast majority of enterprises, we're still at the early stages of effectively implementing this. I think we're still really bringing the right definition to what DevOps is, we're kind of go through those cycles where either a buzzword gets hot, everybody glams onto it, but no one really knows what it means. I think we're really getting into that truly understanding what DevOps means. I know we've been working hard at Atlassian to really define that strong ecosystem of partners. We really see ourselves as kind of in the middle of that DevOps lifecycle, and we integrate with so many great solutions around monitoring and logging, testing, other operational softwares, and things of that nature to really complete that DevOps lifecycle. I think we're really just now finally seeing it come together and finally starting to see even larger organizations, very large Fortune 100 companies talk about how they know they've got to get away from Waterfall, they've got to embrace Agile, and they've got to get to a true DevOps culture, and I think that's where Atlassian is very strong, devs have loved us for a long time. Operations teams are really learning to embrace Atlassian as well. I think we're really going to great position to be at that mesh of what truly is DevOps as it really emerges in the next couple years. >> Brooke, people come to Forty8Fifty, and they say, alright, teach me how to fish in the DevOps world, is that right? >> Yeah, absolutely. I mean, one of the challenges that you have in large enterprises is bringing these two groups of people together, and one of the easy ways is to go out and buy a tool, I think the harder and more difficult challenge that they face is the culture change that's required to really have a successful DevOps transformation. So we do a little bit of consulting in that area with workshops with folks like Gene Kim, Gary Gruver, Jez Humble that we bring in who are sort of industry icons for that sort of DevOps transformation. To assist, based on our experiences ourselves in previous companies or engagements with customers where we've been successful. >> So the cloud native guys, people who are doing predominantly cloud, or smaller companies, tech companies presumably, have glommed onto this, what about the sort of the Fortune 1000, the Global 2000, what are we seeing in terms of their adoption, I mean, you mentioned Waterfall before, you talk to some application development heads will say, well listen, we got to protect some of our Waterfall, because it's appropriate. What are you seeing in the sort of traditional enterprise? >> We see the traditional enterprise really embracing Agile in a very aggressive way. Obviously they wouldn't be working with Atlassian if they weren't, so our view is probably a little bit tilted. Companies that engage with us are the more open to that. But we're definitely seeing that the far and away the vast majority in the reports that we get from our partners like Forty8Fifty Labs is that increasingly larger and larger companies are really aggressively looking to embrace Agile, bring these methodologies in, and the other simple truth is with the way Atlassian sells -- the way we sell our products online, we have always sort of grown kind of bottoms up inside a lot of these large organizations, so where officially IT may still be doing something else, they're always countless smaller teams within the organization that have embraced Atlassian, are using Atlassian products, and then, a year down the road, or two years down the road, we tend to then emerge as the defacto solution for the organization after we kind of spread through all these different groups within the company. It's a great growth strategy, a lot are trying to replicate it. >> Okay, what's the Splunk angle? What do you guys do with Splunk, and how does it affect your business? >> Mike: Do you want to start? >> Sure, so, we're both a partner of Splunk, a customer of Splunk, and we use it in our own products in terms of our hosting, and support methodologies that we leverage at Forty8Fifty. We use the product day in and day out, and so with Atlassian, we have pulled together a connector that is -- one half of it is a Splunk app, it's available on Splunk base, and the other part is in the Atlassian marketplace, which allows us to send events from Juris Service Desk, ticketing events, over to Splunk to be indexed. You have a data model that ties in and allows you to get some metrics out of those events, and then the return trip is to -- based on real time searches, or alerts, or things that you have -- you're very interested in reports, you can trigger issues to be created inside of Jira. >> I think the only thing to add to that, so definitely, that's been a great relationship and partnership, and we're seeing an increasing number of our partners also become partners with Splunk and vice versa, which is great. The other strong side to this as well, is our own internal use of Splunk. So, we as a company, we always like to empower our different teams to pick whatever solution they want to use, and embrace that, and really give that authority to the individual teams. However, with logging, we were having a huge problem where all of our different teams were using over a whole host variety of different logging solutions, and frankly not to go into all the details, it was a mess. Our security team decided to embrace Splunk and start using Splunk, and really got a lot of value out of the solution and fell in love with the solution. Which says a lot, because our security team doesn't normally like much of anything, especially if it's not homegrown. That was a huge statement there, and then quickly Splunk now has spread to our cloud team which is growing rapidly as our cloud scales dramatically. Our developers are using it for troubleshooting, our SREs and our support team for incident management, and it's even spread to our marketplace, which is one of the larger marketplaces out there today for third party apps. Then the new product, Stride, for team collaboration is going to be very dependent on Splunk for logging as well. It's become that uniform fabric. I even heard a dev use a term which I've never heard a dev talk about logs and talk about log love, which is no PR, that is the direct statement from a developer, which I thought was amazing to hear. 'cause you know, they just want to code and make stuff, they don't want to deal when it actually breaks and have to fix it. But with Splunk they've actually -- They're telling me they actually enjoy that. So that's a great -- >> That's more than the answer is in the logs, that's there's value in our logs, right? >> Yeah, a ton of value, right? Because at the end of the day, these alerts are coming in and then we use tools like the Forty8Fifty Labs tool to get those tickets into Jira. Those logs and things are coming in, that means there's an issue and there's something to be resolved and there's customer pain. So the quicker we can resolve that, that log is that first indicator of what's going on in the cloud and in our platforms to help us figure out how do we keep that customer happy? This isn't just work, and just a task, this is about delivering customer value and that log can be that first indicator. The sooner you can get something resolved, the sooner the customer's back to getting stuff done and that's really our focus as a company, right? How do we enable people to get things done? >> Excuse me, when you are talking about your customers, what are their pain points? Today? I mean, big data's getting bigger and more capabilities, you've got all kinds of transport problems and storage problems, and security problems, so what are the pain points for the people who are just trying to get up to speed, trying to get into the game, and that the kind of services you're trying to bring to them to open their eyes. >> I think if you look at the value stream mapping and time to market for most businesses, where Splunk and Atlassian play in is getting that fast feedback. The closer in to the development side, the left hand side of value stream that you can pull in, key metrics, and get an understanding of where issues are, that actually -- it's much less expensive to fix problems in development than when they're in production, obviously. Rolling things like Splunk that can be used as a SIM to do some security analysis on, whether it be product code or business process early, rather than end up with a data breach or finding something after it's already in production. That kind of stuff, those are the challenges that a lot of the companies are facing is -- especially when the news, if you look at all the things that are goin on from a security perspective, taking these two products and being able to detect things that are going on, trends, any sort of unusual activity, and immediately having that come back for somebody in a service desk to work on either as a security incident or if it's a developer finding a bug early in the lifecycle, and augmenting your sort of infrastructure as code, the build out of the infrastructure itself. Being able to log all that data, and look at the metrics around that to help you build more robust enterprise class platforms for your teams. >> We've been sort of joking earlier about how the big data, nobody really talks about big data anymore, interestingly, Splunk who used to never talk about big data is now talking about big data, cause they're kind of living it. It's almost like same wine, new bottle with machine learning and AI and deep learning are all kind of the new big data buzzwords, but my question is, as practitioners, you were describing a situation where you can sort of identify a problem, maybe get an alert, and then manually I guess remediate that problem, how far away are we from -- so the machines automating that remediation? Thoughts on that? >> Am I first up? >> You guys kind of -- >> We've done a lot of automated remdediation. Close with remediation is what you call it. The big challenge is, it's a multi-disciplinary effort, so you might have folks that need to have expertise between network and systems and the application stack, maybe load balancing. There's a lot of different pieces there, so step one is you got to have folks that have the capacity to actually create the automation for their domain of expertise, and then you need to have sort of that cross platform DevOps mindset of being able to pull that together and the coordinator role of let's orchastrate all of the automations, and then hopefully out of that, combined with machine learning, some of the stuff that you can do in AWS, or with IBM's got out. You can take some of that analysis and be a little bit smarter about running the automation. In terms of whether that's scaling things up, or when -- For example, if you're in a financial industry and you've got a webpage that people are doing bill pay for, if you have a single website down, a web server down, out of a farm of 1000, in a traditional NOC, that would be kind of red on a dashboard. It's high, it's low priority, but it's high visibility and it's just noise, and so leveraging machine learning, people do that in Splunk to really refine what actually shows up in the NOC, that's something I think is compelling to customers. >> How are devs dealing with complexity, obviously, collaboration tools help, but I mean, the level of complexity today, versus when you think back to client server, is orders of magnitude greater for admins and developers, now you got to throw in containers and microservices, and the amount of data, is the industry keeping pace with the pace of escalation of complexity, and if so, how? >> I think we're trying. I think that's where we come into play. As this complexity increases really the only way you can solve it is through better communication and better tools to make sure that teams have the right information at their fingertips. The other challenge too is now in the world of the cloud, these teams need to be on 24/7. But you've got to kind of roll across the globe, and have your support teams in different time zones. You don't always have the right people online at the same time to be able to address, and you can't always talk directly, so that's where having the right tools and processes in place are extremely important so that team can know and know what did the team earlier do, how did they resolve this, where's the run book for this issue, and if this happens, how do we resolve it? How do we do so quickly? I think that tooling is key, and also too, this complexity is also as you guys were talking about before, being solved through some automation as well, and we're increasingly seeing that to where if this occurs and a certain thing occurs, then Jira can now automatically start to trigger some things for you, and then report back as to what it did. You're going to see more and more of that going forward as these models become more intelligent and we can redeploy, or if capacity is low, let's pull back resources, and let's not spend all this money on cloud computing platforms that we may not need because utilization is low. You're seeing all of those things start to happen and Jira as that workflow engine is that engine that's making those things happen in either an automated way at times, or just enabling people to communicate and do things in a very logical fashion. >> As ecosystem partners, how do you view the evolution of Splunk, is it becoming a application platform for you? Are you concerned about swim lanes? I wonder if you could talk about that? >> I personally, I don't see any real concerns of overlap between Splunk and Atlassian. In our view at Atlassian is, we tend to work very closely with people kind of fit into that frenemy category, and they're definitely a partner that we overlap with I think in very very few ways. If and when we ever do, I mean in a way, that's kind of something we always embrace as a company. I mean one thing we'll say a lot is overlap is better than a gap. Because if there's a gap between us and a partner, then that's going to result in customer pain. That means there's nothing that's filling that void. I'd rather have some overlap, and then give the customer the power to choose how do they want to do it. I mean, Splunk says you can probably do it this way, Atlassian says you could do it this way, as long as they can get stuff done, and that's always -- it's not a cliche from us, I mean that's a core message from Atlassian, then we're happy. Regardless if they completely embrace it our way, a little bit, a little deviation, that's not what really matters. >> Too much better than too little. >> Exactly. >> Is what it comes down to. Gentlemen, thanks for being with us. >> Thank you. >> We appreciate the time today and look forward to seeing you down the road and looking as your relationship continues. Not only between the two companies, but with Splunk as well. Thanks for being here. >> Mike: Thank you guys. >> We continue theCUBE does, live from Washington DC here at .conf2017, back with more in just a bit.

>> From theCUBE Studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR, this is "Breaking Analysis" with Dave Vellante. >> A decade of big data investments combined with cloud scale, the rise of much more cost effective processing power. And the introduction of advanced tooling has catapulted machine intelligence to the forefront of technology investments. No matter what job you have, your operation will be AI powered within five years and machines may actually even be doing your job. Artificial intelligence is being infused into applications, infrastructure, equipment, and virtually every aspect of our lives. AI is proving to be extremely helpful at things like controlling vehicles, speeding up medical diagnoses, processing language, advancing science, and generally raising the stakes on what it means to apply technology for business advantage. But business value realization has been a challenge for most organizations due to lack of skills, complexity of programming models, immature technology integration, sizable upfront investments, ethical concerns, and lack of business alignment. Mastering AI technology will not be a requirement for success in our view. However, figuring out how and where to apply AI to your business will be crucial. That means understanding the business case, picking the right technology partner, experimenting in bite-sized chunks, and quickly identifying winners to double down on from an investment standpoint. Hello and welcome to this week's Wiki-bond CUBE Insights powered by ETR. In this breaking analysis, we update you on the state of AI and what it means for the competition. And to do so, we invite into our studios Andy Thurai of Constellation Research. Andy covers AI deeply. He knows the players, he knows the pitfalls of AI investment, and he's a collaborator. Andy, great to have you on the program. Thanks for coming into our CUBE studios. >> Thanks for having me on. >> You're very welcome. Okay, let's set the table with a premise and a series of assertions we want to test with Andy. I'm going to lay 'em out. And then Andy, I'd love for you to comment. So, first of all, according to McKinsey, AI adoption has more than doubled since 2017, but only 10% of organizations report seeing significant ROI. That's a BCG and MIT study. And part of that challenge of AI is it requires data, is requires good data, data proficiency, which is not trivial, as you know. Firms that can master both data and AI, we believe are going to have a competitive advantage this decade. Hyperscalers, as we show you dominate AI and ML. We'll show you some data on that. And having said that, there's plenty of room for specialists. They need to partner with the cloud vendors for go to market productivity. And finally, organizations increasingly have to put data and AI at the center of their enterprises. And to do that, most are going to rely on vendor R&D to leverage AI and ML. In other words, Andy, they're going to buy it and apply it as opposed to build it. What are your thoughts on that setup and that premise? >> Yeah, I see that a lot happening in the field, right? So first of all, the only 10% of realizing a return on investment. That's so true because we talked about this earlier, the most companies are still in the innovation cycle. So they're trying to innovate and see what they can do to apply. A lot of these times when you look at the solutions, what they come up with or the models they create, the experimentation they do, most times they don't even have a good business case to solve, right? So they just experiment and then they figure it out, "Oh my God, this model is working. Can we do something to solve it?" So it's like you found a hammer and then you're trying to find the needle kind of thing, right? That never works. >> 'Cause it's cool or whatever it is. >> It is, right? So that's why, I always advise, when they come to me and ask me things like, "Hey, what's the right way to do it? What is the secret sauce?" And, we talked about this. The first thing I tell them is, "Find out what is the business case that's having the most amount of problems, that that can be solved using some of the AI use cases," right? Not all of them can be solved. Even after you experiment, do the whole nine yards, spend millions of dollars on that, right? And later on you make it efficient only by saving maybe $50,000 for the company or a $100,000 for the company, is it really even worth the experiment, right? So you got to start with the saying that, you know, where's the base for this happening? Where's the need? What's a business use case? It doesn't have to be about cost efficient and saving money in the existing processes. It could be a new thing. You want to bring in a new revenue stream, but figure out what is a business use case, how much money potentially I can make off of that. The same way that start-ups go after. Right? >> Yeah. Pretty straightforward. All right, let's take a look at where ML and AI fit relative to the other hot sectors of the ETR dataset. This XY graph shows net score spending velocity in the vertical axis and presence in the survey, they call it sector perversion for the October survey, the January survey's in the field. Then that squiggly line on ML/AI represents the progression. Since the January 21 survey, you can see the downward trajectory. And we position ML and AI relative to the other big four hot sectors or big three, including, ML/AI is four. Containers, cloud and RPA. These have consistently performed above that magic 40% red dotted line for most of the past two years. Anything above 40%, we think is highly elevated. And we've just included analytics and big data for context and relevant adjacentness, if you will. Now note that green arrow moving toward, you know, the 40% mark on ML/AI. I got a glimpse of the January survey, which is in the field. It's got more than a thousand responses already, and it's trending up for the current survey. So Andy, what do you make of this downward trajectory over the past seven quarters and the presumed uptick in the coming months? >> So one of the things you have to keep in mind is when the pandemic happened, it's about survival mode, right? So when somebody's in a survival mode, what happens, the luxury and the innovations get cut. That's what happens. And this is exactly what happened in the situation. So as you can see in the last seven quarters, which is almost dating back close to pandemic, everybody was trying to keep their operations alive, especially digital operations. How do I keep the lights on? That's the most important thing for them. So while the numbers spent on AI, ML is less overall, I still think the AI ML to spend to sort of like a employee experience or the IT ops, AI ops, ML ops, as we talked about, some of those areas actually went up. There are companies, we talked about it, Atlassian had a lot of platform issues till the amount of money people are spending on that is exorbitant and simply because they are offering the solution that was not available other way. So there are companies out there, you can take AoPS or incident management for that matter, right? A lot of companies have a digital insurance, they don't know how to properly manage it. How do you find an intern solve it immediately? That's all using AI ML and some of those areas actually growing unbelievable, the companies in that area. >> So this is a really good point. If you can you bring up that chart again, what Andy's saying is a lot of the companies in the ETR taxonomy that are doing things with AI might not necessarily show up in a granular fashion. And I think the other point I would make is, these are still highly elevated numbers. If you put on like storage and servers, they would read way, way down the list. And, look in the pandemic, we had to deal with work from home, we had to re-architect the network, we had to worry about security. So those are really good points that you made there. Let's, unpack this a little bit and look at the ML AI sector and the ETR data and specifically at the players and get Andy to comment on this. This chart here shows the same x y dimensions, and it just notes some of the players that are specifically have services and products that people spend money on, that CIOs and IT buyers can comment on. So the table insert shows how the companies are plotted, it's net score, and then the ends in the survey. And Andy, the hyperscalers are dominant, as you can see. You see Databricks there showing strong as a specialist, and then you got to pack a six or seven in there. And then Oracle and IBM, kind of the big whales of yester year are in the mix. And to your point, companies like Salesforce that you mentioned to me offline aren't in that mix, but they do a lot in AI. But what are your takeaways from that data? >> If you could put the slide back on please. I want to make quick comments on a couple of those. So the first one is, it's surprising other hyperscalers, right? As you and I talked about this earlier, AWS is more about logo blocks. We discussed that, right? >> Like what? Like a SageMaker as an example. >> We'll give you all the components what do you need. Whether it's MLOps component or whether it's, CodeWhisperer that we talked about, or a oral platform or data or data, whatever you want. They'll give you the blocks and then you'll build things on top of it, right? But Google took a different way. Matter of fact, if we did those numbers a few years ago, Google would've been number one because they did a lot of work with their acquisition of DeepMind and other things. They're way ahead of the pack when it comes to AI for longest time. Now, I think Microsoft's move of partnering and taking a huge competitor out would open the eyes is unbelievable. You saw that everybody is talking about chat GPI, right? And the open AI tool and ChatGPT rather. Remember as Warren Buffet is saying that, when my laundry lady comes and talk to me about stock market, it's heated up. So that's how it's heated up. Everybody's using ChatGPT. What that means is at the end of the day is they're creating, it's still in beta, keep in mind. It's not fully... >> Can you play with it a little bit? >> I have a little bit. >> I have, but it's good and it's not good. You know what I mean? >> Look, so at the end of the day, you take the massive text of all the available text in the world today, mass them all together. And then you ask a question, it's going to basically search through that and figure it out and answer that back. Yes, it's good. But again, as we discussed, if there's no business use case of what problem you're going to solve. This is building hype. But then eventually they'll figure out, for example, all your chats, online chats, could be aided by your AI chat bots, which is already there, which is not there at that level. This could build help that, right? Or the other thing we talked about is one of the areas where I'm more concerned about is that it is able to produce equal enough original text at the level that humans can produce, for example, ChatGPT or the equal enough, the large language transformer can help you write stories as of Shakespeare wrote it. Pretty close to it. It'll learn from that. So when it comes down to it, talk about creating messages, articles, blogs, especially during political seasons, not necessarily just in US, but anywhere for that matter. If people are able to produce at the emission speed and throw it at the consumers and confuse them, the elections can be won, the governments can be toppled. >> Because to your point about chatbots is chatbots have obviously, reduced the number of bodies that you need to support chat. But they haven't solved the problem of serving consumers. Most of the chat bots are conditioned response, which of the following best describes your problem? >> The current chatbot. >> Yeah. Hey, did we solve your problem? No. Is the answer. So that has some real potential. But if you could bring up that slide again, Ken, I mean you've got the hyperscalers that are dominant. You talked about Google and Microsoft is ubiquitous, they seem to be dominant in every ETR category. But then you have these other specialists. How do those guys compete? And maybe you could even, cite some of the guys that you know, how do they compete with the hyperscalers? What's the key there for like a C3 ai or some of the others that are on there? >> So I've spoken with at least two of the CEOs of the smaller companies that you have on the list. One of the things they're worried about is that if they continue to operate independently without being part of hyperscaler, either the hyperscalers will develop something to compete against them full scale, or they'll become irrelevant. Because at the end of the day, look, cloud is dominant. Not many companies are going to do like AI modeling and training and deployment the whole nine yards by independent by themselves. They're going to depend on one of the clouds, right? So if they're already going to be in the cloud, by taking them out to come to you, it's going to be extremely difficult issue to solve. So all these companies are going and saying, "You know what? We need to be in hyperscalers." For example, you could have looked at DataRobot recently, they made announcements, Google and AWS, and they are all over the place. So you need to go where the customers are. Right? >> All right, before we go on, I want to share some other data from ETR and why people adopt AI and get your feedback. So the data historically shows that feature breadth and technical capabilities were the main decision points for AI adoption, historically. What says to me that it's too much focus on technology. In your view, is that changing? Does it have to change? Will it change? >> Yes. Simple answer is yes. So here's the thing. The data you're speaking from is from previous years. >> Yes >> I can guarantee you, if you look at the latest data that's coming in now, those two will be a secondary and tertiary points. The number one would be about ROI. And how do I achieve? I've spent ton of money on all of my experiments. This is the same thing theme I'm seeing across when talking to everybody who's spending money on AI. I've spent so much money on it. When can I get it live in production? How much, how can I quickly get it? Because you know, the board is breathing down their neck. You already spend this much money. Show me something that's valuable. So the ROI is going to become, take it from me, I'm predicting this for 2023, that's going to become number one. >> Yeah, and if people focus on it, they'll figure it out. Okay. Let's take a look at some of the top players that won, some of the names we just looked at and double click on that and break down their spending profile. So the chart here shows the net score, how net score is calculated. So pay attention to the second set of bars that Databricks, who was pretty prominent on the previous chart. And we've annotated the colors. The lime green is, we're bringing the platform in new. The forest green is, we're going to spend 6% or more relative to last year. And the gray is flat spending. The pinkish is our spending's going to be down on AI and ML, 6% or worse. And the red is churn. So you don't want big red. You subtract the reds from the greens and you get net score, which is shown by those blue dots that you see there. So AWS has the highest net score and very little churn. I mean, single low single digit churn. But notably, you see Databricks and DataRobot are next in line within Microsoft and Google also, they've got very low churn. Andy, what are your thoughts on this data? >> So a couple of things that stands out to me. Most of them are in line with my conversation with customers. Couple of them stood out to me on how bad IBM Watson is doing. >> Yeah, bring that back up if you would. Let's take a look at that. IBM Watson is the far right and the red, that bright red is churning and again, you want low red here. Why do you think that is? >> Well, so look, IBM has been in the forefront of innovating things for many, many years now, right? And over the course of years we talked about this, they moved from a product innovation centric company into more of a services company. And over the years they were making, as at one point, you know that they were making about majority of that money from services. Now things have changed Arvind has taken over, he came from research. So he's doing a great job of trying to reinvent themselves as a company. But it's going to have a long way to catch up. IBM Watson, if you think about it, that played what, jeopardy and chess years ago, like 15 years ago? >> It was jaw dropping when you first saw it. And then they weren't able to commercialize that. >> Yeah. >> And you're making a good point. When Gerstner took over IBM at the time, John Akers wanted to split the company up. He wanted to have a database company, he wanted to have a storage company. Because that's where the industry trend was, Gerstner said no, he came from AMEX, right? He came from American Express. He said, "No, we're going to have a single throat to choke for the customer." They bought PWC for relatively short money. I think it was $15 billion, completely transformed and I would argue saved IBM. But the trade off was, it sort of took them out of product leadership. And so from Gerstner to Palmisano to Remedi, it was really a services led company. And I think Arvind is really bringing it back to a product company with strong consulting. I mean, that's one of the pillars. And so I think that's, they've got a strong story in data and AI. They just got to sort of bring it together and better. Bring that chart up one more time. I want to, the other point is Oracle, Oracle sort of has the dominant lock-in for mission critical database and they're sort of applying AI there. But to your point, they're really not an AI company in the sense that they're taking unstructured data and doing sort of new things. It's really about how to make Oracle better, right? >> Well, you got to remember, Oracle is about database for the structure data. So in yesterday's world, they were dominant database. But you know, if you are to start storing like videos and texts and audio and other things, and then start doing search of vector search and all that, Oracle is not necessarily the database company of choice. And they're strongest thing being apps and building AI into the apps? They are kind of surviving in that area. But again, I wouldn't name them as an AI company, right? But the other thing that that surprised me in that list, what you showed me is yes, AWS is number one. >> Bring that back up if you would, Ken. >> AWS is number one as you, it should be. But what what actually caught me by surprise is how DataRobot is holding, you know? I mean, look at that. The either net new addition and or expansion, DataRobot seem to be doing equally well, even better than Microsoft and Google. That surprises me. >> DataRobot's, and again, this is a function of spending momentum. So remember from the previous chart that Microsoft and Google, much, much larger than DataRobot. DataRobot more niche. But with spending velocity and has always had strong spending velocity, despite some of the recent challenges, organizational challenges. And then you see these other specialists, H2O.ai, Anaconda, dataiku, little bit of red showing there C3.ai. But these again, to stress are the sort of specialists other than obviously the hyperscalers. These are the specialists in AI. All right, so we hit the bigger names in the sector. Now let's take a look at the emerging technology companies. And one of the gems of the ETR dataset is the emerging technology survey. It's called ETS. They used to just do it like twice a year. It's now run four times a year. I just discovered it kind of mid-2022. And it's exclusively focused on private companies that are potential disruptors, they might be M&A candidates and if they've raised enough money, they could be acquirers of companies as well. So Databricks would be an example. They've made a number of investments in companies. SNEAK would be another good example. Companies that are private, but they're buyers, they hope to go IPO at some point in time. So this chart here, shows the emerging companies in the ML AI sector of the ETR dataset. So the dimensions of this are similar, they're net sentiment on the Y axis and mind share on the X axis. Basically, the ETS study measures awareness on the x axis and intent to do something with, evaluate or implement or not, on that vertical axis. So it's like net score on the vertical where negatives are subtracted from the positives. And again, mind share is vendor awareness. That's the horizontal axis. Now that inserted table shows net sentiment and the ends in the survey, which informs the position of the dots. And you'll notice we're plotting TensorFlow as well. We know that's not a company, but it's there for reference as open source tooling is an option for customers. And ETR sometimes like to show that as a reference point. Now we've also drawn a line for Databricks to show how relatively dominant they've become in the past 10 ETS surveys and sort of mind share going back to late 2018. And you can see a dozen or so other emerging tech vendors. So Andy, I want you to share your thoughts on these players, who were the ones to watch, name some names. We'll bring that data back up as you as you comment. >> So Databricks, as you said, remember we talked about how Oracle is not necessarily the database of the choice, you know? So Databricks is kind of trying to solve some of the issue for AI/ML workloads, right? And the problem is also there is no one company that could solve all of the problems. For example, if you look at the names in here, some of them are database names, some of them are platform names, some of them are like MLOps companies like, DataRobot (indistinct) and others. And some of them are like future based companies like, you know, the Techton and stuff. >> So it's a mix of those sub sectors? >> It's a mix of those companies. >> We'll talk to ETR about that. They'd be interested in your input on how to make this more granular and these sub-sectors. You got Hugging Face in here, >> Which is NLP, yeah. >> Okay. So your take, are these companies going to get acquired? Are they going to go IPO? Are they going to merge? >> Well, most of them going to get acquired. My prediction would be most of them will get acquired because look, at the end of the day, hyperscalers need these capabilities, right? So they're going to either create their own, AWS is very good at doing that. They have done a lot of those things. But the other ones, like for particularly Azure, they're going to look at it and saying that, "You know what, it's going to take time for me to build this. Why don't I just go and buy you?" Right? Or or even the smaller players like Oracle or IBM Cloud, this will exist. They might even take a look at them, right? So at the end of the day, a lot of these companies are going to get acquired or merged with others. >> Yeah. All right, let's wrap with some final thoughts. I'm going to make some comments Andy, and then ask you to dig in here. Look, despite the challenge of leveraging AI, you know, Ken, if you could bring up the next chart. We're not repeating, we're not predicting the AI winter of the 1990s. Machine intelligence. It's a superpower that's going to permeate every aspect of the technology industry. AI and data strategies have to be connected. Leveraging first party data is going to increase AI competitiveness and shorten time to value. Andy, I'd love your thoughts on that. I know you've got some thoughts on governance and AI ethics. You know, we talked about ChatGBT, Deepfakes, help us unpack all these trends. >> So there's so much information packed up there, right? The AI and data strategy, that's very, very, very important. If you don't have a proper data, people don't realize that AI is, your AI is the morals that you built on, it's predominantly based on the data what you have. It's not, AI cannot predict something that's going to happen without knowing what it is. It need to be trained, it need to understand what is it you're talking about. So 99% of the time you got to have a good data for you to train. So this where I mentioned to you, the problem is a lot of these companies can't afford to collect the real world data because it takes too long, it's too expensive. So a lot of these companies are trying to do the synthetic data way. It has its own set of issues because you can't use all... >> What's that synthetic data? Explain that. >> Synthetic data is basically not a real world data, but it's a created or simulated data equal and based on real data. It looks, feels, smells, taste like a real data, but it's not exactly real data, right? This is particularly useful in the financial and healthcare industry for world. So you don't have to, at the end of the day, if you have real data about your and my medical history data, if you redact it, you can still reverse this. It's fairly easy, right? >> Yeah, yeah. >> So by creating a synthetic data, there is no correlation between the real data and the synthetic data. >> So that's part of AI ethics and privacy and, okay. >> So the synthetic data, the issue with that is that when you're trying to commingle that with that, you can't create models based on just on synthetic data because synthetic data, as I said is artificial data. So basically you're creating artificial models, so you got to blend in properly that that blend is the problem. And you know how much of real data, how much of synthetic data you could use. You got to use judgment between efficiency cost and the time duration stuff. So that's one-- >> And risk >> And the risk involved with that. And the secondary issues which we talked about is that when you're creating, okay, you take a business use case, okay, you think about investing things, you build the whole thing out and you're trying to put it out into the market. Most companies that I talk to don't have a proper governance in place. They don't have ethics standards in place. They don't worry about the biases in data, they just go on trying to solve a business case >> It's wild west. >> 'Cause that's what they start. It's a wild west! And then at the end of the day when they are close to some legal litigation action or something or something else happens and that's when the Oh Shit! moments happens, right? And then they come in and say, "You know what, how do I fix this?" The governance, security and all of those things, ethics bias, data bias, de-biasing, none of them can be an afterthought. It got to start with the, from the get-go. So you got to start at the beginning saying that, "You know what, I'm going to do all of those AI programs, but before we get into this, we got to set some framework for doing all these things properly." Right? And then the-- >> Yeah. So let's go back to the key points. I want to bring up the cloud again. Because you got to get cloud right. Getting that right matters in AI to the points that you were making earlier. You can't just be out on an island and hyperscalers, they're going to obviously continue to do well. They get more and more data's going into the cloud and they have the native tools. To your point, in the case of AWS, Microsoft's obviously ubiquitous. Google's got great capabilities here. They've got integrated ecosystems partners that are going to continue to strengthen through the decade. What are your thoughts here? >> So a couple of things. One is the last mile ML or last mile AI that nobody's talking about. So that need to be attended to. There are lot of players in the market that coming up, when I talk about last mile, I'm talking about after you're done with the experimentation of the model, how fast and quickly and efficiently can you get it to production? So that's production being-- >> Compressing that time is going to put dollars in your pocket. >> Exactly. Right. >> So once, >> If you got it right. >> If you get it right, of course. So there are, there are a couple of issues with that. Once you figure out that model is working, that's perfect. People don't realize, the moment you decide that moment when the decision is made, it's like a new car. After you purchase the value decreases on a minute basis. Same thing with the models. Once the model is created, you need to be in production right away because it starts losing it value on a seconds minute basis. So issue number one, how fast can I get it over there? So your deployment, you are inferencing efficiently at the edge locations, your optimization, your security, all of this is at issue. But you know what is more important than that in the last mile? You keep the model up, you continue to work on, again, going back to the car analogy, at one point you got to figure out your car is costing more than to operate. So you got to get a new car, right? And that's the same thing with the models as well. If your model has reached a stage, it is actually a potential risk for your operation. To give you an idea, if Uber has a model, the first time when you get a car from going from point A to B cost you $60. If the model decayed the next time I might give you a $40 rate, I would take it definitely. But it's lost for the company. The business risk associated with operating on a bad model, you should realize it immediately, pull the model out, retrain it, redeploy it. That's is key. >> And that's got to be huge in security model recency and security to the extent that you can get real time is big. I mean you, you see Palo Alto, CrowdStrike, a lot of other security companies are injecting AI. Again, they won't show up in the ETR ML/AI taxonomy per se as a pure play. But ServiceNow is another company that you have have mentioned to me, offline. AI is just getting embedded everywhere. >> Yep. >> And then I'm glad you brought up, kind of real-time inferencing 'cause a lot of the modeling, if we can go back to the last point that we're going to make, a lot of the AI today is modeling done in the cloud. The last point we wanted to make here, I'd love to get your thoughts on this, is real-time AI inferencing for instance at the edge is going to become increasingly important for us. It's going to usher in new economics, new types of silicon, particularly arm-based. We've covered that a lot on "Breaking Analysis", new tooling, new companies and that could disrupt the sort of cloud model if new economics emerge. 'Cause cloud obviously very centralized, they're trying to decentralize it. But over the course of this decade we could see some real disruption there. Andy, give us your final thoughts on that. >> Yes and no. I mean at the end of the day, cloud is kind of centralized now, but a lot of this companies including, AWS is kind of trying to decentralize that by putting their own sub-centers and edge locations. >> Local zones, outposts. >> Yeah, exactly. Particularly the outpost concept. And if it can even become like a micro center and stuff, it won't go to the localized level of, I go to a single IOT level. But again, the cloud extends itself to that level. So if there is an opportunity need for it, the hyperscalers will figure out a way to fit that model. So I wouldn't too much worry about that, about deployment and where to have it and what to do with that. But you know, figure out the right business use case, get the right data, get the ethics and governance place and make sure they get it to production and make sure you pull the model out when it's not operating well. >> Excellent advice. Andy, I got to thank you for coming into the studio today, helping us with this "Breaking Analysis" segment. Outstanding collaboration and insights and input in today's episode. Hope we can do more. >> Thank you. Thanks for having me. I appreciate it. >> You're very welcome. All right. I want to thank Alex Marson who's on production and manages the podcast. Ken Schiffman as well. Kristen Martin and Cheryl Knight helped get the word out on social media and our newsletters. And Rob Hoof is our editor-in-chief over at Silicon Angle. He does some great editing for us. Thank you all. Remember all these episodes are available as podcast. Wherever you listen, all you got to do is search "Breaking Analysis" podcast. I publish each week on wikibon.com and silicon angle.com or you can email me at david.vellante@siliconangle.com to get in touch, or DM me at dvellante or comment on our LinkedIn posts. Please check out ETR.AI for the best survey data and the enterprise tech business, Constellation Research. Andy publishes there some awesome information on AI and data. This is Dave Vellante for theCUBE Insights powered by ETR. Thanks for watching everybody and we'll see you next time on "Breaking Analysis". (gentle closing tune plays)

>>Hello and welcome back to the Cube's coverage of AWS Reinvent 2022. I'm John Furrier, host of the Cube. We got a great conversation with Patrick Kauflin, vice president of Go to Market Strategy and specialization at Splunk. We're talking about the open cybersecurity scheme of framework, also known as the O C sf, a joint strategic collaboration between Splunk and aws. It's got a lot of traction momentum. Patrick, thanks for coming on the cube for reinvent coverage. >>John, great to be here. I'm excited for this. >>You know, I love this open source movement and open source and continues to add value, almost sets the standards. You know, we were talking at the CNCF Linux Foundation this past fall about how standards are coming outta open source. Not so much the the classic standards groups, but you start to see the developers voting with their code groups deciding what to adopt de facto standards and security is a real key part of that where data becomes key for resilience. And this has been the top conversation at reinvent and all around the industry, is how to make data a key part of building into cyber resilience. So I wanna get your thoughts about the problem that you see that's emerging that you guys are solving with this group kind of collaboration around the ocs f >>Yeah, well look, John, I I think, I think you, you've already, you've already hit the high notes there. Data is proliferating across the enterprise. The attack surface area is rapidly expanding. The threat landscape is ever changing. You know, we, we just had a, a lot of scares around open SSL before that we had vulnerabilities and, and Confluence and Atlassian, and you go back to log four J and SolarWinds before that and, and challenges with the supply chain. In this year in particular, we've had a, a huge acceleration in, in concerns and threat vectors around operational technology. In our customer base alone, we saw a huge uptake, you know, and double digit percentage of customers that we're concerned about the traditional vectors like, like ransomware, like business email compromise, phishing, but also from insider threat and others. So you've got this, this highly complex environment where data continues to proliferate and flow through new applications, new infrastructure, new services, driving different types of outcomes in the digitally transformed enterprise of today. >>And, and what happens there is, is our customers, particularly in security, are, are left with having to stitch all of this together. And they're trying to get visibility across multiple different services, infrastructure applications across a number of different point solutions that they've bought to help them protect, defend, detect, and respond better. And it's a massive challenge. And you know, when our, when our customers come to us, they are often looking for ways to drive more consolidation across a variety of different solutions. They're looking to drive better outcomes in terms of speed to detection. How do I detect faster? How do I bind the thing that when bang in the night faster? How do I then fix it quickly? And then how do I layer in some automation so hopefully I don't have to do it again? Now, the challenge there that really OCF Ocsf helps to, to solve is to do that effectively, to detect and to respond at the speed at which attackers are demanding. >>Today we have to have normalization of data across this entire landscape of tools, infrastructure, services. We have to have integration to have visibility, and these tools have to work together. But the biggest barrier to that is often data is stored in different structures and in different formats across different solution providers, across different tools that are, that are, that our customers are using. And that that lack of data, normalization, chokes the integration problem. And so, you know, several years ago, a number of very smart people, and this was, this was a initiative s started by Splunk and AWS came together and said, look, we as an industry have to solve this for our customers. We have to start to shoulder this burden for our customers. We can't, we can't make our customers have to be systems integrators. That's not their job. Our job is to help make this easier for them. And so OCS was born and over the last couple of years we've built out this, this collaboration to not just be AWS and Splunk, but over 50 different organizations, cloud service providers, solution providers in the cybersecurity space have come together and said, let's decide on a single unified schema for how we're gonna represent event data in this industry. And I'm very proud to be here today to say that we've launched it and, and I can't wait to see where we go next. >>Yeah, I mean, this is really compelling. I mean, it's so much packed in that, in that statement, I mean, data normalization, you mentioned chokes, this the, the solution and integration as you call it. But really also it's like data's not just stored in silos. It may not even be available, right? So if you don't have availability of data, that's an important point. Number two, you mentioned supply chain, there's physical supply chain that's coming up big time at reinvent this time as well as in open source, the software supply chain. So you now have the perimeter's been dead for multiple years. We've been talking with that for years, everybody knows that. But now combined with the supply chain problem, both physical and software, there's so much more to go on. And so, you know, the leaders in the industry, they're not sitting on their hands. They know this, but they're just overloaded. So, so how do leaders deal with this right now before we get into the ocs f I wanna just get your thoughts on what's the psychology of the, of the business leader who's facing this landscape? >>Yeah, well, I mean unfortunately too many leaders feel like they have to face these trade offs between, you know, how and where they are really focusing cyber resilience investments in the business. And, and often there is a siloed approach across security, IT developer operations or engineering rather than the ability to kind of drive visibility integration and, and connection of outcomes across those different functions. I mean, the truth is the telemetry that, that you get from an application for application performance monitoring or infrastructure monitoring is often incredibly valuable when there's a security incident and vice versa. Some of the security data that, that you may see in a security operation center can be incredibly valuable in trying to investigate a, a performance degradation in an application and understanding where that may come from. And so what we're seeing is this data layer is collapsing faster than the org charts are or the budget line items are in the enterprise. And so at Splunk here, you know, we believe security resilience is, is fundamentally a data problem. And one of the things that we do often is, is actually help connect the dots for our customers and bring our customers together across the silos they may have internally so that they can start to see a holistic picture of what resilience means for their enterprise and how they can drive faster detection outcomes and more automation coverage. >>You know, we recently had an event called Super Cloud, we're going into the next gen kind of a cloud, how data and security are all kind of part of this NextGen application. It's not just us. And we had a panel that was titled The Innovators Dilemma, kind of talk about you some of the challenges. And one of the panelists said, it's not the innovator's dilemma, it's the integrator's dilemma. And you mentioned that earlier, and I think this a key point right now into integration is so critical, not having the data and putting pieces together now open source is becoming a composability market. And I think having things snap together and work well, it's a platform system conversation, not a tool conversation. So I really wanna get into where the OCS f kind of intersects with this area people are working on. It's not just solution architects or cloud cloud native SREs, especially where DevSecOps is. So this that's right, this intersection is critical. How does Ocsf integrate into that integration of the data making that available to make machine learning and automation smarter and more relevant? >>Right, right. Well look, I mean, I I think that's a fantastic question because, you know, we talk about, we use Bud buzzwords like machine learning and, and AI all the time. And you know, I know they're all over the place here at Reinvent and, and the, there's so much promise and hope out there around these technologies and these innovations. However, machine learning AI is only as effective as the data is clean and normalized. And, and we will not realize the promise of these technologies for outcomes in resilience unless we have better ways to normalize data upstream and better ways to integrate that data to the downstream tools where detection and response is happening. And so Ocsf was really about the industry coming together and saying, this is no longer the job of our customers. We are going to create a unified schema that represents the, an event that we will all bite down on. >>Even some of us are competitors, you know, this is, this is that, that no longer matters because at the point, the point is how do we take this burden off of our customers and how do we make the industry safer together? And so 15 initial members came together along with AWS and Splunk to, to start to create that, that initial schema and standardize it. And if you've ever, you know, if you've ever worked with a bunch of technical grumpy security people, it's kind of hard to drive consensus about around just about anything. But, but I, I'm really happy to see how quickly this, this organization has come together, has open sourced the schema, and, and, and just as you said, like I think this, this unlocks the potential for real innovation that's gonna be required to keep up with the bad guys. But right now is getting stymied and held back by the lack of normalization and the lack of integration. >>I've always said Splunk was a, it eats data for breakfast, lunch, and dinner and turns it into insights. And I think you bring up the silo thing. What's interesting is the cross company sharing, I think this hits point on, so I see this as a valuable opportunity for the industry. What's the traction on that? Because, you know, to succeed it does take a village, it takes a community of security practitioners and, and, and architects and developers to kind of coalesce around this defacto movement has been, has been the uptake been good? How's traction? Can you share your thoughts on how this is translating across companies? >>Yeah, absolutely. I mean, look, I, I think cybersecurity has a, has a long track record of, of, of standards development. There's been some fantastic standards recently. Things like sticks and taxi for threat intelligence. There's been things like the, you know, the Mir attack framework coming outta mi mir and, and, and the adoption, the traction that we've seen with Attack in particular has been amazing to, to watch how that has kind of roared onto the scene in the last couple of years and has become table stakes for how you do security operations and incident response. And, you know, I think with ocs f we're gonna see something similar here, but, you know, we are in literally the first innings of, of this. So right now, you know, we're architecting this into our, into every part of our sort of backend systems here at Polan. I know our our collaborators at AWS and elsewhere are doing it too. >>And so I think it starts with bringing this standard now that the standard exists on a, you know, in schema format and there, there's, you know, confluence and Jira tickets around it, how do we then sort of build this into the code of, of the, the collaborators that have been leading the way on this? And you know, it's not gonna happen overnight, but I think in the coming quarters you'll start to see this schema be the standard across the leaders in this space. Companies like Splunk and AWS and others who are leading the way. And often that's what helps drive adoption of a standard is if you can get the, the big dogs, so to speak, to, to, to embrace it. And, and, you know, there's no bigger one than aws and I think there's no, no more important one than Splunk in the cybersecurity space. And so as we adopt this, we hope others will follow. And, and like I said, we've got over 50 organizations contributing to it today. And so I think we're off to a running >>Start. You know, it's interesting, choking innovation or having things kind of get, get slowed down has really been a problem. We've seen successes recently over the past few years. Like Kubernetes has really unlocked and accelerated the cloud native worlds of runtime with containers to, to kind of have the consensus of the community to say, Hey, if we just do this, it gets better. I think this is really compelling with the o the ocs F because if people can come together around this and get unified as well as all the other official standards, things can go highly accelerated. So I think, I think it looks really good and I think it's great initiative and I really appreciate your insight on that, on, on your relationship with Amazon. Okay. It's not just a partnership, it's a strategic collaboration. Could you share that relationship dynamic, how to start, how's it going, what's strategic about it? Share to the audience kind of the relationship between Splunk and a on this important OCS ocsf initiative. >>Look, I, I mean I think this, this year marks the, the 10th year anniversary that, that Splunk and AWS have been collaborating in a variety of different ways. I, I think our, our companies have a fantastic and, and long standing relationship and we've, we've partnered on a number of really important projects together that bring value obviously to our individual companies, but also to our shared customers. When I think about some of the most important customers at Splunk that I spend a significant amount of time with, I I I know how many of those are, are AWS customers as well, and I know how important AWS is to them. So I think it's, it's a, it's a collaboration that is rooted in, in a respect for each other's technologies and innovation, but also in a recognition that, that our shared customers want to see us work better together over time. And it's not, it's not two companies that have kind of decided in a back room that they should work together. It's actually our customers that are, that are pushing us. And I think we're, we're both very customer centric organizations and I think that has helped us actually be better collaborators and better partners together because we're, we're working back backwards from our customers >>As security becomes a physical and software approach. We've seen the trend where even Steven Schmidt at Amazon Web Services is, is the cso, he is not the CSO anymore. So, and I asked him why, he says, well, security's also physical stuff too. So, so he's that's right. Whole lens is now expanded. You mentioned supply chain, physical, digital, this is an important inflection point. Can you summarize in your mind why open cybersecurity schema for is important? I know the unification, but beyond that, what, why is this so important? Why should people pay attention to this? >>You know, I, if, if you'll let me be just a little abstract in meta for a second. I think what's, what's really meaningful at the highest level about the O C S F initiative, and that goes beyond, I think, the tactical value it will provide to, to organizations and to customers in terms of making them safer over the coming years and, and decades. I think what's more important than that is it's really the, one of the first times that you've seen the industry come together and say, we got a problem. We need to solve. That, you know, doesn't really have anything to do with, with our own economics. Our customers are, are hurt. And yeah, some of us may be competitors, you know, we got different cloud service providers that are participating in this along with aws. We got different cybersecurity solution providers participating in this along with Splunk. >>But, but folks who've come together and say, we can actually solve this problem if, if we're able to kind of put aside our competitive differences in the markets and approach this from the perspective of what's best for information security as a whole. And, and I think that's what I'm most proud of and, and what I hope we can do more of in other places in this industry, because I think that kind of collaboration from real market leaders can actually change markets. It can change the, the, the trend lines in terms of how we are keeping up with the bad guys. And, and I'd like to see a lot more of >>That. And we're seeing a lot more new kind of things emerging in the cloud next kind of this next generation architecture and outcomes are happening. I think it's interesting, you know, we always talk about sustainability, supply chain sustainability about making the earth a better place. But you're hitting on this, this meta point about businesses are under threat of going under. I mean, we want to keep businesses to businesses to be sustainable, not just, you know, the, the environment. So if a business goes outta business business, which they, their threats here are, can be catastrophic for companies. I mean, there is, there is a community responsibility to protect businesses so they can sustain and and stay Yeah. Stay producing. This is a real key point. >>Yeah. Yeah. I mean, look, I think, I think one of the things that, you know, we, we, we complain a lot of in, in cyber security about the lack of, of talent, the talent shortage in cyber security. And every year we kinda, we kind of whack ourselves over the head about how hard it is to bring people into this industry. And it's true. But one of the things that I think we forget, John, is, is how important mission is to so many people in what they do for a living and how they work. And I think one of the things that cybersecurity is strongest in information Security General and has been for decades is this sense of mission and people work in this industry be not because it's, it's, it's always the, the, the most lucrative, but because it, it really drives a sense of safety and security in the enterprises and the fabric of the economy that we use every day to go through our lives. And when I think about the spun customers and AWS customers, I think about the, the different products and tools that power my life and, and we need to secure them. And, and sometimes that means coming to work every day at that company and, and doing your job. And sometimes that means working with others better, faster, and stronger to help drive that level of, of, of maturity and security that this industry >>Needs. It's a human, is a human opportunity, human problem and, and challenge. That's a whole nother segment. The role of the talent and the human machines and with scale. Patrick, thanks so much for sharing the information and the insight on the Open cybersecurity schema frame and what it means and why it's important. Thanks for sharing on the Cube, really appreciate it. >>Thanks for having me, John. >>Okay, this is AWS Reinvent 2022 coverage here on the Cube. I'm John Furry, you're the host. Thanks for watching.

foreign welcome back to thecube's coverage of AWS re invent 2022 I'm John Furrier host of thecube we've got a great conversation with Patrick Coughlin vice president of go to market strategy and specialization at Splunk we're talking about the open cyber security schema framework also known as the ocsf a joint strategic collaboration between Splunk and AWS it's got a lot of traction momentum Patrick thanks for coming on thecube for reinvent coverage John great to be here I'm excited for this you know I love this open source movement and open source continues to add value almost sets the standards you know we were talking at the cncf Linux Foundation this past fall about how standards are coming out of Open Source not so much the the classic standards groups but you start to see the developers voting with their code groups deciding what to adopt to fact those standards and security is a real key part of that where data becomes key for resilience and this has been the top conversation at re invent and all around the industry is how to make data a key part of building into cyber resilience so I want to get your thoughts about the problem that you see that's emerging that you guys are solving with this group kind of collaboration around the ocsf yeah well look John I I think I think you you've already you've already hit the high notes there uh data is proliferating across the Enterprise uh the attack surface area is rapidly expanding the threat landscape is Ever Changing uh you know we we just had a a lot of uh uh scares around openssl before that we had vulnerabilities and Confluence in atlassian and you go back to log 4J and solarwinds before that um and challenges with the supply chain uh in this year in particular we've had a huge acceleration in in concerns and threat vectors around uh operational technology in our customer base alone we saw a huge uptick you know in double digit percentage of customers that we're concerned about the traditional vectors like like ransomware uh like business email compromise phishing but also from Insider threat and others um so you've got this this highly complex Flex environment where data continues to proliferate and flow through new applications new infrastructure new Services driving different types of outcomes in the digitally transformed Enterprise of today and and what happens there is is our customers particularly in security are left with having to stitch all of this together and they're trying to get visibility across multiple different Services infrastructure applications across a number of different point solutions that they've bought to help them protect defend detect and respond better and it's a massive Challenge and uh you know when our when our customers come to us they are often looking for ways to drive more consolidation uh across a variety of different solutions they're looking to drive better outcomes in terms of speed to detection how do I detect faster how do I find the thing that when banging in the night faster um how do I then fix it quickly and then how do I layer in some automation so hopefully I don't have to do it again now the Challenger that really ocf ocsf helps to to solve is to do that effectively to detect and to respond to the speed at which attackers are demanding today we have to have normalization of data across this entire landscape of tools infrastructure Services we have to have integration to have visibility um and these tools have to work together but the biggest barrier to that is often data is stored in different structures and in different formats across different solution providers across different tools that are that are that our customers are using um and that that lack of data normalization chokes the integration problem and so um you know several years ago a number of very smart people in this position this was a initiative started by Splunk and AWS came together and said look we as an industry have to solve this for our customers we have to start to shoulder this burden for our customers we can't we can't make our customers have to be systems integrators that's not their job our job is to help make this easier for them and so ocsf was born and over the last couple of years um we've built out this this collaboration to not just be AWS and Splunk uh but over uh 50 different organizations um uh um cloud service providers solution providers in the cyber security space have come together and said let's decide on a single unified schema for how we're going to represent event data in this industry um and uh I'm very proud to be here today to say that we've launched it and and um uh I can't wait to see where we go next yeah I mean this is really compelling I mean there's so much packed in that in that statement I mean data normalization you mentioned chokes this the the solution and the integration as you call it but really also it's like data is not just stored in silos it may not even be available right so if you don't have availability of data that's an important Point number two you mentioned supply chain there's physical supply chain is coming up big time at re invent this time as well as in open source the software supply chain so you now have the perimeter has been dead for multiple years we've been talking about that for years everybody knows that but now combined with the supply chain problem both physical and software there's so much more to go on and so you know the leaders in the industry they're not sitting on their hands they know this but they're just overloaded so so how do leaders deal with this right now before we get into the ocsf I want to just get your thoughts on what's the psychology of the of the business leader who's facing this landscape yeah well I mean unfortunately too many leaders feel like they have to face these trade-offs between you know how and where they are really focusing cyber resilience investments in the business um and and often there is a siled approach across security I.T developer operations or engineering rather than the ability to kind of Drive visibility integration and and connection of outcomes across those different functions I mean the truth is the Telemetry that that you get from an application for application performance monitoring or infrastructure monitoring is often incredibly valuable when there's a security incident and vice versa some of the security data um that you may see in a security operations center can be incredibly valuable when trying to investigate a performance degradation in an application and understanding where that may come from and so what we're seeing is this data layer is collapsing faster than the org charts are or the budget line items are in the Enterprise and so at Splunk here you know we believe security resilience is is fundamentally a data problem and one of the things that we do often is is actually help connect the dots for our customers and bring our customers together across the silos they may have internally so that they can start to see a holistic picture of what resilience means for their Enterprise and how they can drive faster detection outcomes and more automation coverage you know we recently had an event called super cloud we're going into the next gen kind of a cloud how data and security are all kind of part of this next-gen applications not just SAS and we had a panel that was titled the innovators dilemma kind of talk about getting some of the challenges and one of the panelists said it's not the innovators dilemma it's the integrators dilemma and you mentioned that earlier I think this is a key point right now integration is so critical not having the data and putting pieces together and now open source is becoming a composability market and I think having things snap together and work well it's a platform system conversation not a tool conversation so I really want to get into where the ocsf kind of intersects with this area people are working on it's not just solution Architects or cloud cloud native sres especially where devsecops is so this this intersection is critical how does ocsf integrate into that integration of the data making that available to make machine learning and automation smarter and more relevant right right well look I mean I I think that's a fantastic question because you know we talk about we use buzzwords like machine learning and AI all the time and you know I I know they're all over the place here at reinvented and and um there's so much promise and hope out there around these Technologies and these Innovations however uh machine learning AI is only as effective as the data is clean and normalized uh and and we will not realize the promise of these Technologies for outcomes in resilience unless we have better ways to normalize data upstream and better ways to integrate that data to the downstream tools where detection and response is happening and so ocsf was really about the industry coming together and saying this is no longer the job of our customers we are going to create a unified schema that represents the an event that we will all bite down on even some of us are competitors you know this is this is that that no longer matters because at the point the point is how do we take this burden off of our customers and how do we make the industry safer together um and so 15 initial members came together um along with AWS and Splunk to to start to create that uh that initial schema and standardize it and if you've ever you know if you ever worked with a bunch of technical grumpy security people it's kind of hard to drive consensus about around just about anything but uh um but I'm really happy to see how quickly this this organization Has Come Together has open sourced the schema um and and just as you said like I think this this unlocks the potential for real Innovation that's going to be required to keep up with the bad guys but right now is getting stymied and held back by the lack of normalization and the lack of integration I've always said Splunk was a it's AIDS data for breakfast lunch and dinner and turns it into insights and I think you bring up The Silo thing what's interesting is the cross company sharing I think this hits point on so I see this as a valuable opportunity for the industry what's the traction on that because you know to succeed it does take a village takes a community of security practitioners and and Architects and developers to kind of coalesce around this de facto movement has been has been uptake been good that's attraction can you share your thoughts on how this is translating across companies yeah absolutely I mean look I I think um cyber security has a long track record of of Standards development um there's been some fantastic standards recently things like um sticks and taxi for threat intelligence there's been things like the you know the minor attack framework coming out of my miter and and the adoption the traction that we've seen with attack in particular has been amazing to watch how that has kind of roared onto the scene in the last couple of years and has become table Stakes for um how you do security operations and incident response um and you know I think with ocsf we're going to see something similar here but you know we are in literally the first Innings of of this um so right now you know we're architecting this into our um into every part of our sort of back end systems here at spelunk I know um our collaborators at AWS and elsewhere are doing it too and so I think it starts with bringing this standard now the standard exists on a uh you know in schema format um and there's you know Confluence and jira tickets around it how do we then sort of build this into the code of of the the collaborators that have been leading the way on this and you know it's not going to happen overnight but I think in the coming quarters you'll start to see this schema um be the standard um across the leaders in this space companies like Splunk and AWS and others who are leading the way and often that's what helps Drive adoption of a standard is if you can get the big dogs so to speak to to embrace it and you know there's no bigger one than AWS and I think there's no no more important one than Splunk in the cyber security space and so as we adopt this we hope others will follow and like I said we've got over 50 organizations contributing to it today and so um I think we're off to a running start you know it's interesting choking Innovation or having things kind of get get slowed down has really been a problem we've seen successes recently over the past few years like kubernetes has really unlocked and accelerated the cloud native worlds of runtime with containers to kind of have the consensus of the community say hey if you we just do this it gets better I think this is really compelling with the ocsf because if people can come together around this and get unified as well as other the other official standards things can go highly accelerated so I think I think it looks really good and I think it's great initiative and I really appreciate your Insight on that on on your relationship with Amazon okay it's not just the Partnerships it's a strategic collaboration could you share that uh relationship Dynamic how to start how's it going what's strategic about it share to the audience kind of the relationship between Splunk and natives on this important ocsf initiative look I I mean I think this this year marks the the 10th year anniversary that that Splunk and AWS have been collaborating in a variety of different ways um I I think our our companies have um a fantastic and long-standing relationship and we've we've partnered on a number of really important projects together that bring value um obviously to our individual companies uh but also to our shared customers um uh when I think about some of the most important customers at Splunk that I spend a significant amount of time with um uh I I know how many of those are our AWS customers as well and I know how important AWS is to them so I think it's it's a it's a collaboration that is rooted in in a respect for each other's Technologies um and Innovation but also in a recognition that that our shared customers want to see us work better together over time and it's not it's not two companies that have kind of decided in a back room that they should work together it's actually our customers that are that are pushing us and I think we're both very customer-centric organizations and I think that has helped us actually be better collaborators and better Partners together um because we're working back backwards from our customers as security becomes a physical and software approach we've seen the trend where even Steven Schmidt at Amazon web services is the CSO he's not the CSO anymore so why he says well security is also physical stuff too so so lens is now expanded you mentioned supply chain physical digital this is an important inflection point can you summarize in your mind why open cyber security scheme information is important I know the unification but beyond that what why is this so important why should people pay attention to this you know I if if you'll let me be just a little abstract and meta for a second yeah I think what's what's really meaningful at the highest level about the ocsf initiative um and then it goes beyond I think the Tactical value it will provide to to organizations and to customers in terms of making them safer um over the coming years and and decades I think what's more important than that is it's really the one of the first times that you've seen um the industry come together and say we got a problem we need to solve that you know doesn't really have anything to do with with our own economics um our customers are are hurting and yeah some of us may be competitors um uh you know we got different cloud service providers that are participating in this along with AWS we've got different cyber security solution providers participating in this along with spelunk um but but folks have come together and say we can actually solve this problem um if if we're able to kind of put aside our competitive differences in the markets and approach this from the perspective of what's best for information security as a whole um and and I think that's what I'm most proud of uh and and what I hope we can do more of in other places in this industry because I think that kind of collaboration from real Market leaders can actually um change markets it can change the the the trend lines in terms of how we are keeping up with the bad guys and and I'd like to see a lot more of that and we're seeing a lot more new kind of things emerging in the cloud next kind of this next Generation architecture and alcohol thumbs are happening I think it's interesting you know we always talk about sustainability supply chain sustainability about making the earth a better place but you're hitting on this this meta point about businesses are under threat of going under I mean we want to keep businesses to businesses to be sustainable not just you know the the environment so if a business goes out of business which the threats here are can be catastrophic for companies I mean there is there is a community responsibility to protect businesses so they can sustain and stay stay producing this is a real key point yeah yeah I mean look I think I think one of the things that you know we We complain a lot in in cyber security about the lack of of talent the talent shortage and cyber security and every year we kind of we kind of uh whack ourselves over the head about how hard it is to bring people into this industry and it's true um but one of the things that I think we forget John is is how important mission is to so many people in what they do for a living and how they work and I think one of the things that cyber security is strongest in information security General and has been for decades is this sense of mission and people work in this industry not because it's it's it's always the the the most lucrative but because it really drives a sense of um Safety and Security in the Enterprises and the fabric of the economy that we use every day to go through our lives and when I think about the sport customers and AWS customers I think about um um the the different products and tools that power my life and and we need to secure them and and sometimes that means coming to work every day at that company and doing your job and sometimes that means working with others better faster and stronger to help drive that level of of maturity and security that this industry needs it's a human it's a human opportunity human problem and and challenge that's a whole other segment the role of the talent and the human machines and with scale Patrick thanks so much for sharing the information and the Insight on the open cyber security schema frame and what it means and why it's important thanks for sharing on thecube really appreciate it thanks for having me John okay this is AWS re invent 2022 coverage here on thecube I'm John Furrier the host thanks for watching foreign [Music]

>>And welcome to this cube conversation here in Palo Alto, California. I'm John furrier, host of the cube. We have a great guest coming in remotely from LA Christian Hernandez developer experienced lead at code fresh code fresh IO. Recently they were on our feature at a startup showcase series, season two episode one cloud data innovations, open source innovations, all good stuff, Christian. Thanks for coming on this cube conversation. >>Thank you. Thank you, John. Thank you for having me on, >>You know, I'm I was really impressed with code fresh. My met with the founders on here on the cube because GI ops AI, everything's something ops devs dev sec ops. You've got AI ops. You've got now GI ops, essentially operationalizing the software future is here and software's eating the world is, was written many years ago, but it's open source is now all. So all things software's open source and that's kind of a done deal. It's only getting better and better. Mainstream companies are contributing. You guys are on this wave of, of this open source tsunami and you got cloud scale. Automation's right there, machine learning, all this stuff is now the next gen of, of, of code, right? So you, your code fresh and your title is developer experience lead. What does that mean right now? What does it mean to be a developer experience lead? Like you make sure people having a good experience. Are you developing you figuring out the product? What does that mean? >>Yeah. That's and it's also part of the, the whole Debre explosion that's happening right now. I believe it's, you know, everyone's always asking, well, what, you know, what is developer advocate? What does that mean developer experience? What does that mean? So, so you, you kind of hit the nail on the head a little bit up there in, in the beginning, is that the, the experience of the developer when using a particular platform, right? Especially the code flash platform. That is my responsibility there at code fresh to enable, to enable end users, to enable partners, to enable, you know, anyone that wants to use the code fresh platform for their C I C D and get ops square flows. So that's, that's really my, my corner of the world is to make sure their experience is great. So that's, it's really what, what I'm here to do >>At food fresh. You know, one of the things I can say of my career, you've been kind of become a historian over time. When I was a developer back in the old days, it was simply you compiled stuff, you did QA on it. You packaged it out. You wanted out the door and you know, that was a workflow right now with the cloud. I was talking with your founders, you got new abstraction layers. Cloud has changed again again, open source. So newer things are coming, right? Like, like, like Kubernetes for instance is a great example that came out of the open source kind of the innovations. But that, and Hadoop, we were mentioning before he came on camera from a storage standpoint, kind of didn't make it because it was just too hard. Right. And it made the developer's job harder. And then it made the developer's requirements to be specialized. >>So you had kind of two problems. You had hard to use a lot of friction and then it required certain expertise when the developers just want to code. Right. So, so you have now the motion of, with GI ops, you guys are in the middle of kinda this idea of frictionless based software delivery with the cloud. So what's different now, can you talk about that specific point because no one wants to be, do hard work and have to redo things. Yeah. Shift left and all that good stuff. What's hard now, what do you guys solve? What's the, what's the friction that you're taking out what's to become frictionless. >>Yeah. Yeah. And you, you, you mentioned a very interesting point about how, you know, things that are coming out almost makes it seem harder nowadays to develop an application. You used to have it to where, you know, kind of a, sort of a waterfall sort of workflow where, you know, you develop your code, you know, you compile it. Right. You know, I guess back in the day, Java was king. I think Java still is, has a, is a large footprint out there where you would just compile it, deploy it. If it works, it works. Alright cool. And you have it and you kind of just move it along in its process. Whereas I think the, the whole idea of, I think Netflix came out with like the, the fail often fail fast release often, you know, the whole Atlassian C I C D thing, agile thing came into play. >>Where now it's, it's a little bit more complex to get your code out there delivered to get your code from one environment to the other environment, especially with the, the Avan of Kubernetes and cloud native architecture, where you can deploy and have this imutable infrastructure where you can just deploy and automate so quickly. So often that there needs to be some sort of new process now into place where to have a new process, like GI ops to where it'll, it it's frictionless, meaning that it's, it, it makes it that process a little easier makes that little, that comp that complex process of deploying onto like a cloud native architecture easier. So that way, as you said before, returning the developers to back to what they care about, mot, the most is just code. I just want to code. >>Yeah. You know, the other thing, cool thing, Christian, I wanna bring up and we'll get into some of the specifics around Argo specifically CD is that the community is responding as a kind of, it takes a village kind of mindset. People are getting into this just saying, Hey, if we can get our act together around some de facto workflows and de facto capabilities, everyone wins. It's a rising tide, floats all boats, kind of concept. CNCF certainly has been a big part of that. Even seen some of the big hyper scales getting behind it. But you guys are part of the founding members of the open get ups working group, Amazon Azure, GitHub, red hat Weaveworks and then a ton of contributors. Okay. So this is kind of cool. This means that there's like people behind this thing. Look, we gotta get here faster. What happened at co con this year? You guys had some news around Argo and you had some news around the hosted solution. Can you take a minute to explain two things, one the open community vibe, and then two, what you guys announced at Coon in Spain. >>Yeah. Yeah. So as far as open get ups, that was, you know, as you said before, code fresh was part of that, that founding committee. Right. Of, of group of people trying to figure out, define what get ups is. Right. We're trying to bring it beyond the, you know, the, the hype word, right beyond just like a marketing term to where we actually define what it actually is, because it is actually something that's out there that people are doing. Right. A lot of people, you know, remember that the, the Chick-fil-A story where it's like, they, they are completely doing, you know, this get ops thing, we're just now wanting, putting definition around it. So that was just amazing to see out at there in, in Cuban. And, but like you said, in QAN, we, you know, we're, we're, we're taking some of that, that acceleration that we see in the community to, and we, we announce our, our hosted get ops offering. >>Right. So hosted get ops is something that our customers have been asking for for a while. Many times when, you know, someone wants to use something like Argo CD, the, in, they install it on their cluster, they get up and running. And, but with, with all that comes like the feed and care of that platform, and, you know, not only just keeping the lights on, but also management security, you know, general maintenance, you know, all the things that, that come along with managing a system. And on top of that comes like the scale aspect of it. Right. And so with scale, so a lot of people go with like a hub and spoke others, go with like a fleet design in, in either case, right. There's, there's a challenge for the feet and care of it. Right. And so with code fresh coast of get ups, we take that management headache away. >>Right? So we, we take the, the, the management of, of Argo CD, the management of, of all of that, and kind of just offer Argo CD as a surface, right. Which offers, you know, allows users to, you know, let us take care of all the, of the get offs, runtime. And so they can concentrate on, you know, their application deployments. Right. And you also get things like Dora metrics, right. Integrated with the platform, you have the ability to integrate multiple CI providers, you know, like get hub actions or whatever, existing Jenkins pipelines. And really that, that code fresh platform becomes like your get ops platform becomes like, you know, your, your central view of the world of, of your, you know, get ups processes. >>Yeah. I mean, that whole single source of truth concept is really kind of needed. I gotta ask you though, with the popularity of the Argo CD on get ups internally, right. That's been clear, right. Kubernetes, the way that's going, it's accelerating fast. People want simple it's scaling, you got automation built in all that good stuff. What was the driver behind the hosted get up solution? Was it customer needs? Was it efficiency all the above? What was specifically and, and why would someone want to have the hosted versus say internal? >>Yeah. So it's, it was really driven by, you know, customer need been something that the customers have been asking for. And it's also been something that, you know, you, you, you have a process of developing an application to, you know, you know, a fleet of clusters in a traditional, you know, I keep saying traditional, get outs practice as if get outs are so old. And, you know, in, you know, when, when, when people first start out, they'll start, you know, installing Argo city on all these clusters and trying to manage that at scale it's, it's, it, it seemed like there was, you know, it it'd be nice if we can just like, be able to consume this as a service. So we don't have to like, worry about, you know, you know, best practices. We don't have to worry about security. We don't just, all of that is taken care of and managed by us at code fresh. So this is like something that, you know, has been asked for and, and something that, you know, we believe will accelerate, you know, developers into actually developing their, their applications. They don't have to worry about managing >>The platform. So just getting this right. Hosted, managed service by you guys on this one, >>Correct? Yes. >>Okay. Got it. All right. So let me, let me get in the Argo real quick, just to kind of just level set for the folks that are, are leaning into this and then kicking the tires. Where are we with Argo? What, why was it so popular? What did it do specifically? Did it just make it easier for developers to manage and monitor Kubernetes, keep 'em updated? What was the specific value behind Argo? Where, where, where did it come from and why is it so popular? >>Yeah, so Argo the Argo project, which is made up of, of a few tools, usually when people say Argo, they meet, they they're talking about Argo CD, but there's also Argo workflows, Argo events, Argo notifications. And, and like I said before, CD with that, and that is something that was developed internally at Intuit. Right? So for those of who don't know, Intuit is the company behind turbo tax. So for those, those of us in the us, we, we know, you know, we know that season all too well, the tax season. And so that was a tool that was developed internally. >>And by the way, Intuit we've done many years. They're very huge cloud adopters. They've been on that train from the day one. They've been, they've been driving a lot of cloud scale too. Sorry >>To interrupt. Yeah. And, and, and yeah, no, and, and, and also, you know, they, they were always open source first, right. So they've always had, you know, they developed something internally. They always had the, the intention of opensourcing it. And so it was really a tool that was born internally, and it was a tool that helped them, you know, get stuff done with Kubernetes. And that's kind of like the tagline they use for, for the Argo project is you need to get stuff done. They wanted their developers to focus less on deploying the application and more right. More than on writing the application itself. And so the, and so the Argo project is a suite of tools essentially that helps deploy onto Kubernetes, you know, using get ups as that, you know, that cornerstone in design, right in the design philosophy, it's so popular because of the ease of use and developer friendliness aspect of it. It's, it's, it's, it's meant to be simple right. In and simple in a, in a good sense of getting up and running, which attracted, you know, developers from, you know, all around the world. You know, other companies like red hat got into it as well. BlackRock also is, is a, is a big contributor, thousands of other independent contributors as well to the Argo project. >>Yeah. Christian, if you bring up a good point and I'm gonna go on a little tangent here, but I wanna get your reaction to something that Dave ante and I, and our cube team has been kind of riffing on lately. You mentioned, you know, Netflix earlier, you mentioned Intuit. There's a kind of a story that's been developing and, and with traction and momentum and trajectory over the past, say 10 years, the companies that went on the cloud, like Netflix into it, snowflake, snowflake, not so much now, but in terms of open source, they're all contributing lift. They're all contributing back to open source, but they're not cloud providers. Right. So you're seeing that kind of first generation, I's a massive contribution to open source. So open source been around for a while, remember the early days, and we'd all participate on projects, but now you have real companies building IP going open source first because they're on a hyperscale cloud, but they're not the cloud themselves. They took advantage of that. So there's kind of this cycle of flywheel of cloud to open source, not from the vendors themselves like Amazon, which services or Azure, but the people who rode their CapEx and built on that scale, feeding into the open source. And then coming back, this is kind of an interesting dynamic. What's your reaction to that? Do you see that? Yeah. Super cloud kind of vibe there. >>Yeah. Yeah. Well, and, and also it, it, I think it's, it's a, it's indicative that, you know, open source is not only, you know, a way to develop, you know, applications, a way to engineer, you know, your project, but also kind of like a strategic advantage in, in, in such a way. Right. You know, you, you see, you see companies like, like, like even like Microsoft has been going into, you know, open source, right. They they've been going to open source first. They made a, a huge pivot to, you know, using open source as, you know, like, like a, like a strategic direction for, for the company. And I think that goes back to, you know, a little bit for my roots, you know, I, I, I always, I always talk about, you know, I always talk about red hat, right. I always talk about, you know, I was, I was, I was in red hat previously and, you know, you know, red hat being, you know, the first billion dollar open source company. >>Right. I, we always joke is like, well, you know, internally, like we know you were a billion dollar company that sold free software. How, you know, how, how does that happen? But it's, it's, it's really, you know, built into the, built into being able to tap into those expert resources. Yeah. You know, people love using software. People love the software they love using, and they wanna improve it. Companies are now just getting out of their way. Yeah. You know, companies now, essentially, it's just like, let's just get out of the way. Let's let people work on, you know, what they wanna work on. They love the software. They wanna improve it. Let's let them, >>It's interesting. A lot of people love the clouds have all this power. If you think about what we are just riffing on and what you just said, the economics and the organic self-governing has always been the open source way where commercial value is enabled. If you play ball, right. Like, oh, red hat, for instance. And now you're seeing the community kind of be that arbiter of the cloud. So, Hey, if everyone can create value on say AWS or Azure, bring it to open source, everyone benefits across all clouds hope eventually. So the choice aspect comes in. So this community angle is huge. And I think it's changing a lot for the better. And I think this is where we're seeing a lot of that growth. And you guys have been the middle level with the Argo project and get ups specifically in that, in that sector. How have you seen that growth? What some dynamics have you seen power dynamics, organic? Is it governed well, whats some of the, the successes, what are some of the challenges? Can you share your thoughts on the community's growth around get ops and Argo project? >>Yeah, yeah. Yeah. So I've been, you know, part of some of these communities, right? Like the, the open, get, get ops community, the Argos community pretty much from the beginning and, and seeing it developed from an idea to, you know, having all these contributors, having, you know, the, the, the buzzword come out of it, you know, the get ups and it be that being the, you know, having it, you know, all over the, you know, social media, all over LinkedIn, all over all, all these, all these different channels, you know, I I've seen things like get ops con, right. So, you know, being part of the, get ops open, get ops community, you know, one of the things we did was we did get ops con it started as a meetup, you know, couple years ago. And now, you know, it was a, you know, we had an actual event at Cuan in Los Angeles. >>You know, we had like, you know, about 50 people there, but then, you know, Cuan in Valencia this past Cuan we had over 200 people, it was a second largest co-located events in, at Cuan. So that just, just seeing that community and, you know, from a personal standpoint, you know, be being part of that, that the, the community being the, the event chair, right. Yeah. Being, being one of the co-chairs was a, was a moment of pride for me being able to stand up there and just seeing a sea of people was like, wow, we just started with a handful of people at a meetup. And now, you know, we're actually having conferences and, and, and speaking of conference, like the Argo community as well, we put in, you know, we put on a virtual only event on Argo con last year. We're gonna do it in person today. You know, this year. >>Do you have a date on that? Do you have a date on that Argo con 22? >>Two? Yeah, yeah, yeah. Argo con September 19th, 2022. So, you know, mark your calendars, it it's, you know, it's a multi-day event, you know, it's, it's part of something else that I've seen in the community where, you know, first we're talk talking about these meetups. Now we're doing multi-day events. We're, you know, in talks of the open, get ups, you know, get ups can also make that a multi-day event. There's just so many talks in so many people that want to be involved in network that, you know, we're saying, well, we're gonna need more days because there's just so many people coming to these events, you know, in, in, you know, seeing these communities grow, not just from like the engineering standpoint, but also from the end user standpoint, but also from the people that are actually doing these things. And, you know, seeing some of these use cases, seeing some of the success, seeing some of the failures, right? Like people love listening to those talks about postmortems, I think are part of my favorite talks as well. So seeing that community grow is, is, you know, on a personal level, it's, it's a point >>It's like CSI for software developers. You want to curious about >>Exactly >>What happened. You know, you know, it's interesting, you mentioned about the, the multiple events at Coon. You know, the vibe that's going on is a very festival vibe, right? You have organic groups coming together. I remember when they had just started doing the day zero programs. Now you have like, almost like multiple stages of content at these events. It feels like, like a Coachella vibe or some sort of like festival vibe, like a lot of things going on and you, and if you pick your kind of area, but you can move around, I find that the kind of the format de Azure I think is going well these days. What do you think about that? >>Yeah, yeah. No, for sure. It's and, and, and I love that that analogy of Coachella, it does feel like, you know, it's, there's something for everyone and you can find what you like, and you'll find a little, you know, a little group, right. A little click of, of, of people that's probably the wrong term to use, but you know, you, you find, you know, you, you know, like-minded people and, you know, passionate about the same thing, right? Like the security guys, they, you know, you see them all clump together, right? Like you see like the, the developer C I CD get ops guys, we all kind of clump together and start talking, you know, about everything that we're doing. And it's, that's, that's, I think that's really something special that coupon, you know, some, you know, it's gotten so big that it's almost impossible to fit everything in a, in a week, because unless there's just so much to do. And there's so much that that interests, you know, someone, but it's >>A code, a code party is what we call it. It's a code party. Yeah. >>It's, it's a code party for sure. For >>Sure. Nerd nerd Fest on, on steroids. Hey, I gotta get, I wanna wrap this up and give you the final word, Christian. Thanks for coming on. Great insight, great conversation. There's a huge, you guys are in the middle of a hot area, obviously large scale data growth. Kubernetes is scaling beautifully and making it easier at managed services. What people want machine learning's kicking in and, and you get automation building in all favoring, the developer and C I CD pipeline and all that good stuff. People want to learn more. Can you take a minute to put the plug in for code fresh on the certification? How do I get involved? Where are you? Is there levels if I want to jump in and get trained and get fluent on code fresh, can you share commentary and, and, and what the status is? >>Yeah, yeah, for sure. So code fresh is offering a free certification, right? For get ups or Argo CD and get ops. The first of it's kind for Argo CD, first of it's kind for get ops is you can actually go get certified with Argo CD and get ops. You know, we there level one is out right now. You can go take that code, fresh.io/certification. It's out there, sign up, you know, you, you don't, you don't need to pay anything, right. It's, it's something it's a, of a free course. You could take level two is coming soon. Right? So level two is coming soon in the next few months, I believe I don't wanna quote a specific day, but soon because I, but soon I, it it's soon, soon as in, as in months. Right? So, you know, we're, we're counting that down where you can not only level one cert level certification, but a level, two more advanced certification for those who have been using Argo for a while, they can still, you know, take that and be, you know, be able to get, you know, another level of certification for that. So also, you know, Argo con will be there. We're, we're part of the programming committee for Argo con, right? This is a community driven event, but, you know, code fresh is a proud diamond sponsor. So we'll be there. >>Where's it located up to us except for eptember 19th multiday or one day >>It's a, it's a multi-day event. So Argo con from 19, 19 20 and 21 in a mountain view. So it'll be in mountain view in the bay area. So for those of you who are local, you can just drive in. Great. >>I'm write that down. I'll plug it. I'll put in the show notes. >>Awesome. Awesome. Yeah. And you will be there so you can talk to me, you can talk to anyone else at code, fresh talking about Argo CD, you know, find, find out more about hosted, get ups code, fresh.io. You know, you can find us in the Argo project, open, get ups community, you know, we're, we're, we're deep in the community for both Argo and get ups. So, you know, you can find us there as well. >>Well, let's do a follow up in when you're in town, so's only a couple months away and getting through the summer, it's already, I can't believe events are back. So it's really great to see face to face in the community. And there was responding. I mean, co con in October, I think that was kind of on the, that was a tough call and then get to see your own in Spain. I couldn't make it. Unfortunately, I had got COVID came down with it, but our team was there. Open sources, booming continues to go. The next level, new power dynamics are developing in a great way. Christian. Thanks for coming on, sharing your insights as the developer experience lead at code fresh. Thanks so much. >>Thank you, John. I appreciate it. >>Okay. This is a cube conversation. I'm John feer, host of the cube. Thanks for watching.

(upbeat music) >> Hey everyone. Welcome back to Las Vegas. The Cube is live. I can't say that enough. We are alive at AWS re:Invent 2021. Lisa Martin with Dave Nicholson. Hey Dave. >> Hey Lisa. >> Having a good day so far. >> So far, so good. >> We have an alumni back with us. We have about a hundred segments on the cube at AWS remit. We've got one of our original alumni back with us. Zaki Bajwa joins us the global head of partner solution engineers at Stripe. Zaki welcome back. >> Thank you, Lisa, thank you, Dave. Pleasure to be here. >> Lisa: Isn't it great to be back in person? >> Love it. Love it. Can't do a whiteboard virtually, you can, it's not the same. >> It's not the same and all those conversations I'm sure that you've had with partners and with customers the last couple of days that you just can't replicate that over zoom. >> Zaki: Exactly. >> So just for anyone who doesn't understand, AWS has a massive ecosystem of partners. So we'll get to talk about Stripe and AWS, but for anyone that doesn't know what Stripe is, give us the lowdown. You guys started 10 years ago. Talk to us about Stripe, the business strategy, what it's like today. >> Yeah, sure. So you guys know Stripe started 10 years ago by two brothers, John and Patrick Collison. And they've really focused on the developer and helping the developers accelerate digital commerce. Why? Cause the status quo at the time was one where a developer needed to, you know, build banking relationships with issuing banks, merchant banks, card networks, payment networks, tax liabilities, data compliance, and all of these manual processes that they had to deal with. So what Stripe aspires to do is build a complete commerce platform. Leveraging our integrated suite of products that is really allowing us to build what we call the global payments and treasury network. So if you think about the global payment and treasury network or what we call the G P T N it's meant to not only help abstract all of that complexity from a global payment infrastructure point of view, but also help move money in a simple and borderless and a programmable way just like we do in the internet. So that's the core essence of Stripe is to build this global payment treasury network to allow for money movement to happen in a simple and borderless manner. >> Simple and borderless two key things there. How has the business strategy evolved in the last 10 years and specifically in the last 20, 22 months? >> Yeah. Great question. So as you can imagine with COVID, you know, David you can order a cup of coffee or a brand new car, and that whole direct to consumer model has accelerated in COVID right. We've accelerated ourselves going to upwards of 6,000 employees. We've been able to answer or manage upwards of 170 billion API requests in the last 12 months alone. Right we deliver upwards of five nines from a availability performance point of view. That means 13 seconds of downtime or less a month. And we're doing this originally starting off for the developer David as you talked about allowing developers to deliver, you know, what I call process payments, accept payments and reconcile payments. But the evolution that you're talking about Lisa has really led to three key areas of focus that our users are requesting from us. And Stripe's first operating principle is really that user first mentality similar to the Amazons where we listen to our users and they're really asking for three key areas of focus. Number one is all around modernizing their digital commerce. So this is big enterprises coming to us and saying, whether I'm a uni lever or a Ford, how do you help me with a direct to consumer a e-commerce type platform? Number one. Secondly, is companies like Deliveroo and Lyft creating what we call marketplaces. Also think about Twitter and clubhouse, more solopreneurs entrepreneurs kind of marketplaces. Third is all around SaaS business models. So think about slack and Atlassian. That are customer vivers and accelerating the journey with us around digitizing digital commerce. So that's the first area of evolution. The second area is all around what we call embedded FinTech. So we know just like Amazon helped accelerate infrastructure as a service, platform as a service and function as a service. We're helping accelerate FinTech as a service. So we believe every company in every industry aspires to add more and more FinTech capabilities in their core services that they offer to their customers. So think about a Shopify or a Lyft they're adding more FinTech capabilities, leveraging Stripe APIs that they offer to their consumers. Likewise, when you think about a Monzo bank or a and 26, what we call Neo banks. They're creating more banking as a service component so a second area of evolution is all around FinTech as a service or embedded FinTech. And the third area of focus again, listen to our users is all around users are saying. Hey, Stripe, you have our financial data. How do you help us more with business operations and automating and optimizing our business operations? So this is revenue management, revenue reconciliation, financial reporting, all of the business processes, you and I know, code to cash, order to cash, pay to procure. Help us automate, optimize, and not just optimize, but help us create net new business models. So these are the three key areas of evolution that we've seen modernizing digital commerce, embedded FinTech, and then certainly last but not least business operations and automating that. >> And your target audience is the developers. Or are you having conversations now that are more, I mean, this is like transformative to industries and disruptive. Are you having conversations higher up in the chain? >> Great, great question. And this is the parallel with Amazon, just like Amazon started with developers, AWS. And then what up to the C-suite, if you will, we're seeing the same exact thing. Obviously our DNA is developer first making it intuitive, natural easy for developers to build on Stripe. But we're seeing more and more C-suite leaders come to us and saying, help us evolve our business model, help us modernize and digitize net new business models to get new revenue streams. So those parallel work streams are both developer mindset and C-suite led is certainly a big evolution for us. And we're looking to learn from our Amazon friends as to the success that they've had there. >> Do you have any examples of projects that developers have proposed that were at first glance, completely outlandish? Something that, you know, is there any sort of corner of the chart use case where Stripe didn't think of it, some developer came up with the idea, maybe it can't be done yet. If you have an example of that, that would be very interesting. >> Yeah, I'll give you two examples. So as I said, we're definitely a user first entity. That's our operating principle. We always think about the user. So let me go to developers and say, what are you struggling with? What are you thinking about? What are the next set of things you need from us? And a simple comment around tax started to come up and do you know in the U S there's 11,000 tax jurisdictions that you and you're selling something online have to abide to these different jurisdictions. So one of the things that we then evolved into is created a Stripe tax product, which initially users or developers were really struggling with and working on. So we created a Stripe tax product. We've done an acquisition called tax jar that helps us accelerate that journey for tax. The other one is this notion of low code that we see in the marketplace right now, where developers saying. Hey, give me more embeddables on top of the primitives that you've created on top of the APIs. So we went leveraging what our customers have already done, created things like a checkout capability, which is a simple redirect highly customized for conversion, which you can just integrate to one API. You have a full checkout capability. You can embed that into your platform, which didn't exist before and needed you to really integrate into different APIs. So all of these capabilities are what developers have really focused on and built that we've done leverage and Excel on. >> Yeah, I think between Lisa and myself, we've paid taxes in about 7,000 of those >> Lisa: Yeah, probably. >> Not 11,000 jurisdictions, but all the various sales taxes and everything else. So we're sort of familiar with it. >> I think so, so here we are, you know, on the floor at re-invent. Great, as we said to be back in person, the 10th annual, but with, as each year goes by AWS has a ecosystem of partners gets bigger and bigger. The flywheel gets, I don't know, I think faster and faster, the number of announcements that came out yesterday and today talk to us about some of the common traits that Stripe and AWS share. >> Yeah. So I've mentioned a few of them. One is certainly the user first mentality where we're listening to users. That tax example is a perfect one of how do we decide new features, new capability based on user first, Amazon does that better than anyone else. Second is that developer mindset focus on the developer. Those will be the core persona we target give you an example, Lyft, we all know Lyft. They wanted to create instant payouts for their drivers. So their developers came to us and say, our developers don't want to get paid. I'm sorry. Our drivers don't want to get paid in a week or two weeks. So we work with their developers who create a instant payout mechanism. Now in six months, over 40% of their drivers are using Stripe instant payout powered by Stripe. And that's a developer first mindset again, back to AWS. And then the third is really around the go to market. And the market opportunity is very similar. You talked about the developer persona and the C-suite very similar to Amazon. But also we're not just catering to enterprise and strategic big customers. We are just so much focused on startups, SMB, mid-market, digital native, just like Amazon is. And I would say the last parallel, which is probably the most important one is innovation. I come from enterprise software where we looked at monthly, quarterly, biannual, annual release cycles. Well, as Stripe, all of that goes out the door just like Amazon. We may have a hundred to a thousand APIs in motion at any time in alpha beta production. And just like Amazon we're iterating and releasing new innovations consistently. So I would say that's probably the most important one that we have with Amazon. >> So a lot of synergies there like deep integrated trusted partner synergies it sounds like. >> Agreed, definitely and then we're seeing this. I was going more as we are going more up market. We're seeing a demand for end to end solutions that require integrations with a CRM vendor for customer 360 with our accounting vendor for pivotal procure order to cash, billing accounting with a e-commerce company like Adobe Magento to do better econ. So more end to end solutions with these tech partners, we're working with our GSI to help deliver those end to end solutions. And certainly, but not least the dev agencies who are still sort of our core constituents that help us keep relevant with those developers. >> You mentioned this at the outset, but some things bear repeating. Can you go into a little more detail on the difference between me wanting to start up a business and take credit cards as payment 10 years ago? Let's say versus today, how much of the friction have you removed from that system? >> It is literally an hour to two hour process versus weeks and months before. >> But what are those steps? Like who would I, you mentioned this, again you mentioned this already, but the go through that, go through that again who would I have to reach out to, to make this happen? And we were talking, you know, relationships with banks, et cetera, et cetera. >> Yeah. So it starts at initiating and registering that company. So imagine you going and having to register a company today, you can do that with a Stripe Atlas product in a matter of hours, get your EIN number, get your tax jurisdictions on your registration as a Delaware entity within the U S you can be anywhere at globally and go do that within a matter of one hour. That's number one, you start there. From there, then it's a matter of embedding payment embeddables within your e-commerce platform, marketplace platform, et cetera. As you've heard us talk about seven lines of code to get payments going, you can quickly onboard accept payments, process payments, reconcile payments all within an hour. And that's just the start. But now you get into more complex use cases around marketplaces and multi-party connection. Multi-party payouts, different commission rates, different subscription models. Think about a flat tier model, a metered tier model, all of these different things that we've abstracted and allow you to just use one to three different integrations to help accelerate and use that in your digital commerce platform. So all of these different workflows have is what we've automated through our APIs. >> Dave: That's unbelievable. >> Yeah. >> It really is. >> It is unbelievable, the amount of automation and innovation that's gone on in such a short time period. What are some of the things as we kind of wrap up here that we can look forward to from stripe from a roadmap perspective, technology wise, partner wise? >> Yes. I mean, we have a slew of data you can imagine billions of billions of transactional data. And you guys know what we do with data is we're looking at fraud prevention. We're looking at, we have a product called radar that looks at fraud, we're doing acceptance, adaptive acceptance to do more AIML learned data and authorization. We're also looking at how do we feed a lot of this financial data into the right mechanisms to allow you to then create new business models on top of this, whether it's cross sell upsell to new user business capture. As well as you know, one of the things I did not talk about, which coming from a farming background is this notion of Stripe climate. Where we have upwards of 2000 companies across 37 countries that are leveraging our Stripe climate product to give back to tech advanced companies that are helping in carbon offset. And super exciting times there from an ESG environmental social governance point of view. So all of those combined is what excites us about the future at Stripe. >> Wow. The future seems unlimited. Lots going on. >> Super excited. Zaki, thank you so much for joining Dave and me talking about what's going on with Stripe. All the innovation that's going on. The synergies with AWS and what's coming down the pipe. We appreciate your insights and your time. >> Thank you, Lisa, thank you, David. Appreciated All right. For Dave Nicholson, I'm Lisa Martin. You're watching the Cube. The global leader in live tech coverage. (lighthearted piano music)

(upbeat music) >> Hey everyone and welcome back to the CUBE's continuous live coverage of AWS re:Invent 2021. continuous live coverage of AWS re:Invent 2021. Lisa Martin here with David Nicholson. We have two live sets going on, we've got two remote sets, over 100 guests working with AWS and it's a massive ecosystem of partners, really digging into the next decade of cloud innovation and we're pleased to welcome back one of our CUBE alumni, Edith Harbaugh, CEO, and co-founder of LaunchDarkly. We're going to be talking about a blueprint for continuous modernization, Edith, it's great to have you, thanks for coming. >> Thanks for having me. >> So I was doing some research, you guys raised 200 million in series D in August, just a few months ago, that new funding tripled your valuation to 3 billion up more than 3x from the previous funding rounds, so rocket ship. >> Edith: Yeah. >> I also noticed you guys are on the Forbes Cloud 100 2nd year on the list, you jumped dramatically from 100 in 2020 to 47 this year, talk to us about all the innovation and acceleration that's going on at LaunchDarkly. >> Yeah, well, it's, it's great to be here, you know, I'm the CEO and co-founder and we started seven years ago in 2014 and what we were doing back then was a really new field, like I actually came up with the name feature management, just to describe what we were doing and it was this idea that you could release features to different people at different times, which sounds really simple, but it really allows you to have valves to different populations that you can then turn something on, turn something off, run a beta, do personalization and then if something's going wrong out on the field quickly and easily, turn it off. >> So as an engineer, as a long standing engineer, what were the things that really frustrated you, that you thought this, this is missing, we've got to focus on this. >> Oh my gosh, so, I was an engineering manager, I actually do a podcast too called To Be Continuous just about all the bad things I saw happen, the worst thing you could do is, is build something that nobody wants, which is really frustrating, so I think a lot of continuous delivery came out of the urge to just get stuff out quicker. The flip side of that is that if you moved too fast, a release can be catastrophic. We used to call them the push and pray release because you push stuff out and then you're just crossing your fingers that nothing breaks because if something breaks it's extremely stressful. Your mind starts flooding with endorphins and hormones, your heart rate increases and you sometimes make even worse decisions, so what LaunchDarkly and feature management allow you to do is push it out to who you want and if something is going wrong, you could turn it off without a redeploy, if things are going right, you can continue to push it out. >> And when you say feature management, you're talking about, you're talking about a level of granularity that is finer than a release version. that is finer than a release version. >> Edith: Yeah. How do you do that? >> Our customers do it, so we provide a platform where our customers and we have 2,500 worldwide, everything from IBM and Atlassian, down to like three person startups, they decide how to encapsulate a feature >> David: Okay. >> So they could push it to who they want, so, so there's a lot of really neat use cases. >> So knowing that you're providing them with the valves. >> Edith: Yes. Then they can- think differently about how they're actually developing in anticipation of delivering encapsulated features, as opposed to, here's your new release. >> Edith: Exactly. >> Okay. >> Exactly, so we have some customers who've used LaunchDarkly to actually move to the cloud. So like TrueCar was running their own data centers and they wanted a way to start moving all of that data center traffic into AWS, so they could use LaunchDarkly to manage that traffic flow and do it in a controlled way instead of just one quick switch. >> I was looking at that case study of TrueCar, they migrated 500 websites to AWS without downtime and deploying 20x per day, which is up from 1x a week, that's a massive change. >> Yeah, I think really what we give our customers is confidence that if you know that you can always have control over stuff with feature management, you actually move much quicker. You can, you can move 20 times a day if you know that if something goes wrong, you can always turn it off, you have much more confidence. >> Where are your, you having customer conversations? I know you, you coined to the term feature management, I'd love to know a bit more contextually about the evolution from feature flags to feature management and where are those customer conversations happening? are they kind of down in the technical ways? are they more higher level? given the fact that we're in such a, still a, such a state of flux with COVID? >> Yeah, so we, we didn't invent feature flagging like the smart companies like Amazon, Facebook, Netflix have been doing feature flagging for decades now, it was always a secret sauce of this is how they could manage their own functionality. What LaunchDarkly did was kind of changed it to feature management about doing it where any other customer also had the same set of tools and platforms and also on top of that things like a workflow, scheduling, integrations. So that for example, a developer could develop something and then give the keys to the product manager, say product manager, you get to, you get to run the beta now. >> So putting, putting more control back in the hands of the folks that are, that really are touching and feeling and smelling the product. >> Yeah or customer support, you know, >> Yeah or customer support, you know, if something is going wrong in the field, instead of having to wait for an engineer to fix a bug customer support could just turn it off. >> So I'm curious about, you know, when we talk about it's a, this, this sort of dovetails with something that was discussed in the keynote today, out of the gate, Adam comes out and he's talking about microprocessor technology. Now in the era of cloud, generally people would say, that stuff doesn't matter, right? It's all about the feeling of being in the cloud and the flame, you know, the, the, the field of wheat blowing in the wind and it's a feeling that you get, it's really interesting what you're doing under the covers, but who is the, who is the audience? Who, who buys this? Because I can imagine some in the engineering, on the engineering side of things, feeling like maybe they're giving up some control, but really you're giving them more tools, but is it business people who are demanding this? how, how do you go to market? >> Yeah, so it's really interesting because our core audience is developers and VP of engineering, like they love the platform. Like our Net Promoter Score is extremely high, engineers say like, this gave me my weekends back because if a bug happens I don't have to come in. >> David: Okay so they get it. >> They get it. This isn't being pushed down- from executives that don't understand the technology. >> No, I mean, a typical thing is a developer's, like, I need this to do my job and then the business people say, well, if the developers are happy, we're happy, you know, it's, it's a developers world now, you know, they're hard to hire, you have to have them and if you have anything that will make their job easier and them happier, why wouldn't you buy it? >> That's a big facilitator, so you mentioned a high, high NPS, high Net Promoter Score, we, we talk with Amazon folks about their their focus on the customer and their customer obsession if you will, that everything starts backwards, we start from the customer, 2,500 customers in such a short time period, we talked about the funding, I imagine culturally there's similarities there, if one of the things that you're able to confidently give your customers is that confidence in LaunchDarkly. >> Yeah, you know, one of the happiest parts of my job is visiting customers, you know, I, my co-founder and I personally visited, I think the first 10 or 20 customers and if they had a bug, if they wanted something, and if they had a bug, if they wanted something, we built it. And I love going on customer sites, cause it's. And I love going on customer sites, cause it's. >> When they're telling you that you gave them their weekend back. >> Edith: Yeah. >> Huge. >> That's, yeah, that.- that's not an insignificant thing when you think about what people do with their weekends, you know, so? >> Yeah, you know, it, it feels really good to have customers say, like this literally has changed the way they built software for the better. >> I can't imagine this, you know, with everything that's happened in the last 22 months with the acceleration to cloud, but all these massive pivots by businesses, in every industry just to survive in the beginning, were an advantage, something like LaunchDarkly is for those organizations, so you have to move really, really quickly and keep changing direction to kind of figure out how do we stay afloat and now how do we thrive in that, that this has probably been a real lifesaver for a lot of organizations. >> Yeah, I mean, we've seen like a ten year roadmap at our customers compressed into a month, like we had a, a retail chain in the Midwest that was thinking about doing in-store pickup and then when COVID hit, they're like, okay, this changed from a, maybe to a, we need to have this to stay afloat and now, now they can help people pick up, same with, same with restaurants having a mobile app to do delivery or pickup, it used to be when we'll get to that next year, now it's something that you have to have. >> Oh yeah. >> Because if, if you're going to go get coffee and one place has a mile long line and their place has an app, which one are you going to pick? >> So, what do people do that don't have this capability? This, I mean, this might sound like a completely naive question, I know a lot about a lot of things, so I'm okay looking dumb sometimes, it's how I learn, so I'm okay looking dumb sometimes, it's how I learn, but seriously, if you don't have these valves, then aren't you doomed then aren't you doomed to releases that are going to be panic inducing. >> It's really, it's really painful, like, I mean, that's, that's the way I used to, to release, you know, I remember it, like you're released and you would have tried to have caught all the bugs, but it would go out and if something happened, you had to fix it on the fly and even if you have a really good deployment process, that's 20 minutes, maybe two hours. >> David: Sure, and, and. >> Which, which if you're a mobile app, it could be a business killer. >> Yeah, well we're here at AWS reinvent, I mean, how does, how does this dovetail with this, the AWS mission to migrate and modernize into the cloud native world? we're talking about cloud native, you know, development and operations that you're involved with, so there's obviously a synergy there, but why specifically AWS? >> Oh, I mean, I think one of the biggest tailwinds we've had as a business is if you're releasing twice a year, we've had as a business is if you're releasing twice a year, you don't really need a tool like this, or a platform like this, your business process is completely different, but you're going to die as a company cause you can't survive on two releases a year. If you're moving to the cloud, we help you get there and once you're in the cloud, if you want to move at the speed of business, but safely, we give you that platform. Like, so, I think continuous delivery got this bad wrap because people thought that meant that you push out stuff every second and break everything. >> David: Right. >> What we do is we allow you to innovate as fast as you want, but release in a controlled way. >> I got to ask you a question, you, you talked about the customers and your love of being with customers, one of the things I can't help thinking is that what you're helping facilitate is brand reputation. If, you know, if we have an expectation and we want to go on a, on an app and order coffee, and it's down, we're going to go to the next competitor, so from a brand reputation perspective, I'm just wondering if, if any of your customer conversations kind of go in addition to the VP of engineering kind of go in addition to the VP of engineering and it focused on the folks that are leading these companies going, our reputation is on the line, people are, let's face it during COVID far less patience than we've, we've seen a lot of really impatient people, but is, is that something that you also facilitate, is the brand reputation? >> Oh, not just a brand reputation that an outage can be costly of millions of dollars, like. that an outage can be costly of millions of dollars, like. >> Lisa: $5,600 a minute, I think is what Gartner estimates. >> Yeah, but depending on what business you're in, like if you're in a bank, you absolutely need to be reliable. If you're a streaming service like streaming, one of the biggest horse races in Australia, you need to have uptime. >> Everybody needs uptime, let's, let's just be clear if I can't get door dash or whatever, it's a disaster from my perspective as a consumer and yes, we have, we have far less patience than we've ever had. >> Yeah, I mean, we have a really interesting, we have both B2C, like streaming ash, streaming apps, delivery apps, as well as B2B streaming apps, delivery apps, as well as B2B and they both have problems that we solve but honestly, the, the, the business problems with a B2B are much more challenging sometimes. >> Well Edith, thank you so much for joining David and me on the program, talking about LaunchDarkly, what you're enabling organizations to achieve in every industry, it sounds like you're riding a rocket ship. >> It's been really fun, you know, I, I love seeing a customer that's been using us for three, five years. >> David: Wow. >> And how much their life has gotten better. >> And as you said, that's, that's no small statement. Thank you so much for joining us on the program, we appreciate your insights and look forward to hearing more news from LaunchDarkly coming out. >> Thanks. >> All right, for David Nicholson, I'm Lisa Martin, you're watching theCUBE's coverage of AWS:reinvent 2021, theCUBE, the global leader in live tech coverage. (upbeat music)

>>Welcome everyone to the cubes, continuing coverage of AWS reinvent 2021. I'm Lisa Martin. We are running one of the industry's most important and largest hybrid tech events this year to live sets to remote studios with AWS and its ecosystem partners. We've got over a hundred guests on the program this year, going deep as we enter the next decade of cloud innovation. We are pleased to welcome for the first time to the cube. John the CTO and co-founder of LaunchDarkly. John is here to talk about modern DevOps with feature management, John, welcome to the program. >>Thanks for having me, Lisa. >>Great to have you on the program. Let's talk a little bit about LaunchDarkly. I know it's been on the cube a couple of times, but it's been a while. Give the audience an overview of LaunchDarkly what it is that you do and what's new. >>Yeah. LaunchDarkly is the leading platform for feature management. We allow developers, product managers, anyone in the practice of building software to leverage feature flags, to deliver better software, faster, a better product experiences through the use of feature flags. >>And one thing that I noticed, um, on the website is you guys have some big customer names square. I noticed I also saw Adidas, NBC, at least you've got some, some pretty big organizations that are relying on LaunchDarkly to deliver and control their software. What can you tell us about it from a customer perspective? >>Yeah. You know, it's an amazing thing. We have over 30% of the fortune 100 using the LaunchDarkly platform for feature management. And, uh, you know, I think it is, it's been incredible to see how basically anyone building software can leverage feature flags to, to, to deliver better customer experience. And so the companies you named, I mean, they're all over the map in terms of the kinds of products they deliver to consumers from, uh, from square to Adidas. I mean, those are totally different companies, but, uh, I think the thing that they all have in common is that they're increasingly becoming they're, you know, they're, they're either already a software company or they're increasingly becoming a software company and that's where we help, uh, our customers, the customers that are, uh, you know, delivering more digital experiences to their consumers >>That is table stakes. These days, you mentioned all software, all companies rather becoming software companies. If they're not probably not going to be around much longer and you, you right. You mentioned, you know, w that's a, quite a variety NBC to Adidas as I talked about there, but in terms of what they have in common, talk to me a little bit about feature management. What is it and how can it help to bridge the divide between the developer folks, the business side of the organization? >>Absolutely. Uh, I think the fundamental thing that, that feature management provides, the simplest thing, that the thing that people first utilize LaunchDarkly for is to separate the, the processes of deploying software from releasing software. So it used to be in, in a pre-launch darkly world, when you deploy a soft, a new piece of software, you package the artifact that you put it out on your service, and then your entire customer base was experiencing that new version of the software. So if things were going wrong, if there was a bug, something wasn't working, right, your blast radius was enormous. Literally your entire customer base was impacted. Um, and one of the things that LaunchDarkly does, the first thing that we do, the first piece of value that we provide is we help you sort of reduce that risk. So when you release a change, you can deliver that change to a much more targeted, smaller, safer cohort of users measure the impact of what's going on. Is it, is it, are there any bugs, uh, are there any performance problems, whereas everything's smooth sailing, and if it is, then you can use LaunchDarkly to rapidly and with a lot of visibility control scale that release and scale that roll out out. And that's the most fundamental value that we provide. >>Big value there. Speaking of value, let's talk about the partnership with LaunchDarkly and AWS. I know you have a lot of experience working with AWS for many years back when you were at Atlassian, but give us an overview of the partnership and that shared developer audience that you're both working with. >>Yeah. I've got a number of years of experience working with, with, uh, AWS. So you mentioned my time private prior to starting LaunchDarkly. I was at Atlassian for many years, uh, and I was added last year. And during that time period where Atlassian was switching from, um, traditional hosting providers to, uh, to public cloud, to AWS specifically, um, and the capabilities that an unlocked, not only for our operations teams, but for our developers were pretty incredible. Um, one of the things that we, uh, we launched almost immediately on my team, uh, was the ability to sort of like preview environments, uh, through AWS hosting and have that experience not happen on the local developers, desktop, but rather in the cloud. Uh, and that was incredibly helpful for improving our velocity and helping us preview changes. Um, uh, since, since starting LaunchDarkly, I mean, we've leveraged cloud and AWS in particular from the earliest days, we started the platform, uh, on AWS and we've been consuming more and more services through AWS and, and seeing more and more value, um, from a partnership perspective, we're incredibly excited because we have a massive number of customers that are either just beginning their public cloud journey or are making significant migrations or significant infrastructure changes. >>And they're using the LaunchDarkly platform to control the release of those changes to mitigate risk. Um, we have customers using us to do migrations from one cloud provider to another, or go through modernization efforts and push change out, uh, safely, uh, as they migrate to a provider like AWS. >>Talk to me about some of the things that you've seen in the last year and a half 20 months, or more probably since the pandemic started, we've seen so much acceleration to cloud so much cloud migration. It's so many companies, not only becoming software companies, because they need to be competitive, but understanding not why move to the cloud. It's when did you, how have you, how have you helped organizations, you know, from the NBCs the media folks to, to the retailers, to, to undergo those migrations safely, but quickly in a time of such dynamics? >>Yeah, I mean, the, that, that is exactly what we saw during the pandemic, a massive amount of change, not just in the move to digital and digital experiences, but also in the need to sort of adapt to rapidly changing conditions. Uh, we had customers, uh, in for example, food delivery that were, that needed to rapidly change the way their software behaved, uh, in response to changes in regulations or guidelines around things like COVID. And our platform really was transformative for many of those organizations, as they sort of, um, needed to become more flexible and adapt, not only to changing rules and regulations, but changing consumer behavior and changing end user behavior. Um, so it was, it was an incredible year. It was, uh, a year that was sort of fraught with uncertainty, but it was a year where, you know, LaunchDarkly, our platform really helped many of our customers, um, sort of navigate the waters and, and figure out how to get, um, the experiences they needed to and the change they needed to in front of their customers rapidly. >>Yeah. Rapid being a keyword of the last 20 years, both 20, 20 years, it feels like 20 years, doesn't it? Two years guardians lived there. Um, but talk to me a little bit about some of the other trends that you're seeing from a cloud perspective. We talked about the acceleration of migration. What are some of the other trends that your customers are facing and how are, is LaunchDarkly helping them to address those trends? >>Yeah, well, one of the trends that we're seeing is, is the rapidity of change, um, is forcing companies that, um, even companies that were really software driven, uh, at their heart, uh, to iterate more rapidly. And I think there's this story around modernization that that is becoming more and more, um, common where you normally think of modernization as sort of like legacy companies, sort of non software driven companies having to make that shift and modernize their software stacks. But the rapid pace of change is, uh, it's shifting things into a world where even companies like, like my own company, like LaunchDarkly are having to modernize our stack. Our company is seven years old. Um, and some of the things that we were doing seven years ago, um, they've been eclipsed in terms of like processes, tools, technologies, and use. And so we've had to go through modernization as well to keep up with the times and to, to give our developers the quality of, of, of tools and processes that they expect. >>I think that's an important point, John, that you bring up is that modernization isn't just for legacy applications, legacy businesses, and that's, and I'll be honest. That's how I normally think about it. I don't think of a company as a young, as LaunchDarkly needed to modernize, but you bring up a point that really what it is, is an ongoing process for businesses in any industry. >>I mean, if you think about the, what the landscape looked like seven years ago and you fast forward to today, um, so many of the practices are different. Um, so even companies like us, we're having to change. I mean, uh, seven years ago, it wasn't really clear, um, that Kubernetes was going to be a platform that was, that was going to end up being the winner and sort of like the orchestration space. And so when we were starting out, um, none of our workloads were, were on Kubernetes. And even today, uh, we're not really significantly using Kubernetes. We're, we're sort of like legacy container-based. Um, and that's just us, we're, we're still a startup and we're still able to move pretty rapidly. Um, but even for us, we're, we're having to sort of revisit the technologies and use and modernize our stack, um, and kind of look around and see what's not working anymore and what we need to change. Um, it's certainly a pace that is massively different from, uh, uh, you know, a company that is, um, relying on a legacy software stack. I don't want to pretend like LaunchDarkly is, uh, I would compare us to a company that's, um, moving off of mainframes and cobalt or anything like that. Um, but it's still, uh, it's still something that we're cognizant and cognizant of and something that we have to invest in, >>But you bring up a good point is that it's it's. And as we talk about this, when we're talking with any vendor about, from the customer's perspective, it's a journey, it's the same thing that you're talking about here. It's, it's evaluating what you have under the hood what's working, what needs to be better as the markets change as the dynamics change as trends change. >>Yeah. That, that's, that's exactly how I think about it. And that's how a lot of these companies that are becoming more software driven are thinking about it too. Just sort of like assessing the catalog of tools and technologies and saying what's working, what's not working. Um, and I, I think one of the trends that we're seeing is that re-evaluation is happening more and more frequently and the frequency of new technologies and tools being adopted, uh, is increasing. And so it's something that you have to spend an enormous amount of effort, um, just to stay ahead of the game and stay ahead of what's what's modern and what's, what's the practices that we've determined are really working for organizations. >>Right? Exactly. So I mentioned a few customers by name that work with LaunchDarkly, but can you tell me an example of one of your favorite customer stories that you think really articulate the value that LaunchDarkly is delivering to your customers across industries? >>Yeah. Um, one that comes to mind is, is true car. Um, true car has been a LaunchDarkly customer for a long time. They're great partners of ours. Um, we have a case study up with them. And one of the stories that they talked about was their own cloud migration. Um, they shifted, uh, their, their workloads from one cloud provider to another and feature flags were sort of instrumental in that. So feature flags allowed them to sort of, um, gate the flow of traffic from one cloud to another. And just sort of like, uh, sort of in real time assess whether things were working or not, as they did that migration, it took a process that would have been incredibly risky, risky, and scary, and made it sort of business as usual for that. Um, so that's a company that I think of, um, that is really, that really understands the value of LaunchDarkly and has really leveraged us to our full potential. >>Awesome. Something I want to ask you about as well, is this concept of release impact compare and contrast that to like the traditional optimization focused AB testing? What's the difference? What are the similarities? >>Yeah. You know, AB testing has been around for a long time, uh, and it's used in software. Uh, definitely in the past decade has grown tremendously, uh, as a, as a piece of the software development experience. Um, but when I think about, um, the practice of building deep product experiences and contrast that to sort of like, um, uh, AB testing on a marketing site and, you know, testing out the layout of a page we're testing out, uh, you know, which call to action button color ends up creating more engagement. That's a very different world than, you know, I'm building a, I'm building a SAS product and I'm building this a new feature within that SAS product. Um, traditionally, um, you wouldn't really AB test that. Um, and part of the reason for that is it's really too expensive to build software. And it's not really a reality that most companies have where they can take a team and have them go build a feature for multiple weeks or months, um, pry it out in production and then say, you know what, that didn't work, um, that million dollar expense that we just made, we're just going to roll that back and, and, and not use it. >>So, um, that's sort of the way I think about the difference between a traditional optimization focused AB testing, where it's, it's sort of like smaller bets designed to move the needle on a metric, um, where if it doesn't work, you can turn it off versus these deep product experiences where when you're more interested in is being more quantitative about the impact of that release, but you're not necessarily interested in, um, sort of like AB testing, uh, focused, optimization, picking a winner in a short period of time. Um, one of the things that we've realized that LaunchDarkly is those are two separate tasks. Um, they're two separate processes and they require, um, different analyses and, and different tools under the hood. And so, uh, we're really excited at LaunchDarkly to be innovating on sort of both fronts. Um, not only just providing a platform for, uh, optimization focused AB testing, but providing a platform where product managers can be more quantitative about the, the capabilities that they're building and not thinking about it in terms of optimization, but just in terms of measuring the impact of the work that they're, they're shipping to customers, >>The impact. And of course, it's all outcomes focus as we talk about with customers and vendors and at any industry, last question, John, for you, as we're coming up on re-invent in-person, what are some of the things that, um, attendees can learn and see at the LaunchDarkly booth? >>Yeah. You're going to learn a lot about if you visit our booth, you're going to learn a lot about sort of like the, the direction that we're taking, which is, um, I think the exciting thing about LaunchDarkly as a platform is we really provide two capabilities for engineering teams. We help mitigate risks. We help you move more efficiently. That gives you more at bats as a team. Uh, it lets you, uh, ship more product and see whether it's working while shortly also they'll provide something on the flip side of that, which is the ability, the ability for product managers to measure whether the changes that they're making are the right changes for their customers. And when you combine those two things in one platform, you get the ability to, for the engineering team to have more at bats, to create more, uh, uh, uh, uh, change in production and see whether it's working. And then you get product managers, the ability to measure, uh, the impact on their customers. And you combine that together. And, uh, at the end of the day, what LaunchDarkly provides is the ability for you as an organization to deliver business value better, um, more quickly, um, through the R and D investments that you're making the software that you're producing. >>And that's critical. I love that baseball analogy more at bats. Fantastic. John, thank you for joining me, talking to the audience about launch directly, what you're doing, the trends that you're helping customers address the partnership with AWS and what folks can learn when they visit the LaunchDarkly booth at re-invent. We appreciate your time. >>Thank you so much, Lisa. I really enjoyed our conversation. >>Me too, for John Coda mall. I'm Lisa Martin, and you're watching the cube continuous coverage of AWS reinvent 2021.

(upbeat music) >> Welcome everyone to theCUBE, continuing coverage of AWS re:Invent 2021. I'm Lisa Martin. We are running one of the industry's most important and largest hybrid tech events this year, two live sets, two remote studios with AWS, and its ecosystem partners. We've got over a hundred guests on the program this year, going deep as we enter the next decade of cloud innovation. We are pleased to welcome for the first time to theCUBE, John Kodumal, the CTO and co-founder of LaunchDarkly. John is here to talk about modern DevOps with feature management. John, welcome to the program. >> Thanks for having me, Lisa. >> Great to have you on the program. Let's talk a little bit about LaunchDarkly. I know it's been on theCUBE a couple of times, but it's been awhile. Give the audience an overview of LaunchDarkly, what it is that you do and what's new. >> Yeah. LaunchDarkly is the leading platform for feature management. We allow developers, product managers, anyone in the practice of building software to leverage feature flags, to deliver better software faster, a better product experiences through the use of feature flags. >> One thing that I noticed on the website is you guys have some big customer names, Square, I also saw Adidas, NBC, at least you've got some pretty big organizations that are relying on LaunchDarkly to deliver and control their software. What can you tell us about it from a customer perspective? >> Yeah. You know, it's an amazing thing. We have over 30% of the Fortune 100 using the LaunchDarkly platform for feature management. And, you know, I think it's been incredible to see how basically anyone building software can leverage feature flags to deliver better customer experiences. So, the companies you named, I mean, they're all over the map in terms of the kinds of products they deliver to consumers from Square to Adidas. I mean, those are totally different companies, but I think the thing that they all have in common is that they're increasingly becoming... They're either already a software company or they're increasingly becoming a software company and that's where we help our customers, the customers that are delivering more digital experiences to their consumers. >> That is table stake these days, you mentioned all software, all companies rather becoming software companies. If they're not, they're probably not going to be around much longer and you're right. You mentioned that's a quite a variety, NBC to Adidas as I talked about there, but in terms of what they have in common, talk to me a little bit about feature management. What is it and how can it help to bridge the divide between the developer folks, the business side of the organization? >> Absolutely. I think the fundamental thing that feature management provides, the simplest thing, that the thing that people first utilize LaunchDarkly for is to separate the processes of deploying software from releasing software. So it used to be in a pre-LaunchDarkly world, when you deploy a new piece of software, you package the artifact up, you put it out on your servers, and then your entire customer base was experiencing that new version of the software. So, if things were going wrong, if there was a bug, something wasn't working right, your blast radius was enormous. Literally, your entire customer base was impacted. And one of the things that LaunchDarkly does, the first thing that we do, the first piece of value that we provide is we help you sort of reduce that risk. So when you release a change, you can deliver that change to a much more targeted, smaller, safer cohort of users, measure the impact of what's going on. Are there any bugs? Are there any performance problems? Or is everything's smooth sailing? And if it is, then you can use LaunchDarkly to rapidly, and with a lot of visibility control, scale that release and scale that roll-out out. And that's the most fundamental value that we provide. >> Big value there. Speaking of value, let's talk about the partnership with LaunchDarkly and AWS. I know you have a lot of experience working with AWS for many years back when you were at Atlassian, but give us an overview of the partnership and that shared developer audience that you're both working with. >> Yeah. I've got a number of years of experience working with AWS. So, you mentioned my time prior to starting LaunchDarkly, I was at Atlassian for many years, and I was at Atlassian and during that time period where Atlassian was switching from traditional hosting providers to public cloud, to AWS specifically, and the capabilities that an unlocked, not only for our operations teams, but for our developers were pretty incredible. One of the things that we launched almost immediately on my team was the ability to like preview environments through AWS hosting and have that experience not happen on the local developers desktop, but rather in the cloud. And that was incredibly helpful for improving our velocity and helping us preview changes. Since starting LaunchDarkly, I mean, we've leveraged cloud and AWS in particular from the earliest days, we started the platform on AWS and we've been consuming more and more services through AWS and seeing more and more value. From a partnership perspective, we're incredibly excited because we have a massive number of customers that are either just beginning their public cloud journey or are making significant migrations or significant infrastructure changes, and they're using the LaunchDarkly platform to control the release of those changes to mitigate risk. We have customers using us to do migrations from one cloud provider to another, or go through modernization efforts and push change out safely as they migrate to a provider like AWS. >> Talk to me about some of the things that you've seen in the last year and a half, 20 months or more probably. Since the pandemic started, we've seen so much acceleration to cloud, so much cloud migration, so many companies, not only becoming software companies because they need to be competitive but understanding it's not why move to the cloud, it's when. How have you helped organizations, you know, from the NBC, the media folks to the retailers, to undergo those migrations safely but quickly in a time of such dynamics? >> Yeah, I mean, that is exactly what we saw during the pandemic, a massive amount of change, not just in the move to digital and digital experiences, but also in the need to sort of adapt to rapidly changing conditions. We had customers in, for example, food delivery that needed to rapidly change the way their software behaved in response to changes in regulations or guidelines around things like COVID. And our platform really was transformative for many of those organizations as they sort of needed to become more flexible and adapt, not only to changing rules and regulations, but changing consumer behavior and changing end-user behavior. So, it was an incredible year. It was a year that was sort of fraught with uncertainty, but it was a year where LaunchDarkly, our platform really helped many of our customers sort of navigate the waters and figure out how to get the experiences they needed to and the change they needed to in front of their customers rapidly. >> Yeah. Rapid being a keyword of the last 20 years, it feels like 20 years, doesn't it? Two years, 40 and slipped there. But talk to me a little bit about some of the other trends that you're seeing from a cloud perspective. We talked about the acceleration of migration. What are some of the other trends that your customers are facing and how is LaunchDarkly helping them to address those trends? >> Yeah. One of the trends that we're seeing is the rapidity of change is forcing companies that even companies that were really software driven at their heart to iterate more rapidly. I think there's this story around modernization that is becoming more and more common where you normally think of modernization as sort of like legacy companies, sort of non software-driven companies, having to make that shift and modernize their software stacks, but the rapid pace of change is it's shifting things into a world where even companies like my own company, like LaunchDarkly are having to modernize our stack. Our company is seven years old. And some of the things that we were doing seven years ago, they've been eclipsed in terms of like processes, tools, technologies, and use. And so we've had to go through modernization as well to keep up with the times and to give our developers the quality of tools and processes that they expect. >> I think that's an important point, John, that you bring up is that modernization isn't just for legacy applications, legacy businesses, and I'll be honest, that's how I normally think about it. I don't think of a company as young as LaunchDarkly needing to modernize, but you bring up a point that really what it is is an ongoing process for businesses in any industry. >> Yeah, absolutely. I mean, if you think about what the landscape looked like seven years ago and you fast forward to today, so many of the practices are different. So even companies like us, we're having to change. I mean, seven years ago, it wasn't really clear that Kubernetes was going to be a platform that was going to end up being the winner and sort of like the orchestration space. And so when we were starting out, none of our workloads were on Kubernetes. And even today, we're not really significantly using Kubernetes, we're sort of like legacy container-based. And that's just us, we're still a startup and we're still able to move pretty rapidly. But even for us, we're having to sort of like revisit the technologies and use and modernize our stack and kind of look around and see what's not working anymore and what we need to change. It's certainly a pace that is massively different from a company that is relying on a legacy software stack, I don't want to pretend like LaunchDarkly is, I would compare us to a company that's moving off of mainframes and COBOL or anything like that, but it's still something that we're cognizant of and something that we have to invest in. >> But you bring up a good point. And as we talk about this when we're talking with any vendor about, from the customer's perspective, it's a journey, it's the same thing that you're talking about here. It's evaluating what you have under the hood, what's working, what needs to be better as the markets change, as the dynamics change, as trends change. >> Yeah. That's exactly how I think about it and that's how a lot of these companies that are becoming more software-driven are thinking about it too. Just sort of like assessing the catalog of tools and technologies and saying what's working, what's not working. And I think one of the trends that we're seeing is that re-evaluation is happening more and more frequently and the frequency of new technologies and tools being adopted is increasing. And so, it's something that you have to spend an enormous amount of effort just to stay ahead of the game and stay ahead of what's modern. The practices that we've determined are really working for organizations. >> Right, exactly. So, I mentioned a few customers by name that work with LaunchDarkly, but can you tell me an example of one of your favorite customer stories that you think really articulate the value that LaunchDarkly is delivering to your customers across industries? >> Yeah. What comes to mind is TrueCar. TrueCar has been a LaunchDarkly customer for a long time. They're great partners of ours. We have a case study up with them. And one of the stories that they talked about was their own cloud migration. They shifted their workloads from one cloud provider to another and feature flags were instrumental in that. So, feature flags allowed them to sort of gate the flow of traffic from one cloud to another and to sort of in real-time assess whether things were working or not as they did that migration. It took a process that would have been incredibly risky and scary, and made it sort of business as usual for that organization. So, that's a company that I think of that really understands the value of LaunchDarkly and has really leveraged us to our full potential. >> Awesome. Something I want to ask you about as well, is this concept of release impact. Compare and contrast that to like the traditional optimization focused A/B Testing. What's the difference? What are the similarities? >> Yeah. You know, A/B Testing has been around for a long time and it's used in software, definitely in the past decade has grown tremendously as a piece of the software development experience. But when I think about the practice of building deep product experiences and contrast that to sort of like A/B testing on a marketing site, you know, testing out the layout of a page, we're testing out which call to action button color ends up creating more engagement. That's a very different world than I'm building a SaaS product and I'm building this a new feature within that SaaS product. Traditionally, you wouldn't really A/B test that. And part of the reason for that is it's really too expensive to build software. And it's not really a reality that most companies have where they can take a team and have them go build a feature for multiple weeks or months, pry it out in production and then say, "You know what, that didn't work. That million dollar expense that we just made. We're just going to roll that back and not use it." So, that's sort of the way I think about the difference between a traditional optimization focused A/B Testing, where it's sort of like smaller bets designed to move the needle on a metric where if it doesn't work, you can turn it off versus these deep product experiences where what you're more interested in is being more quantitative about the impact of that release, but you're not necessarily interested in sort of like A/B testing focused optimization, picking a winner in a short period of time. One of the things that we've realized at LaunchDarkly is those are two separate tasks, they're two separate processes, and they require different analysis and different tools under the hood. And so, we're really excited at LaunchDarkly to be innovating on sort of both fronts, not only just providing a platform for optimization focused A/B Testing, but providing a platform where product managers can be more quantitative about the capabilities that they're building and not thinking about it in terms of optimization, but just in terms of measuring the impact of the work that they're shipping to customers. >> The impact, and of course, it's all outcomes focus as we talk about with customers and vendors and at any industry. Last question, John, for you as we're coming up on re:Invent in-person, what are some of the things that attendees can learn and see at the LaunchDarkly booth? >> Yeah. You're going to learn a lot about, if you visit our booth, you're going to learn a lot about sort of like the direction that we're taking, which is I think the exciting thing about LaunchDarkly as a platform is we really provide two capabilities. For engineering teams, we help you mitigate risks. We help you move more efficiently. That gives you more at bats as a team. It lets you ship more product and see whether it's working. LaunchDarkly also though provide something on the flip side of that, which is the ability for product managers to measure whether the changes that they're making are the right changes for their customers. And when you combine those two things in one platform, you get the ability for the engineering team to have more at bats, to create more change in production and see whether it's working. And then you get product managers the ability to measure the impact on their customers. And you combine that together, and at the end of the day, what LaunchDarkly provides is the ability for you as an organization to deliver business value better, more quickly through the R&D investments that you're making, the software that you're producing. >> And that's critical. I love that baseball analogy, more at bats. Fantastic, John, thank you for joining me talking to the audience about LaunchDarkly, what you're doing, the trends that you're helping customers address, the partnership with AWS, and what folks can learn when they visit the LaunchDarkly booth at re:Invent. We appreciate your time. >> Thank you so much, Lisa. I really enjoyed our conversation. >> Me too, for John Kodumal, I'm Lisa Martin and you're watching theCUBE's continuous coverage of AWS re-Invent 2021. (upbeat music)

(upbeat music) >> From the Bellagio Hotel in Las Vegas, It's theCUBE. Covering UiPath FORWARD IV. Brought to you by UiPath. >> Welcome back to theCUBE's coverage of UiPath FORWARD IV. Live from the Bellagio, in Las Vegas. I'm Lisa Martin with Dave Vellante. We're here all day today and tomorrow. We're going to talk about process mining next. We've got two guests here. Mike Engel is here, intelligent automation and process intelligence leader at PWC. And Ryan McMahon, the SVP of growth at UiPath. Gentlemen, welcome to the program. >> Thank you, Lisa. >> Thank you. >> So Ryan, I'm going to start with you. Talk to us about process mining. How does UiPath do it differently and what are some of the things being unveiled at this event? >> So look, I would tell you it's actually more than process mining and hopefully, not only you but others saw this this morning with Param. It's really about the full capabilities of that discovery suite. In which, obviously, process mining is part of. But it starts with task capture. So, going out and actually working with subject matter experts on a process. Accounts payable, accounts receivable, order to cash, digitally capturing that process or how they believe it should work or execute across one's environment. Right Mike? And then from there, actually validating or verifying with things or capabilities like process mining. Giving you a full digital x-ray of actually how that process is being executed in the enterprise. Showing you process bottlenecks. For things like accounts payable, showing you days outstanding, maverick buying, so you can actually pin point and do a few things. Fix your process, right? Where process should be fixed. Fix your application because it's probably not doing what you think it is, and then third, and where the value comes, is in our platform of which process mining is a capability, our PA platform. Really moving directly to automations, right? And then, having the ability with even task mining to drill into a specific bottleneck. Capturing keystrokes, clicks, and then moving to, with both of those, process mining and task mining, into Automation Hub, as part of our discovery platform as well. Being able to crowdsource, prioritize, all of those potential, if you will, just capabilities of automations, and saying, "Okay, let's go and prioritize these. These deliver to the greatest value," and executing across them. So, as much as it is about process mining, it's actually the whole entire discovery suite of capabilities that differentiates UiPath from other RPA vendors, as the only RPA vendor that delivers process mining, task mining and this discovery suite as part of our enterprise automation platform. >> Such a critical point, Ryan. I mean, it's multi-dimensional. It's not just one component. It's not just process mining or task mining, it's the combination that's really impactful. Agree with you a hundred percent. >> So, one of the things that people who watch our shows know, I'm like a broken record on this, the early days of RPA, I called it paving the cow path. And that was good because somebody knew the process, they just repeat it. But the problem was, the process wasn't necessarily the best process. As you just described. So, when you guys made the acquisition of ProcessGold, I said, "Okay, now I'm starting to connect the dots," and now a couple years on, we're starting to see that come together. This is what I think is most misunderstood about UiPath, and I wonder, from a practitioner's perspective, if you can sort of fill in some of those gaps. It's that, it's different from a point tool, it's different from a productivity tool. Like Power Automate, I'll just say it, that's running in Azure Cloud, that's cool or a vertically integrated part of some ERP Stack. This is a horizontal play that is end to end. Which is a bigger automation agenda, it's bold but it's potentially huge. $60 billion dollar TAM, I think that's understated. Maybe you could, from a practitioner's perspective, share with us the old way, >> Yeah. >> And kind of, the new way. >> Well obviously, we all made a lot of investments in this space, early on, to determine what should we be automating in the first place? We even went so far as, we have platforms that will transcribe these kind of surveys and discussions that we're having with our clients, right. But at the end of the day all we're learning is what they know about the process. What they as individuals know about the process. And that's problematic. Once we get into the next phase of actually developing something, we miss something, right? Because we're trying to do this rapidly. So, I think what we have now is really this opportunity to have data driven insights and our clients are really grabbing onto that idea, that it's good to have a sense of what they think they do but it's more important to have a sense of what they actually do. >> Are you seeing, in the last year in a half we've seen the acceleration of a lot of things, there's some silver linings but we've also seen the acceleration in automation as a mandate. Where is it? In terms of a priority, that you're seeing with customers, and are there any industries that you're seeing that are really leading the edge here? >> Well I do see it as a priority and of course, in the role that I have, obviously everybody I talk to, it's a priority for them. But I think it's kind of changing. People are understanding that it's not just a sense of, as Ryan was pointing out, it's not just a sense of getting an understanding of what we do today, it's really driving it to that next step of actually getting something impactful out the other end. Clients are starting to understand that. I like to categorize them, there's three types of clients, there's starters, there's stall-ers and those that want to scale. >> Right? So we're seeing a lot more on the other ends of this now, where clients are really getting started and they're getting a good sense that this is important for them because they know that identifying the opportunities in the first place is the most difficult part of automation. That's what's stalling the programs. Then on the other end of the spectrum, we've got these clients that are saying, "Hey, I want to do this really at scale, can you help us do that?" >> (Ryan) Right. >> And it's quite a challenge. >> How do I build a pipeline of automations? So I've had success in finance and accounting, fantastic. How do I take this to operations? How do I take this this to supply chain? How do I take this to HR? And when I do that, it all starts with, as Wendy Batchelder, Chief Data Officer at VMware, would say and as a customer, "It starts with data but more importantly, process." So focusing on process and where we can actually deliver automation. So it's not just about those insights, it's about moving from insights to actionable next steps. >> Right. >> And that is where we're seeing this convergence, if you will, take place. As we've seen it many times before. I mentioned I worked at Cisco in the past, we saw this with Voice Over IP converging on the network. We saw this at VMware, who I know you guys have spoken to multiple times. When a move from a hypervisor to including NSX with the network, to including cloud management and also VSAN for storage, and converging in software. We're seeing it too with process, really. Instead of kids and clipboards, as they used to call it, and many Six Sigma and Lean workshops, with whiteboards and sticky papers, to actually showing people within, really, days how a process is being executed within their organization. And then, suggesting here's where there's automation capabilities, go execute against them. >> So Ryan, this is why sometimes I scoff at the TAM analysis. I get you've got to do the TAM analysis, you've got to communicate to Wall Street. But basically what you do is you pull out IDC or Gartner data, which is very stovepipe, and you kind of say, "Okay we're in this market." It's the convergence of these markets. It's cloud, it's containers, it's IS, it's PaaS, it's Saas, it's blockchain, it's automation. They're all coming together to form this, it sound like a buzzword but this digital matrix, if you will. And it's how well you leverage that digital matrix, which defines your digital business. So, talk about the role that automation, generally, RPA specifically, process mining specifically, play in a digital business. >> Do you want to take that Mike or do you want me to take it? >> We can both do it? How about that? >> Yeah, perfect. >> So I'll start with it. I mean all this is about convergence at this point, right? There are a number of platform providers out there, including UiPath, that are kind of teaching us that. Often times led by the software vendors in terms of how we think of it but what we know is that there's no one solution. We went down the RPA path, lots of clients and got a lot of excitement and a lot of impact but if you really want to drive it broader, what clients are looking at now, is what is the ecosystem of tools that we need to have in place to make that happen? And from our perspective, it's got to start with really, process intelligence. >> What I would say too, if you look at digital transformation, it was usually driven from an application. Right? Really. And what I think customers found was that, "Hey," I'm going to name some folks here, "Put everything in SAP and we'll solve all your problems." Larry Ellison will tell you, "Put everything into Oracle and we'll solve all your problems." Salesforce, now, I'm a salesperson, I've never used an out of the box Salesforce dashboard in my life, to run my business because I want to run it the way I want to run it. Having said that though, they would say the same thing, "Put everything into our platform and we'll make sure that we can access it and you can use it everywhere and we'll solve all of your problems." I think what customers found is that that's not the case. So they said, "Okay, where are there other ways. Yes, I've got my application doing what it's doing, I've improved my process but hang on. There's things that are repeatable here that I can remove to actually focus on higher level orders." And that's where UiPath comes in. We've kind of had a bottom up swell but I would tell you that as we deliver ROI within days or weeks, versus potentially years and with a heavy, heavy investment up front. We're able to do it. We're able to then work with our partners like PWC, to then demonstrate with business process modeling, the ability to do it across all those, as I call, Silo's of excellence in an organization, to deliver true value, in a timeline, with integrated services from our partner, to execute and deliver on ROI. >> You mentioned some of the great software companies that have been created over the years. One you didn't mention but I want you to comment on it is Service Now. Because essentially McDermott's trying to create the platform of platforms. All about workflow and service management. They bought an RPA company, "Hey we got this too." But it's still a walled garden. It's still the same concept is put everything in here. My question is, how are you different? Yeah look, we're going to integrate with customers who want to integrate because we're an open platform and that's the right approach. We believe there will be some overlap and there'll be some choices to be made. Instead of that top down different approach, which may be a little bit heavy and a large investment up front, with varied results, as far as what that looks like, ours is really a bottoms up. I would tell you too, if you look at our community, which is a million and a half, I believe, strong now and growing, it's really about that practitioner and those people that have embraced it from the bottom up that really change how it gets implemented. And you don't have what I used to call the white blood cells, pushing back when you're trying to say, "Hey, let's take it from this finance and accounting to HR, to the supply chain, to the other sides of the organization," saying, "Hey look, be part of this," instead of, "No, you will do." >> Yeah, there's no, at least that I know of, there's no SAP or Salesforce freemium. You can't try it before you buy. And the entry price is way higher. I mean generally. I guess Salesforce not necessarily but I could taste automation for well under $100,000. I could get in for, I bet you most of your customers started at 25 of $50,000 departmental deployments. >> It's a bottoms up ground swell, that's exactly right. And it's really that approach. Which is much more like an Atlassian, I will tell you and it's really getting to the point where we obviously, and I'm saying this, I work at UiPath, we make really good software. And so, out of the box, it's getting easier and easier to use. It all integrates. Which makes it seamless. The reason people move to RPA first was because they got tired of bouncing between applications to do a task. Now we deliver this enterprise automation platform where you can go from process discovery to crowd sourcing and prioritizing your automations with your pipeline of automations, into Studio, into creating those automations, into testing them and back again, right? We give you the opportunity not to leave the platform and extract the most value out of our, what we call enterprise automation platform. Inclusive of process mining. Inclusive of testing and all those capabilities, document understanding, which is also mine, and it's fantastic. It's very differentiated from others that are out there. >> Well it's about having the right framework in place. >> That's it. From an automation perspective. I think that's a little bit different from what you would expect from the SAP's of the world. Mike, where are you seeing, in the large organizations that you work with, we think of what you describe as the automation pipeline, where are some of the key priorities that you're finding in large organizations? What's in that pipeline and in what order? >> It's interesting because every time we have a conversation whether it's internal or with our clients, we come up with another use case for this type of technology. Obviously, when we're having the initial conversations, what we're talking about is really automation. How do we stuff that pipe with automation. But you know, we have clients that are saying, "Hey listen, I'm trying to carve out of a parent company and what I need to do is document all of my processes in a meaningful way, that I can, at some point, take action on, so there's meaningful outcomes." Whether it be a shared services organization that's looking to outsource, all different types of use cases. So, prioritizing is, I think, it's about impact and the quickest way to impact seems to be automation. >> Is it fair to say, can I look at you UiPath as automation infrastructure? Is that okay or do you guys want to say, "Oh, we're an application." The reason I ask, so then you can answer, is if you look at the great infrastructure plays, they all had a role. The DBA, the CCIE from Cisco, the Cloud Architect, the VMware admin, you've been at all of them, Ryan. So, is there a role emerging here and if it's not plumbing or infrastructure, I know, okay that's cool but course correct me on the infrastructure comment and then, is there a role emerging? >> You know, I think the difference between UiPath and some of the infrastructure companies is, it used to take, Dave, years to give an ROI, really. You'd invest in infrastructure and it's like, if we build it they will come. In fact, we've seen this with Cloud, where we kind of started doing some of that on prem, right? We can do this but then you had Amazon, Azure and others kind of take it and say, "Look, we can do it better, faster and cheaper." It's that simple. So, I would say that we are an application and that we reference it as an enterprise automation platform. It's more than infrastructure. Now, are we going to, as I mentioned, integrate to an open platform, to other capabilities? Absolutely. I think, as you see with our investments and as we continue to build this out, starting in core RPA, buying ProcessGold and getting into our discovery suite of capabilities I covered, getting into, what I see next is, as you start launching many bots into your organization, you're touching multiple applications, so you got to test it. Any time you would launch an application you're going to test it before you go live, right? We see another convergence with testing and I know you had Garrett on and Matt, earlier, with testing, application testing, which has been a legacy, kind of dinosaur market, converging with RPA, where you can deliver automations to do it better, faster and cheaper. >> Thank you for that clarification but now Mike, is that role, I know roles are emerging in RPA and automation but is there, I mean, we're seeing centers of excellence pop up, is there an analogy there or sort of a similar- >> Yeah, I think the new role, if you will, it's not super new but it's really that sense of an automation solution architect. It's a whole different thing. We're talking about now more about recombinant innovation. >> Mike: Yeah. >> Than we are about build it from scratch. Because of the convergence of these low-code, no-code types of solutions. It's a different skill set. >> And we see it at PWC. You have somebody who is potentially a process expert but then also somebody who understands automations. It's the convergences of those two, as well, that's a different skill set. It really is. And it's actually bringing those together to get the most value. And we see this across multiple organizations. It starts with a COE. We've done great with our community, so we have that upswell going and then people are saying, "Hang on, I understand process but I also understand automations. let me put the two together," and that's where we get our true value. >> Bringing in the education and training. >> No question. >> That's a huge thing. >> The traditional components of it still need to exist but I think there are new roles that are emerging, for sure. >> It's a big cultural shift. >> Oh absolutely, yeah. >> How do you guys, how does PWC and UiPath, and maybe you each can answer this in the last minute or so, how do you help facilitate that cultural shift in a business that's growing at warp speed, in a market that is very tumultuous? How do you do that? >> Want to go first or I can go? >> I'll go ahead and go first. It's working with great partners like Mike because they see it and they're converging two different practices within their organization to actually bring this value to customers and also that executive relevance. But even on our side, when we're meeting with customers, just in general, we're actually talking about, how do we deal with, there's what? 13 and a half million job openings, I guess, right now and there's 8500 people that are unemployed, is the last number that I heard. We couldn't even fill all of those jobs if we wanted to. So it's like, okay, what is it that we could potentially automate so maybe we don't need all those jobs. And that's not a negative, it's just saying, we couldn't fill them anyway. So let's focus on where we can and where, there again, can extract the most value in working with our partners but create this new domain that's not networking or virtualization but it's actually, potentially, process and automation. It's testing and automation. It might even be security and automation. Which, I will tell you, is probably coming next, having come out of the security space. You know, I sit there and listen to all these threats and I see these people chasing, really, automated threats. It's like, guys a threat hunter that's really good goes through the same 15 steps that they would when they're chasing a false positive, as if a bot would do that for them. >> I mean, I've written about the productivity declines over the past several decades in western countries, it's not universal around the world and maybe we have a productivity boost because of Covid but it's like this perpetual workday now. That's not sustainable. So we're not going to be able to solve the worlds great problems. Whether it's climate change, diversity, massive deaths, on and on and on, unless we deal with that labor gap. >> That's right. >> And the only way to do that is automation. It's so clear to me that that's the answer. Part of the answer. >> It is part of the answer and I think, to your point Lisa, it's a cultural shift that's going to happen whether we want it to or not. When you think about people that are coming into the work force, it's an expectation now. So if you want to retain or you know, attract and retain the right people, you'd better be prepared for it as an organization. >> Yeah, remember the old, proficient in Word and Excel. Makes it almost trivial. It's trivial compared to that. I think if you don't have automation chops, going forward, it's going to be an issue. Hey, we have whatever, 5000 bots running at our company, how could you help? Huh? What's a bot? >> That's right. You're right. We see this too. I'll give you an example at Cisco. One of their financial analysts, junior starter, he says, "Part of our training program, is creating automations. Why? Because it's not just about finance anymore. It's about what can I automate in my role to actually focus on higher level orders and this for me, is just amazing." And you know, it's Rajiv Ramaswamy's son who's over there at Cisco now as a financial analyst. I was sitting on my couch on a Saturday, no kidding, right Dave? And I get a text from Rajiv, who's now CEO at Nutanix, and he says, "I can't believe I just created a bot." And I said, "I'm at the right place." Really. >> That's cool, I mean hey, you're right too. You want to work for Amazon, you got to know how to provision a EC2 instance or you don't get the job. >> Yeah. >> You got to train for that. And these are the types of skills that are expected- >> That's right. >> For the future. >> Awesome. Guys- >> I'm glad I'm older. >> Are you no longer proficient in Word is the question. >> Guys, thanks for joining us, talking about what you guys are doing together, how you're really facilitating this massive growth trajectory. It's great to be back in person and we look forward to hearing from some of your customers later today. >> Terrific. >> Great. >> Thank you for the opportunity. >> Thank you for having us. >> Thank you guys. >> Our pleasure. For Dave Vellante, I'm Lisa Martin, you're watching theCUBE live from the Bellagio in Las Vegas, at UiPath FORWARD IV. Stick around. We'll be back after a short break. (upbeat music)

from the cube studios in palo alto in boston connecting with thought leaders all around the world this is a cube conversation hi lisa martin with the cube here covering some news from dell technologies i'm pleased to welcome one of its customers liam furlong the i.t manager from revelation software liam great to see you today thanks lisa it's fantastic to be with you and we're socially distant california you're down in australia i know it's early morning for you but we're pleased to be chatting with you so give me and our audience an overview of revelation software who are you and what do you do yeah sure revelation software is a software development company no surprises there and our primary product is a tool called revtrack and for all those sap users out there we help you get your changes navigated safely through the wide landscapes and the open seas of your sap environment so we're all about change management and delivering certainty in what is really rapidly changing landscapes uh in the it world so customers can go to you for all of their challenges with all their sap data and sort of offload that basically i mean that sounds lovely i'm sure many of them would take that so talk to me about your itune manager talking about your i.t environment i know you're highly virtualized just give us an overview of what your data environment looks like we um like a lot of software companies we give our development teams a lot of freedom and so over the years a lot has definitely built into our environment we have hundreds of vms and even more sap landscapes we are committed to our customers to provide a lot of previous version compatibility both in our product but also in sap we support more of sap's old versions than they do we just want to make sure that everyone is able to do their job and focus on what they're trying to do rather than worrying about you know do i have to upgrade am i going to be forced ahead uh in you know especially in a change management landscape and so we have a lot of history a lot of old environments and we manage that by using a lot of on-prem we have local data centers like everyone i guess but also we've got a great multi-cloud environment now and it helps us to really uh provide an excellent environment for our teams to develop in the way that they want to support our customers uh in an efficient way but also without us having to over commit to hardware and so on so you have highly virtualized environment about 150 vms nearly 500 sap landscapes so big administration of overhead talk to me about how you were protecting your data i'm assuming vms maybe some sap databases and servers how are you protecting that before using dell's new integrative approach yeah we uh used a targeted appliance uh style i guess we built up what we thought was the right solution we had a lot of legacy thinking really but uh tools we used a lot of scripts previously we used the veeam platform and that presented an ever increasing set of challenges as you can imagine with s s3 s4 hana rolling along the environment just had to change our backup load was increasing our backup windows weren't getting any larger and our backup targets weren't getting any larger so we really needed to ask some hard questions about what we were doing and whether it was working for us we had absolutely no cloud integration our off-site copies were completely inadequate and so as an i-team manager who is um the guy at the end of the road when it comes to rpo and rto and uh certainty of restorability i was not sleeping well it's fair to say well and that's something that obviously you you look to a company like dell technologies to help with sleep as a sleep aid but you guys i saw that after 20 years you were testing and a hosted version of your rev track insights product and needed cloud dr and you kind of talked about meeting customer slas and i was reading your case study and there was some big challenges there with respect to the sla front yeah definitely um i guess uh actually we were really fortunate to have started a conversation with dell even before we were bringing our cloud platform online we knew we were going to need to be able to address cloud dr it was on the horizon for us and so being able to talk with a vendor that had everything wrapped in uh the idea of an integrated appliance was really quite foreign to me the um the the thought that i could trust dell technologies to actually do this better than me i made that that sounds a bit uh arrogant but the truth is you know i knew my environment and they didn't but what was really stand out for us in the process is dell knew that too and they climbed into our environment and worked really hard they really actually wanted to understand well what were our challenges and what were our loads what was our environment really like and then work with us on a strong solution and i was amazed it felt really like the cavalry had arrived and they knew exactly what they were doing and then they worked overtime to help us find a great solution and it has been a fantastic solution not only solving the challenges we faced at that time of deployment but knowing what was on the horizon going into the cloud and having a sas platform uh we were future-proofed in a way that i was hopeful about but now that we're using it in that way i'm confident and every day i know that it's working properly for us that confidence is absolutely critical but you use the term that we hear so often in technology future proof talk to me about when you hear that as an i.t manager what does that mean to you and how is dell tech with the integrated approach delivering that yeah i think um i mean if i'm just being honest uh i generally dismiss that when i hear anyone say that they're future-proofed because no one knows what's coming i mean here we are living this year outright and uh we we knew 2020 was going to be a big year but not in the ways that it has been uh i think that even though we wanted to believe that this backup tool would cover us we weren't sure uh what it has meant is there are two real standout things one there's a suite of functionality and in the integrated appliance which we didn't need then but it was standing by and it was easy to turn on it wasn't like oh and now you'll have to pay this extra fee or now you'll have to deploy these extra tools it was all ready to go and so they've brought their years of experience and forecasting and built in a bunch of functions which you're not going to need and no one is going to need all of the tools out of the box but over time you can deploy it and the other really big one for us is all of the extra storage that we might need as our backup requirements grow shipped in the box which is a huge cost to the vendor um but it's just sitting there ready for us to consume as we need which is absolutely fantastic for me i don't need to take our backup system offline to upgrade i don't need to consume more rack space i don't need to use more power it's already doing everything it needs to and it's just about rolling forward easily as we move forward as a company so walk us through what the environment looks like now we mentioned 150 vms a big sap landscape give us a picture of the technologies and what dell is helping to protect in your environment yeah so um dell dell covers everything the the integrated appliance we're using um actually it meets all of our needs uh i'm a paranoid and in my job so we have extra bits and pieces kicking around but the power protect device is our go-to we know that it's going to be there it's going to be online it's going to have covered everything from our on-prem so we use a vmware environment locally and we're backing up all of those vms every night about 54 terabytes of data and we knock that out in about a 90 minute window which is absolutely fantastic so that backs up to local and then it ships up to our cloud environment so we've got our offsite covered in that same night then we've also got environment i guess using the amazon example we have a multi-cloud so we've got things in a couple of different cloud providers but to use amazon as an example we have production systems running up there we have our sas environment running up there and we capture that also with our power protect device and bring everything back down and so now we've got that covered as well and so no matter what our problem is i've just got one place to go to to say i need to restore this and i need to do it fast and we can get that done uh straight away it's fantastic and that's what i've been hearing i've spoken with a number of folks already including the vp of product marketing caitlin gordon and we're hearing a lot of that one-stop shop sort of description for the integrated appliance i'm wondering if you could give us a compare and contrast uh power protect the integrated appliance as you said and described the benefits that you've already achieved versus the targeted approach with theme that you had before yeah sure um what we came from was only being able to back up mission critical systems nightly and everything else had to be backed up weekly to achieve our backup windows even still monday morning was uh was a nerve wracking a few hours while the weekend back up kind of crawled through and finished and people are like oh systems are a bit slow this morning like oh yeah we're looking at that you know um we came from that to getting as i said earlier everything done every night which is a complete transformation for us it means that we don't need to worry about we used to have to supplement our veeam backup with scripts because we could get the scripted backup done uh much faster and so we would go oh we'll restore with veeam and then we'll lay a script over the top to recover everything up to last night but now um it's just all uh covered through that one appliance um again in our cloud environments we use the local tools to provide a local backup and that's great to have previously that was mission critical we had to have that working and we had to have our technicians up to speed with four or five different uh tool sets but now they it's great that they are aware of those tools but really it's just about understanding uh one application in regards to a targeted solution you end up having really all these building blocks that only one person really knows how they all string together but now not only do our whole team understand how it works together but it's one phone number to find a whole group of people who know how it works together and they can help us you know from upgrades deployments restores anything we need if i'm on leave then i know that someone else from deltek can step in and cover me for any of their questions it might normally bubble up to my level um one of my favorite numbers i'm sorry i feel like i'm ranting but one of my favorite numbers is you know we came from using a different hardware vendor's san and we were getting compression maybe of three to six times uh on data we get compression from a month view of 150 to 200 times and if we expand that out to an annual view we get compression rates of 300 times on our data which means instead of having literally 15 ru of storage we have two u of storage uh the cost per terabyte is down by hundreds of dollars it it makes me look really good and i haven't had to do anything all i did was just go yep you guys do it you guys deploy your solution so it's been those are huge deduplication numbers i know caitlin gordon shared with me on average 65 to 1 but you you basically at least double that and in terms of of making you look good that's something that's actually quite important in terms of i.t and the business uh making sure that what you can deliver to the business is the confidence and you and your team that their data is protected can you share a little bit about maybe the i.t business relations and how this technology has helped them just have that confidence yeah definitely um i mean as you say every part of the business sees a different thing our development team are paying attention to very different things to our accounting team these numbers definitely help me to make friends in both teams as a it manager if the backups do their job properly if this all works no one notices if this goes wrong i break the business so the stakes are pretty high with backup but even though that's true and we know that's true committing a big financial investment is still hard it's still a moment where you hold your breath and ask was it worth it but now that we've been able to show the numbers to our executive teams and they can see how much money they're saving how much money we would normally be reinvesting at this point but we can now make that available for other projects we can put that into further development we can put that into improving our sas platform that really works for us as a business we want to serve our customers better we don't want to waste our time and money on stuff that affects just our day-to-day we want to be really focused where our people are and with what they care about so by putting money back in the pockets uh that's a big win and by making our uh infrastructure teams more free their time is freer because they're not spending you know we do restores every week pardon me every week because those restores now run more smoothly and they are faster and there's less hunting around to try and find the backup that actually worked then that means our infrastructure teams are free to also now do other upgrades to work alongside say our developers they want to be running the current versions of the atlassian suite not you know a version from a year ago but we've got more time to do that work now it makes a big difference well that workforce productivity that you're alluding to it can be hugely impactful across the business it's not just that now you know you've got one solution one phone number to call if there's issues you've got more time back to be more innovative more strategic and so do the rest of the folks on your team so the business overall that workforce productivity can really be very widespread in a good way absolutely and it's well felt i think you know one of the things that it's really hard to put a dollar value on but it is really key is people don't like doing rework and backup recovery feels like reworking like i've been here before and so by mitigating particularly this aspect of our roles our teams are happier they generally are enjoying their work more because they're as i say they've got more time to work on things that are energizing and rewarding and across the business people feel better as well there are a lot of complications in anyone's job but certainly from the direction that hardware and storage and backup is being uh concerned we've taken away a big stress for me for example it's important that we test our dr scenario obviously everyone says that but now we can actually do it and now we can actually do a full dr production outage and go okay great let's shut it down and see what happens and we were able to do that a couple of times a year we don't have to pay for a cold dc or a warm dc in the wings we can recover to the cloud we our dr site is vmware cloud on amazon so we can spin it up and do the whole dr scenario our dr is engaged within about three hours from a full building loss and not only is that great peace of mind but also again it puts great data into the hands of my cio he's able to present on business continuity issues to the executive team and show that we're actually caring about the business and caring about the things that people do worry about and again makes people look good which is uh which is always helpful it is absolutely as you said it's really you know if you can't restore the data you're kind of stuck so now i know why you look so rested because you you have the solution you're sleeping better at night liam has been such a pleasure talking to you great work and we look forward to hearing more great stories to come from revelation software thanks so much lisa it's been a wonderful time per liam furlong i'm lisa martin you're watching the cube you

>> Narrator: From theCUBE studios in Palo Alto in Boston, connecting with all leaders all around the world, this is theCUBE conversation. >> Hi, everybody, this is Dave Vellante, and welcome to this episode of Startup Insights. Andrew Lau is here with me, he's the co founder and CEO of a relatively new company called Jellyfish. They focus on engineering management, which is kind of a new space that we want to present to you, Andrew, great to see you, thanks for coming on. >> Hey, Dave, thanks for having me on. >> So when I see co founder and a title I always ask why did you start a company? What's your Why? >> So the three co founders myself, Dave Gourley, and Phil Braden,we actually met geez more than 20 years ago, at a company called Endeca. We had the chance to kind of bring the proverbial band back together, just 'cause rare is the chance to work with great people. And for us, Jellyfish really was coming out of our own experiences. All three of us grew our careers running big engineering teams big product teams. We realized how hard it was to really lead those teams and connect them to the business at scale. And that's the problem we just got together to solve. >> Yeah, so interesting right, and Endeca, another East Coast company, great exit, brought by Oracle. And of course, when you say in Endeca, I always think okay, Jellyfish, you guys in the search business, but you're not in engineering management. What is engineering management? And, you know, how does it relate to some of your past experiences? >> Great question, well, so engineering management engineering management sector or management platform, as we talked about, really comes down to how we can facilitate the tools to make leading big teams easier, right, we've realized that as you get to larger teams, teams bigger than 50, bigger than 100 engineers, it's really hard to understand what the team is doing really hard to make sure that they're working their best in making sure they're pointed in the right direction. And even Furthermore, how to connect that to the business and how to make sure the business successful after the team dies. And as for us, is really around making tools and processes available that they really help accelerate the act of leading big teams. >> So when you think about it, I mean, it's really the problem you're solving is visibility, kind of what's going on in engineering and providing metrics is that a sort of a fair, high level? >> I think it's a perfect high level statement. When we actually got together and talked about the problem space we all saw as we lead these big teams. We quickly drew an analogy to you know, when I started my career in the late 90s, there was a time before Salesforce and CRM were pervasive, right And so really quickly, we drew a quick and easy analogy that said, like, hey, why isn't there a Salesforce for engineering? That is, why isn't it that same leadership and executive visibility into how a team is progressing and to make sure it's aligned with the business? >> Well, when you think about it, right, Salesforce, what 1999 we had the first Salesforce clouding before the real cloud hit, you know, we certainly have marketing clouds now capital Management clouds, customer service management clouds, why not an engineering cloud right? >> I think it's probably you follow that map there, I think we've seen the last 20 years, that clouds actually kind of progressed from sales to marketing to success to HR, as you pointed out, I think the last bastion in organizations is the engineering team right? It just so happens right now that engineering teams probably are the most strategic, if not the most expensive team in most companies roster. And so really, providing visibility is really necessary at this time. >> So I don't run engineering it's Silicon Angle by my co-CEO and partner John furrier does. But I'm always like asking him like, hey, what are you guys working on? What are the deliverables? What am I going to get it and when? how we do it on quality, and so forth and so I think just scanning your website, reading some of your blogs, these are some of the things that you really focus on. You wrote a five part series, I think, I don't know if I'm dying to see part five but four parts are out. I want to dig into some of those, I think they're in, I think you called it, you know, five things that you should present to the board. >> Yeah, yeah Like, you know, a great question around like, there's a series were four or five in two, I'm talking about what are five slides that heads of engineering should be showing to their boards or even their executive management teams? You know, early on in the process, I'm aware, before we actually started the company, we did a ton of research, talking with leaders at scale, trying to ask them, like, how do you manage your team? How do you connect into the business? And out of those conversations fell, you know, asking folks like, well, what slides do you show your board meeting? And and the answer was, there wasn't ubiquitous answer. People were looking for answers and so we really synthesized a number of different leaders that we thought were really successful in this world, and really put together this series to talk about like, what are the metrics that people should be measuring? >> Well, let's talk more about some of those metrics so you know, I mentioned so what am I going to get? When what are some of the other key things that people are focused on? I mean, obviously, quality, where I'm spending my money. But what are some of the ones that you're seeing, aligning with business and really driving business outcomes? >> So okay, so part one I think you talked about right which is, you know, what are you getting? What got shipped out the door? what's coming down the pipe? I think job one for a leader of engineering and Product is to talk about what's coming out, right in the same way that sales job is actually hit a revenue number and talk about the pipeline coming down the way it's an engineers job to actually ship new product that you can sell your users can engage with so that's definitely slide one. Slide two, you already alluded to here two is about quality, right? I think if you're shipping product that actually can't hold quality in the eyes of your customers, it won't last very long. So I think it's really important to show command of quality and actually show metrics that actually measure quality over time in the lens of your customer. Slide three, I think we're really talking about alignment. Making sure that your team is spending your dollars, time and effort in the right way that actually aligns with what the business wants to right? So examples might be, if you're a company that splits time between enterprise and SMB efforts, well, making sure that the features the team is working on actually aligned to the strategy of the company, right? And you can't do that if you don't measure that. And then slide four is really around capturing broad level productivity. But is the team healthy moving forward? And then the last of which is really the preview coming up for the next segment here is going to be about really around the team in hiring right. How is the team holding up? How's the morale? And how's the actual hiring pipeline looking and ramp? All right in terms of new employees right, it's really the people side of the engineering leadership. >> Cool, thanks for teasing that a little bit. I'm glad to hear it's not just about productivity, 'cause there are tools out there that can measure developer productivity, but seems like you're taking a broader approach and building a platform to really take a more end to end sort of lifecycle view. >> Yeah, I think we really look about, think about it as look productive is really important. I think it's necessary but not sufficient. You can talk about a boat, you can roll faster. But if you're rolling the boat in the wrong direction, who cares? So it's as important to make sure that alignments in place and actually making sure that the rest of communication and context is really in place to make sure that team succeeds and the lens of the business. >> Andrew I'm interested in the market, I mean, my sense is engineering management is sort of new, very new, actually, although we talked about productivity being sort of one of the metrics and there are tools out there, but how do you look at this market? Is there a big whale that you're targeting? Or is this more of a Greenfield up? >> I think really, it's a Greenfield opportunity. I think if you were to you know, people don't wake up right now with a quadrant they look at or a wave that they're actually measuring on at the moment. It doesn't say isn't coming. I mean, if you look at this as a baseball game, we're probably getting one and defining market right now. And you can tell because if you look across the vendor space, I think there's a lot of variety of different solutions in these places. And so, from my view, you know, there's room right now just to innovate and describe what this platform really is going to be right and that's what the next few years are going to look like and defining this market. >> Did the unknown nature of the market? Was that was that a challenge in terms of your race? >> I actually think that's actually the opportunity. I think both as founders and and our investors, I think really, whenever you have these Greenfield opportunities, it helps you create big opportunities, I either grow you know, this better than anyone, I think, define the markets are very clear in the box you're trying to fill in, it's a race to do that. You know, this space here is a space ripe for innovation. It needs innovation, both in product and process and how you going to market and the story will be told five years from now. >> So I want to talk about your go to market but before we do, so one of the things you got to do when you're doing your investor pitches, you got to figure out the total available market your served market. I mean, how did you do that? What can you share with us about the TAM? >> I mean, okay, there's the quantitative answer, right, you could pick apart all the companies in the world count all the software engineers and I can tell you, it's going to be a big number, right You can also map it to other large software engineering companies like Atlassian, or even Microsoft and talk about the markets there. But I think, you know, look, the world has moved far for long with that like, what's the word every company is a software company now? I think it's not a necessary part of the pitch anymore. I think everyone intuits the TAM is large, because even air conditioning companies now have hundreds of software engineers, It's no longer this niche thing like it was 20 years ago. I think literally, you know, every company in the planet could be a potential customer of Jellyfish in the future. >> You know I feel like some sometimes if you can actually size the TAM, it's maybe a negative in your race because if the TAM is just so obviously large then investors say hey, okay, check the markets huge, and that's what they want to see. >> And I think part of it too, is like we've seen the last five years, not just has you know, every company become a software company. This also means the engineering departments and how they recruit have been really scrutinized. Everybody needs more wants more engineers, they're hard to get and expensive. I think everyone's realizing like, because of both of those things, everyone cares a lot more, It's no longer this, you know, small number of people have low cost. It's actually just an expensive investment, a strategic one. And I want to make sure everyone wants to make sure it's pointing in the right direction now. >> So there's a lot of people in our community, young people get, you know, either just graduated from college or been out for, you know, 567 years working at a company and feel like they want to do their own thing. And they're always interested in how you did it, how you got started, how you ascended the company where you know how you seated it, I think, I think you guys started in 2017, I think you've raised $12 million. But take us back to the beginning, how did you and your co founders get launched, you know, how did you see the company and bootstrap it? So I mean, I, I think for us, like we're lucky to have actually been through all three of us through a number of different startups. So I think this is for us coming with a lot of awareness of actually how to build the company, we had the chance, you know, in at the talent 2016 to actually get the proverbial band back together. We hadn't worked together in probably shy of 15 years. But I think we really respected the chance to do so. And so we got together and said, like, hey, let's see if there's an opportunity for us to do something together. And so that was a real journey, you know, we pushed through a number of different concepts, we largely fell into this one simply because of our backgrounds, right, it is an area that we actually bring some personal expertise to and our networks bring it that way, but also some passion around wanting to actually solve it. So I think it's probably at the end of 16 that we actually said, like, hey, this is a space we might be interested in. We actually spent I think the the first couple months of 17 just interviewing every VPE CTO CEO, exactly we could find I think we probably talked to north of 65 technology leaders in that time period. Largely just actually asked him like, hey, what do you think about this space, this idea? What do you do instead? In fact, tell me not to start the startup, I don't want to invest x years of my life to find out that there's a better solution out there. The part that was I think amazing was that everyone was interviews, everybody kind of stopped at the end of it and leaned in and said, "hey, can you do me a favor? Can you write up the, whatever your notes on this and just send me the actual answer? So at that moment, we knew that, you know, we didn't have the solution yet. But we knew there's pain out there an opportunity to actually solve something, we weren't the only ones that actually identified that. And so that became the mission, which is how do we make people in this seat? How do we make their lives better, right? And, you know, sliding forward that you know, concurrent with actually, the early checks coming in, we actually call those same folks back and said, hey, can we work with you to build the product? So from a philosophical standpoint, we really believe in actually building with our customers, right, and so, from the first moment, you know, pre product, you know, pre code, we sat down with those same people and said, hey, let's work with you. let's do things by hand, let's do with your data, just to make sure that we understand what we're building a use case that you care about. >> Okay, so you co-created really with the customers as you actually started generating revenue, kind of a sell design build model, is that right, or? >> Yeah, I want to think of a much more of a, as an alpha product development, right, I think, you know, our philosophy on that early on, let's say June of 17. It was, look, we'll do your manual, we'll do your board decks for you, we'll do your management team slides, we'll do your metrics will do your capitalization, right. We'll do whatever you need on a manual basis, as long as we can work with you and your data. And you know, because we always had an eye on building the platform there. And so behind the scenes, of course, we're automating all of this. But that helps make sure that the use cases that we're building for were things they actually needed, we're going to use. >> Did you find you had to leave a lot on the cutting room floor? In other words, a lot of times when you take that approach, and you kind of try to generate maybe early revenue from customers, you sometimes get sick, especially in the enterprise, you get sucked into specials and some of your custom work that might not scale across the the other organizations. You guys obviously experienced, was that something you guys put a lot of thought into and how did you manage that? >> Look, I don't think it's magic, I think we were aware. To your point we've done this a few times, so at least we knew the pitfalls, like yeah, so some stuff has been left on the cutting for in the sense that we probably, you know, pushed harder on areas that we push less on today. I don't think anything was abandoned. I think part of it is that, you know, there's two sides of it, right, which is, if you're able to think about where you want to go, which is building a platform, you can always take any engagement and trade it off and say like, hey, is this something we want to build? Does this make sense for actually leading us towards the long platform story? you don't have to do every opportunity that comes along. So I think you need to thread the needle and I should take advantage of working with customers, but also making sure you have an eye for where you're going the North star so you can pick and choose which project you want to work on or which customers you know you want to work with because end of the day products are really the byproduct of who you work for who you serve as who you actually build for. And so we're very conscious along the way to choose the right individuals, the right partners that helps shape the product over time. >> And you guys had some some engineering chops and your own Andy Jassy says there's no compression algorithm, the algorithm for experience and then maybe that's an Amazon thing, but I hear him all the time. So you know, they had that to your advantage. And so, okay, so i got your website I see you've got customers so you know, you're well into your journey here. You've got product market fit, I presume you've got you know, your SaaS model your pricing down, but where are you in your sort of journey and you're phasing? >> I mean, geez, those words all change quite a bit these days, I would actually say product market fit is never a binary thing, that the constant journey, right so I would say that we're always working on that because the markets are always moving. And we have a market that is changing month on month and a quarter on quarter, right, so I never want to declare victory on that. Because that's going to get left behind. I think in terms of our journey, like we have on the team right now is 27 people normally based in downtown Boston, We're all working from home at the moment. You know, we have, you know, a sales team in place now of I think six, seven folks now. So we're in market actually pushing this forward. But, you know, I think for us, we're out there really kind of scaling the story right now. I think we've had some tremendous customers we had the chance to work with, we have a product that we're really proud of. I think we just need to put more units out there and more customers to actually make them more successful. So I think anything we're really in the act of repeating every function of the organization right now. It's really kind of build it up. So okay, so I mean, normally when you do a startup, you go to your friends, first the people in your core circles, you get them to that's I'm sure you did something similar. You're obviously beyond that phase of six to seven salespeople, you're starting to scale up, you've probably got a good sense as to that types of salespeople that you're looking for. And then now you're trying to figure out okay, it sounds like how much do I spend on marketing? How does that affect sales productivity? And then how do we scale that whole thing up and then go hyper scale and build a moat and all that other good stuff. >> Yeah, I think you're exactly right, I mean, I think we're at the point now where we can actually start making trade offs, right, like, you know, like, you know, do we actually add a additional salesperson? Do we actually invest in marketing programs? Do we actually build out more strategic product? I think you know, we are still the point we have to make trade offs, right, but the business is mature enough that we can make trade offs right, that makes sense. >> So let's talk about customers, I mean, maybe you could give us some examples, some of your, your favorite examples how you've impacted their business. >> Well, I think if you look at actually our website, I think there's a few case studies up there, I think there's building them up there, there's like salsify books at toast. I think all three of those actually really talk about different kinds of use cases around how we actually affect them. So One of which will we're really helping them actually on alignment, right and making sure that their team is working on the most important things, And, and in those situations, when you're working on the most important thing, you're really kind of essentially getting opportunity cost of engineers and making sure that they're driving towards things that you really will help the business, right so if you're, if you're looking at it that way, you're finding engineers that can help you progress faster, but you're building more product faster, because you can focus the energy where the team is going. And so that's case one, I think another case is really is around, making sure around quantitative management visibility, right, making sure that the team is visible in the metrics and in their output to make sure that they're performing their best, right, and that might mean everything from automated performance management to just making sure that people aren't left behind and making sure that they're the team is actually healthy in their function. And then the last of which is really around capitalization, which is a financial process and really automating that, that's out of the house which is, you know, capitalization requires a traditionally, engineering leaders have to manually fill out spreadsheets for finance, and for the accounting team to make sure that they're actually able to account for where the team effort is going, and then it can actually capitalize it correctly when we treat it from a financial perspective. And so we automate that process that just makes everyone's lives easier. So you're no longer manually data on a week by week basis. >> So I may have obviously seen some of your product and some of the outputs but your your SaaS based model, you know, cloud pricing, all that sort of modern, you know, approaches and business practices. And but what else can you tell us about sort of your, pricing model and how you're going to market? >> Yeah, so I think, you know, we are a SaaS hosted application. We also have, you know, open source agents that have been deployed on premise to actually, you know, whether to work with complex network architectures or deal specific redaction concerns, so we got to operate an on premise environments in that way. You mentioned our pricing is SaaS base we broadly price annually, you know, from broad strokes perspective, it's relative to the the size of the engineering team. Very simply like a, an engineering team of 300 people is a lot more complicated than engineering team of 30 people, right, put it that way. And the pricing reflects that. And then to your question around, like, what else I can talk about? Well, you know, I made the analogy earlier around like they were trying differentiating what Salesforce did for sales. What I mean by that really is providing that executive and leadership visibility to that department, right, if you're looking at the innovation that Salesforce brought in the early aughts, it was really getting stuff out of notebooks into the cloud through manual data entry, and in contemporary sales, I think there's less of that these days, it's all through plugins and voice recognition and stuff. And in the same way, in the engineering side, we're not in the business of actually asking for new data entry. In fact, we connect with systems engineers already using You know, the JIRA is the GitHub to get labs, all they know that their testing tools and their CI tools and then all of those things really we emit what we call engineering signals. So the engineers don't do anything differently. We collect that data, we connect to those systems, we clean that data, normalize and contextualize it with respect to business data. And that's actually where the insight actually comes from. Because if you just look at the raw engineering data by itself, there's not a lot you can do with it, right you know, I joked with you in our kind of earlier conversation, which is, you might look at, you know, your 300 get or request your engineers are produced thing. Like, it doesn't really help you figure out if you're going to do great this quarter, right? And so for us, we really bring that in contextualize it and make sure that you understand it in a business context, to talk about like, hey, is the team accelerating and being successful in the ways that we need the business needs them to do. >> Yeah, you're so right I get a stream of those every day every week and I open them up and I go, okay, I don't know. There's people that work in sort of last sort of topic areas is I want to understand where you want to take this thing. I think I'm writing that you've raised about $12 million, you obviously got a big market seems like you've got a great product. I mean, if I'm you, I'm throwing gasoline on the fire, I want to run the table, you got to create the market . So that's sometimes kind of expensive. Where do you want to take this thing? >> I mean, look, this may sound hubris bowl, but our ambitions are to build a large multi billion dollar standalone software company. And and I think, you know, part of the reason why I say it that way is that I think it's important to have a North star, right. It's important to have a North star to make sure that we're all headed in the right direction. We get the right team members, actually, as we grow the team, and then we actually capitalize it accordingly. I think if you look at the analogy, we started out earlier around sales and marketing. Every time someone's actually cracked that leadership visibility, for each function, there has been a multi billion dollar opportunity there, if not, a multi, you know multi multi billion dollar opportunity out there. So I don't think it's a overly anim facies that where we're going. But I think there's a lot of work to get from here to there. >> Yeah, I mean, I didn't ask you directly about the competition, I did ask you if there's a big whale, is there a big entity, you know, like a database, guys, is they want to target oracle, for example. And I looked around and I, I really didn't see it. It really does look like a Greenfield opportunity, which is absolutely enormous. I mean, I think I'm getting that right. >> Yeah, I think you're right on, and look at I think there are going to be more small players actually entering the market. Like I think whenever we look at new markets, and as they actually kind of build momentum, that always happens. And so of course, I you know, in that sense, I want competition to be here. But right now, I really don't focus on that. I think as for us, It's really about our product in the hands of our customers, how we make them successful. And then let's rinse and rinse and repeat that over and over again to more and more companies. >> Yeah, you don't have to compete you guys have to create. Andrew, great to have you on thanks so much for sharing your insights on your company, good luck with Jellyfish and come back anytime you know, in the future would love to track your progress and see how you're doing. >> Right Dave, thank you so much for having me here and I hope you, your family or team are staying healthy and all this and I look forward to next time. okay, and thank you for watching everybody this is Dave Vellante for theCUBE and startup insights. We'll see you next time. (upbeat music)

(upbeat music) >> From theCUBE studios in Palo Alto, in Boston, connecting with thought leaders all around the world. This is a Cube conversation. >> Hi everybody, this is Dave Vellante. And as you know, I've been running a CxO series in this COVID economy. And as we go into the post-isolation world, really want to focus and expand our scope and really look at startups. And of course, we're going to look at startups, let's follow the money. And I want to start with the investor. Mark Roberge is here. He's the managing director at Stage 2 capital. He's a professor at the Harvard Business School, former CRO over at HubSpot. Mark, great to see you. Thanks for coming on. >> Yeah, you bet, Dave. Thanks for having me. >> So I love that, you know... looking at your career a little bit, on your LinkedIn and following some of your videos, I love the fact that you did, and now you teach and you're also applying it with Stage 2 Capital. Tell us a little bit more about both of your career and Stage 2. >> Yeah, I mean, a lot of it's a bit serendipitous, especially last 10 years, but I've always had this learn, do, teach framework in my, in mind as I go through the decades of my career, you know, like you're probably like 80% learning in your twenties, early thirties and you know, 20% doing. Then, you know, I think my thirties was like leading the HubSpot sales team, a lot of doing, a little bit of teaching, you know, kind of hopping into different schools, et cetera, and also doing a lot of, some writing. And now like, I'm teaching it. I think investing kind of falls into that too, you know, where you've got this amazing opportunity to meet, the next generation of, of extraordinary entrepreneurs and engage with them. So yeah, that, that has been my career. You know, Dave, I've been a, passionate entrepreneur since 22 and then, the last one I did was HubSpot and that led to just an opportunity to build out one of the first sales teams in a complete inside environment, which opened up the doors for a data driven mindset and all this innovation that led to a book that led to recruitment on HBS's standpoint, to like come and teach that stuff, which was such a humbling honor to pursue. And that led to me a meeting my co-founder, Jay Po, of Stage 2 Capital, who was a customer to essentially start the first VC fund, running back by sales and marketing leaders, which was his vision. But when he proposed it to me, addressed a pretty sizeable void, that I saw, in the entrepreneur ecosystem that I thought could make a substantial impact to the success rate of startups. >> Great, I want to talk a little bit about how you guys compete and what's different there, but you know, I've read some of your work, looked at some of your videos, and we can bring that into the conversation. But I think you've got some real forward-thinking for example, on the, you know, the best path to the upper right. The upper right, being that, that xy-axis on growth and adoption, you know, do you go for hyper-growth or do you go for adoption? How you align sales and marketing, how you compensate salespeople. I think you've got some, some leading-edge thinking on that, that I'd love for you to bring into the conversation, but let's start with Stage 2. I mean, how do you compete with the big guys? What's different about Stage 2 Capital? >> Yeah, I mean, first and foremost, we're a bunch of sales and marketing and execs. I mean, our backing is, a hundred plus CROs, VPs of marketing, CMOs from, from the public companies. I mean, Dropbox, LinkedIn, Oracle, Salesforce, SurveyMonkey, Lyft, Asana, I mean, just pick a unicorn, we probably have some representation from it. So that's, a big part of how we compete, is most of the time, when a rocket ship startup is about to build a sales team, one of our LPs gets a call. And because of that, we get a call, right. And, and so there's, we're just deep in, in helping... So first off, assess the potential and risks of a startup in their current, go to market design, and then really, you know, stepping in, not just with capital, but a lot of know-how in terms of, you know, how to best develop this go-to-market for their particular context. So that's a big part of our differentiation. I don't think we've ever lost a deal that we tried to get into, you know, for that reason, just because we come in at the right stage, that's right for our value prop. I'd say Dave, the biggest, sort of difference, in our investing theme. And this really comes out of like, post HubSpot. In addition to teaching the HBS, I did parachute into a different startup every quarter, for one day, where you can kind of like assess their go-to-market, looking for, like, what is the underlying consistency of those series A businesses that become unicorns versus those that flatline. And if I, you know, I've now written like 50 pages on it, which I, you know, we can, we can highlight to the crew, but the underlying cliffnotes is really, the avoidance of a premature focus on top line revenue growth, and an acute focus early on, on customer attention. And, I think like, for those of you, who run in that early stage venture community these days, and especially in Silicon Valley, there's this like, triple, triple, double, double notion of, like year one, triple revenue, year two, triple revenue, year three, double revenue, year four, double revenue, it's kind of evolved to be like the holy grail of what your objectives should be. And I do think like there is a fraction of companies that are ready for that and a large amount of them that, should they pursue that path, will lead to failure. And, and so, we take a heavy lens toward world-class customer retention as a prerequisite, to any sort of triple, triple, double, double blitzscaling type model. >> So, let me ask you a couple of questions there. So it sounds like your LPs are heavily, not only heavily and financially invested, but also are very active. I mean, is that a, is that a fears thing? How active are the LPs in reality? I mean, they're busy people. They're they're software operators. >> Yeah. >> Do they really get involved in businesses? >> Absolutely. I mean, half of our deals that we did in fund one came from the LPs. So we get half of our funnel, comes from LPs. Okay. So it's always like source-pick-win-support. That's like, what basically a VC does. And our LPs are involved in every piece of that. Any deal that we do, we'll bring in four or five of our LPs to help us with diligence, where they have particular expertise in. So we did an insuretech company in Q4, one of our LPs runs insurance practice at Workday. And this particular play he's selling it to big insurance companies. He was extremely helpful, to understand that domain. Post investment, we always bring in four or five LPs to go deeper than I can on a particular topic. So one of our plays is about to stand up in account based marketing, you know, capability. So we brought in the CMO, a former CMO at Rapid7 and the CMO at Unisys, both of which have, stood in, stood up like, account based marketing practices, much more deeply, than I could. You know of course, we take the time to get to know our LPs and understand both their skills, and experiences as well as their willingness to help, We have Jay Simons, who's the President of Atlassian. He doesn't have like hours every quarter, he's running a $50 billion company, right? So we have Brian Halligan, the CEO of HubSpot, right? He's running a $10 billion company now. So, we just get deal flow from them and maybe like an event once or twice a year, versus I would say like 10 to 20% of our LPs are like that. I would say 60% of them are active operators who are like, "You know what? I just miss the early days, and if I could be active with one or two companies a quarter, I would love that." And I would say like a quarter of them are like semi-retired and they're like, they're choosing between helping our company and being on the boat or the golf course. >> Is this just kind of a new model? Do you see having a different philosophy where you want to have a higher success rate? I mean, of course everybody wants to have a, you know, bat a thousand. >> Yeah. >> But I wonder if you could address that. >> Yeah. I don't think it, I'm not advocating slower growth, but just healthier growth. And it's just like an extra, it's really not different than sort of the blitzscaling oriented San Francisco VC, okay? So, you know, I would say when we were doing startups in the nineties, early 2000s before The Lean Startup, we would have this idea and build it in a room for a year and then sell it in parallel, basically sell it everywhere and Eric Ries and The Lean Startup changed all that. Like he introduced MVPs and pivots and agile development and we quickly moved to, a model of like, yeah, when you have this idea, it's not like... You're really learning, keep the team small, keep the burn low, pivot, pivot, pivot, stay agile and find product-market fit. And once you do that, scale. I would say even like, West Coast blitzscaling oriented VCs, I agree with that. My only take is... We're not being scientifically rigorous, on that transition point. Go ask like 10 VCs or 10 entrepreneurs, what's product-market fit, and you'll get 10 different answers. And you'll get answers like when you have lots of sales, I just, profoundly disagree with that. I think, revenue in sales has very little to do with product-market fit. That's like, that's like message-market fit. Like selling ice to Eskimos. If I can sell ice to Eskimos, it doesn't mean that product-market fit. The Eskimos didn't need the ice. It just means I was good at like pitching, right? You know, other folks talk about like, having a workable product in a big market. It's just too qualitative. Right? So, that's all I'm advocating is, that, I think almost all entrepreneurs and investors agree, there's this incubation, rapid learning stage. And then there's this thing called product-market fit, where we switch to rapid scale. And all I'm advocating is like more scientist science and rigor, to understanding some sequences that need to be checked off. And a little bit more science and rigor on what is the optimal pace of scale. Because when it comes to scale, like pretty much 50 out of 50 times, when I talk to a series A company, they have like 15 employees, two sales reps, they got to like 2 million in revenue. They raise an 8 million-dollar round in series A, and they hired 12 salespeople the next month. You know, and Dave, you and your brother, who runs a large sales team, can really understand how that's going to failure almost all the time. (Dave mumbles) >> Like it's just... >> Yeah it's a killer. >> To be able to like absorb 10 reps in a month, being a 50, it's just like... Who even does all those interviews? Who onboards them? Who manages them? How do we feed them with demand? Like these are some of the things I just think, warrant more data and science to drive the decisions on when and how fast to scale. >> Mark, what is the key indicator then, of product-market fit? Is it adoption? Is it renewal rates? >> Yeah. It's retention in my opinion. Right? So, so the, the very simple framework that I require is you're ready to scale when you have product-market and go to market-fit. And let's be, extremely precise, and rigorous on the definitions. So, product-market fit for me, the best metric is retention. You know, that essentially means someone not only purchased your offering, but experienced your offering. And, after that experience decided to repurchase. Whether they buy more from you or they renew or whatever it is. Now, the problem with it is, in many, like in the world we live inside's, it's like, the retention rate of the customers we acquire this quarter is not evident for a year. Right, and we don't have a year to learn. We don't have a year to wait and see. So what we have to do is come up with a leading indicator to customer retention. And that's something that I just hope we see more entrepreneurs talking about, in their product market fit journey. And more investors asking about, is what is your lead indicator to customer retention? Cause when that gets checked off, then I believe you have product-market fit, okay? So, there's some documentation on some unicorns that have flirted with this. I think Silicon Valley calls it the aha moment. That's great. Just like what. So like Slack, an example, like, the format I like to use for the lead indicator of customer retention is P percent of customers, do E event, in T time, okay? So, it basically boils it down to those three variables, P E T. So if we bring that to life and humanize it, 70% of the customers, we sign up, this is Slack, 70% of the customers who sign up, send 2000 team messages in 30 days, if that happens, we have product-market fit. I like that a lot more, than getting to a million in revenue or like having a workable product in a big market. Dropbox, 85% of customers, share one file in one hour. HubSpot, I know this was the case, 75% of customers, use five or more of the 25 features in the platform, within 60 days. Okay? P percent, do E event, in T time. So, if we can just format that, and look at that through customer cohorts, we often get visibility into, into true product market-fit within weeks, if not like a month or two. And it's scientifically, data-driven in terms of his foundation. >> Love it. And then of course, you can align sales compensation, you know, with that retention. You've talked a lot about that, in some of your work. I want to get into some of the things that stage two is doing. You invest in SaaS companies. If I understand it correctly, it's not necessarily early stage. You're looking for companies that have sort of achieved some degree of revenue and now need help. It needs some operational help and scaling. Is that correct? >> Yeah. Yeah. So it's a little bit broader in size, as any sort of like B2B software, any software company that's scaling through a sales team. I mean, look at our backers and look at my background. That's, that's what we have experience in. So not really any consumer plays. And yeah, I mean, we're not, we have a couple product LPs. We have a couple of CFO type LPs. We have a couple like talent HR LPs, but most of us are go-to-market. So we don't, you know, there's awesome seed funds out there that help people set up their product and engineering team and go from zero to one in terms of the MVP and find product-market fit. Right? We like to come in right after that. So it's usually like between the seed and the A, usually the revenue is between half a million and 1.5 million. And of course we put an extraordinary premium on customer retention, okay? Whereas I think most of our peers put an extraordinary premium on top line revenue growth. We put an extraordinary premium on retention. So if I find a $700,000 business that, you know, has whatever 50, 70 customers, you know, depending on their ticket size, it has like North of 90% local retention. That's super exciting. Even if they're only growing like 60%, it's super exciting. >> What's a typical size of investments. Do you typically take board seats or not? >> Yeah. We typically put in like between like seven hundred K, one and a half million, in the first check and then have, larger amounts for follow on. So on the A and the B. We try not to take board's seats to be honest with you, but instead the board observers. It's a little bit selfish in terms of our funds scale. Like the general counsel from other venture capitalists is of course, like, the board seat is there for proper governance in terms of like, having some control over expenditures and acquisition conversations, et cetera, or decisions. But a lot of people who have had experience with boards know that they're very like easy and time efficient when the company is going well. And there are a ton of work when the company is not going well. And it really hurts the scale, especially on a smaller fund like us. So we do like to have board observers seats, and we go to most of the board meetings so that our voice is heard. But as long as there's another fund in there that, has, world-class track record in terms of, holding proper governance at the board level, we prefer to defer to them on that. >> All right, so the COVID lock down, hit really in earnest in March, of course, we all saw the Sequoia memo, The Black Swan memo. You were, I think it HubSpot, when, you remember the Rest In Peace Good Times memo, came out very sort of negative, put up all over the industry, you know, stop spending. But there was some other good advice in there. I don't mean to sort of, go too hard on that, but, it was generally a negative sentiment. What was your advice to your portfolio companies, when COVID hit, what were you telling them? >> Yeah, I summarized this in our lead a blog article. We kicked off our blog, which is partially related to COVID in April, which has kind of summarize these tips. So yes, you are correct, Dave. I was running sales at HubSpot in '08 when we had last sort of major economic, destabilization. And I was freaking out, you know (laughs briefly) at the time we were still young, like 20, 30 reps and numbers to chase. And... I was, actually, after that year, looking back, we are very fortunate that we had a value prop that was very recession-proof. We were selling to the small business community, who at the time was cutting everything except new ways to generate sales. And we happen to have the answer to that and it happened to work, right? So it showed me that, there's different levels of being recession proof. And we accelerated the raise of our second fund for stage two with the anticipation that there would be a recession, which, you know, in the venture world, some of the best things you could do is close a fund and then go into a recession, because, there's more deals out there. The valuations are lower and it's much easier to understand, nice to have versus must have value props. So, the common theme I saw in talking to my peers who looked back in the '01 crisis, as well as the '08 crisis, a year later was not making a bolder decision to reorient their company in the current times. And usually on the go-to-market, that's two factors, the ICP who you're selling to, ideal customer profile and the CVP, what your message is, what's your customer value prop. And that was really, in addition to just stabilizing cash positions and putting some plans in there. That was the biggest thing we pushed our portfolio on was, almost like going through the exercise, like it's so hard as a human, to have put like nine months into a significant investment leading up to COVID and now the outcome of that investment is no longer relevant. And it's so hard to let that go. You know what I mean? >> Yeah. >> But you have to, you have to. And now it's everything from like, you spent two years learning how to sell to this one persona. And now that persona is like, gyms, retail and travel companies. Like you've got to let that go. (chuckle simultaneously) You know what I mean? Like, and, you know, it's just like... So that's really what we had to push folks on was just, you know, talking to founders and basically saying this weekend, get into a great headspace and like, pretend like you were parachuted into your company as a fresh CEO today. And look around and appreciate the world and what it is. What is this world? What are the buyers talking about? Which markets are hot, which markets are not, look at the assets that you have, look at your product, look at your staff, look at your partners, look at your customer base, and come up with a strategy from the ground up based on that. And forget about everything you've done in the last year. Right? And so, that's really what we pushed hard on. And in some cases, people just like jumped right on it. It was awesome. We had a residential real estate company that within two weeks, stood up a virtual open house module that sold like hotcakes. >> Yeah. >> That was fantastic execution. And we had other folks that we had to have like three meetings with to push them deep enough, to go more boldly. But that, was really the underlying pattern that I saw in past, recessions and something I pushed the portfolio on, is just being very bold on your pivots. >> Right? So I wanted to ask you how your portfolio companies are doing. I'm imagining you saw some looked at this opportunity as a tailwind. >> Yeah. >> You mentioned the virtual, open house, a saw that maybe were exposed, had, revenue exposure to hard-hit industries and others kind of in the middle. How are your portfolio companies doing? >> Yes, strong. I'm trying to figure out, like, of course I'm going to say that, but I'm trying to figure out like how to provide quant, to just demonstrate that. We were fortunate that we had no one, and this was just dumb luck. I mean, we had no one exclusively selling to like travel, or, restaurants or something. That's just bad luck if you were, and we're fortunate that we got a little lucky there, We put a big premium, obviously we had put a big premium on customer retention. And that, we always looked at that through our recession proof lens at all our investments. So I think that helped, but yeah, I mean, we've had, first off, we made one investment post COVID. That was the last investment on our first fund and that particular company, March, April, May, their results were 20% higher than any month in history. Those are the types of deals we're seeing now is like, you literally find some deals that are accelerating since COVID and you really just have to assess if it's permanent or temporary, but that one was exciting. We have a telemedicine company that's just like, really accelerating post COVID, again, luck, you know, in terms of just their alignment with the new world we're living in. And then, jeez! I mean, we've had, I think four term sheets, for markups in our portfolio since March. So I think that's a good sign. You know, we only made 11 investments and four of them, either have verbal or submitted term sheets on markups. So again, I feel like the portfolio is doing quite well, and I'm just trying to provide some quantitative measures. So it doesn't feel like a political answer. (Mark chuckles) >> Well, thank you for that, but now, how have you, or have you changed your sort of your thesis post COVID? Do you feel like your... >> Sure. >> Your approach was sort of geared towards, you know, this... >> Yeah. >> Post COVID environment? But what changes have you made. >> A little bit, like, I think in any bull market, generally speaking, there's just going to be a lot of like triple, triple, double, double blitzscaling, huge focus on top-line revenue growth. And in any down market, there's going to be a lot of focus on customer retention unit economics. Now we've always invested in the latter, so that doesn't change much. There's a couple of things that have changed. Number one, we do look for acceleration post COVID. Now, that obviously we were not, we weren't... That lens didn't exist pre-COVID, So in addition to like great retention, selling through a sales team, around the half million to a million revenue, we want to see acceleration since COVID and we'll do diligence to understand if that's a permanent, or a temporary advantage. I would say like... Markets like San Francisco, I think become more attractive in post COVID. There's just like, San Francisco has some magic happening there's some VC funds that avoid it, cause it's too expensive. There's some VC funds that only invest in San Francisco, because there's magic happening. We've always just been, you know... we have two portfolio companies there that have done well. Like we look at it and if it's too expensive, we have to avoid it. But we do agree that there's magic happening. I did look at a company last week. (chuckles inaudibly) So Dave, there are 300K in revenue, and their last valuation is 300 million. (both chuckle) >> Okay, so why is San Francisco more attractive, Mark? >> Well, I mean and those happened in Boston too. >> We looked at... (Mark speaks inaudibly) >> I thought you were going to tell me the valuations were down. (Dave speaks inaudibly) >> Here's the deal all right, sometimes they do, sometimes they don't and this is one, but in general, I think like they have come down. And honestly, the other thing that's happened is good entrepreneurs that weren't raising are now raising. Okay? So, a market like that I think becomes more attractive. The other thing that I think that happens is your sort of following strategies different. Okay so, there is some statistical evidence that, you know, obviously we're coming out of a bear market, a bullish market in, in both the public and the private equities. And there's been a lot of talk about valuations in the private sector is just outrageous. And so, you know, we're fortunate that we come in at this like post seed, pre-A, where it's not as impacted. It is, but not as or hasn't been, but because there's so many more multibillion-dollar funds that have to deploy 30 to 50 million per investment, there's a lot of heating up that's happened at that stage. Okay? And so pre COVID, we would have taken advantage of that by taking either all or some of our money off the table, in these following growth rounds. You know, as an example, we had a company that we made an investment with around 30 million evaluation and 18 months later, they had a term sheet for 500. So that's a pretty good return in 18 months. And you know, that's an expensive, you know, so that that's like, wow, you know, we probably, even though we're super bullish on the company, we may want to take off a 2X exposition... >> Yeah. >> And take advantage of the secondaries. And the other thing that happens here, as you pointed out, Dave is like, risk is not, it doesn't become de-risk with later rounds. Like these big billion dollar funds come in, they put pressure on very aggressive strategic moves that sometimes kills companies and completely outside of our control. So it's not that we're not bullish on the company, it's just that there's new sets of risks that are outside of the scope of our work. And so, so that that's probably like a less, a lesser opportunity post COVID and we have to think longer term and have more patient capital, as we navigate the next year or so of the economy. >> Yeah, so we've got to wrap, but I want to better understand the relationship between the public markets and you've seen the NASDAQ up, which is just unbelievable when you look at what's happening in main street, and the relationship between the public markets and the private markets, are you saying, they're sort of tracking, but not really identical. I mean, what's the relationship. >> Okay, there's a hundred, there's thousands of people that are better at that than me. Like the kind of like anecdotal thoughts that I, or the anecdotal narrative that I've heard in past recessions and actually saw too, was the private market, when the public market dropped, it took nine months roughly for the private market to correct. Okay, so there was a lag. And so there's, some arguments that, that would happen here, but this is just a weird situation, right? Of like the market, even though we're going through societal crazy uncertainty, turmoil and, and tremendous tragedy, the markets did drop, but they're pretty hot right now, specifically in tech. And so there's a number of schools of thoughts there that like some people claim that tech is like the utilities companies of the eighties, where it's just a necessity and it's always going to be there regardless of the economy. Some people argue that what's happened with COVID and the remote workplace have made, you know, accelerated the adoption of tech, the inevitable adoption, and others could argue that like, you know, the worst is still the come. >> Yeah. And of course, you've got The Fed injecting so much liquidity into the system, low interest rates, Mark, last question. Give me a pro tip for entrepreneurs. (Mark Sighs) >> I would say, like, we've talked a lot about, this methodology with, you know, customer retention, really focusing there, align everything there as opposed to top line revenue growth initially. I think that the extension I do at this point is, do your diligence on your investors, and what their thoughts are on your future growth plans to see if they're aligned. Cause that, that becomes like, I think a lot of entrepreneurs, when they dig into this work, they do want to operate around it. But that becomes that much harder when you have investors that think a different way. So I would just, you know, just always keep in mind that, you know, I know it's so hard to raise money, but you know, do the diligence on your investors to understand, what they'd like to see in the next two years and how it's aligned with your own vision. >> Mark is really great having you on. I'd love to have you back and as this thing progresses, and see how it all shakes out. It really a pleasure. Thanks for coming on. >> No, thanks, Dave. I appreciate you having me on. >> And thank you everybody for watching. This is Dave Vellante for The Cube. We'll see you next time. (music plays)