>>Good morning friends. Live from Las Vegas. It's the Cube Day four of our coverage of AWS. Reinvent continues. Lisa Martin here with Dave Valante. You >>Can tell it's day four. Yeah. >>You can tell, you >>Get punchy. >>Did you? Yes. Did you know that the Vegas rodeo is coming into town? I'm kind of bummed down, leaving tonight. >>Really? You rodeo >>Fan this weekend? No, but to see a bunch of cowboys in Vegas, >>I'd like to see the Raiders. I'd like to see the Raiders get tickets. >>Yeah. And the hockey team. Yeah. We have had an amazing event, Dave. The cubes. 10th year covering reinvent 11th. Reinvent >>Our 10th year here. Yeah. Yes. Yeah. I mean we covered remotely in during Covid, but >>Yes, yes, yes. Awesome content. Anything jump out at you that we really, we, we love talking to aws, the ecosystem. We got a customer next. Anything jump out at you that's really a kind of a key takeaway? >>Big story. The majority of aws, you know, I mean people ask me what's different under a Adam than under Andy. And I'm like, really? It's the maturity of AWS is what's different, you know, ecosystem, connecting the dots, moving towards solutions, you know, that's, that's the big thing. And it's, you know, in a way it's kind of boring relative to other reinvents, which are like, oh wow, oh my god, they announced outposts. So you don't see anything like that. It's more taking the platform to the next level, which is a good >>Thing. The next level it is a good thing. Speaking of next level, we have a couple of next level guests from T-Mobile joining us. We're gonna be talking through their customers story, their business transformation with aws. Glenn Curing joins us, the director product and technology. And Nick Chris, senior manager, product and technology guys. Welcome. Great to have you on brand. You're on T-Mobile brand. I love it. >>Yeah, >>I mean we are always T-Mobile. >>I love it. So, so everyone knows T-Mobile Blend, you guys are in the digital commerce domain. Talk to us about what that is, what functions that delivers for T-Mobile. Yeah, >>So the digital commerce domain operates and runs a platform called the Digital commerce platform. What this essentially does, it's a set of APIs that are headless that power the shopping experiences. When you talk about shopping experiences at T-Mobile, a customer comes to either a T-Mobile website or goes to a store. And what they do is they start with the discovery process of a phone. They take it through the process, they decide to purchase the phone day at, at the phone to cart, and then eventually they decide to, you know, basically pull the trigger and, and buy the phone at, at which point they submit the order. So that whole experience, essentially from start to finish is powered by the digital commerce platform. Just this year we have processed well over three and a half million orders amounting to a billion and a half dollars worth of business for T-Mobile. >>Wow. Big outcomes. Nick, talk about the before stage, obviously the, the customer experience is absolutely critical because if, if it goes awry, people churn. We know that and nobody wants, you know, brand reputation is is at stake. Yep. Talk about some of the challenges before that you guys faced and how did you work with AWS and part its partner ecosystem to address those challenges? >>Sure. Yeah. So actually before I started working with Glen on the commerce domain, I was part of T-Mobile's cloud team. So we were the team that kind of brought in AWS and commerce platform was really the first tier one system to go a hundred percent cloud native. And so for us it was very much a learning experience and a journey to learn how to operate on the cloud and which was fundamentally different from how we were doing things in the old on-prem days. When >>You talk about headless APIs, you talk, I dunno if you saw Warren a Vogel's keynote this morning, but you're talking about loosely coupled, a loosely coupled system that you can evolve without ripping out the whole system or without bringing the whole system down. Can you explain that in a little bit more >>Detail? Absolutely. So the concept of headless API exactly opens up that possibility. What it allows us to do is to build and operator platform that runs sort of loosely coupled from the user experiences. So when you think about this from a simplistic standpoint, you have a set of APIs that are headless and you've got the website that connects to it, the retail store applications that connect to it, as well as the customer care applications that connect to it. And essentially what that does is it allows us to basically operate all these platforms without being sort of tightly coupled to >>Each other. Yeah, he was talking about this morning when, when AWS announced s3, you know, there was just a handful of services maybe at just two or three. I think now there's 200 and you know, it's never gone down, it's never been, you know, replaced essentially. And so, you know, the whole thing was it's an asynchronous system that's loosely coupled and then you create that illusion of synchronicity for the customer. >>Exactly. >>Which was, I thought, you know, really well described, but maybe you guys could talk about what the genesis was for this system. Take us kind of to the, from the before or after, you know, the classic as as was and the, and as is. Did you talk about that? >>Yeah, I can start and then hand it off to Nick for some more details. So we started this journey back in 2016 and at that point T-Mobile had seven or eight different commerce platforms. Obviously you can think about the complexity involved in running and operating platforms. We've all talked about T-Mobile being the uncarrier. It's a brand that we have basically popularized in the telco industry. We would come out with these massive uncarrier moves and every time that announcement was made, teams have to scramble because you've got seven systems, seven teams, every single system needs to be updated, right? So that's where we started when we kicked off this transformational journey over time, essentially we have brought it down to one platform that supports all these experiences and what that allows us to do is not only time to market gets reduced immensely, but it also allows us to basically reduce our operational cost. Cuz we don't have to have teams running seven, eight systems. It's just one system with one team that can focus on making it a world class, you know, platform. >>Yeah, I think one of the strategies that definitely paid off for us, cuz going all the way back to the beginning, our little platform was powering just a tiny little corner of the, of the webspace, right? But even in those days we approached it from we're gonna build functions in a way that is sort of agnostic to what the experience is gonna be. So over time as we would build a capability that one particular channel needed primary, we were still thinking about all the other channels that needed it. So now over a few years that investment pays off and you have basically the same capabilities working in the same way across all the channels. >>When did the journey start? >>2016. >>2016, yeah. It's been, it's been six years. >>What are some of the game changers in, in this business transformation that you would say these are some of the things that really ignited our transformation? >>Yeah, there's particularly one thing that we feel pretty proud about, which is the fact that we now operate what we call active active stacks. And what that means is you've got a single stack of the eCommerce platform start to finish that can run in an independent manner, but we can also start adding additional stacks that are basically loosely coupled from each other but can, but can run to support the business. What that basically enables is it allows us to run in active active mode, which itself is a big deal from a system uptime perspective. It really changes the game. It allows us to push releases without worrying about any kind of downtime. We've done canary releases, we are in the middle of retail season and we can introduce changes without worrying about it. And more importantly, I think what it has also allowed us to do is essentially practice disaster recovery while doing a release. Cuz that's exactly what we do is every time we do a release we are switching between these separate stacks and essentially are practicing our DR strategy. >>So you do this, it's, it's you separate across regions I presume? Yes. Is that right? Yes. This was really interesting conversation because as you well know in the on-prem world, you never tested that disaster recovery was too risky because you're afraid you're gonna take your whole business down and you're essentially saying that the testing is fundamental to the implementation. >>Absolutely. >>It, it is the thing that you do for every release. So you know, at least every week or so you are doing this and you know, in the old world, the active passive world on paper you had a bunch of capabilities and in in incidents that are even less than say a full disaster recovery scenario, you would end up making the choice not to use that capability because there was too much complexity or risk or problem. When we put this in place. Now if I, I tell people everything we do got easier after that. >>Is it a challenge for you or how do you deal with the challenge? Correct me if it's not a, a challenge that sometimes Amazon services are not available in both regions. I think for instance, the observability thing that they just announced this week is it's not cross region or maybe I'm getting that wrong, but there are services where, you know, you might not be able to do data sharing across region. How do you manage that? Or maybe there's different, you know, levels of certifications. How do you manage that discontinuity or is that not an issue for you? >>Yeah, I mean it, it is certainly a concern and so the stacks, like Glen said, they are largely decoupled and that what that means is practically every component and there's a lot of lot of components in there. I have redundancy from an availability zone point of view. But then where the real magic happens is when you come in as a user to the stack, we're gonna initially kind of lock you on one stack. And then the key thing that we do is we, we understand the difference between what, what we would call the critical data. So think of like your shopping carts and then contextual data that we can relatively easily reload if we need to. And so that critical data is constantly in an async fashion. So it's not interrupting your performance, being broadcast out to a place where we can recover it if we need to, if we need to send you to another stack and then we call that dehydration. And if you end up getting bumped to a new stack, we rehydrate you on that stack and reload that, that contextual data. So to make that whole thing happen, we rely on something we call the global cart store and that's basically powered by Dynamo. So Dynamo is highly, highly reliable and multi >>Reason. So, and, and presume you're doing some form of server list for the stateless stuff and, and maybe taking control of the run time for the stateful things you, are you leaning into to servers and lambda or Not yet cuz you want control over the, the, the EC two and the memory configs. What, what's, I mean, I know we're going inside the plumbing a little bit, but it's kind of fun. >>That's always fun. You >>Went Yeah, and, and it has been a journey. Back in 2016 when we started, we were all on EC twos and across, you know, over the last three or four years we have kind of gone through that journey where we went from easy two to, to containers and we are at some point we'll get to where we will be serverless, we've got a few functions running. But you know, in that journey, I think when you look at the full end of the spectrum, we are somewhere towards the, the process of sort of going from, you know, containers to, to serverless. >>Yeah. So today your team is setting up the containers, they're fencing 'em off, fencing off the app and doing all that sort of sort of semi heavy lifting. Yeah. How do you deal with the, you know, this is one of the things Lisa, you and I were talking about is the skill sets. We always talk about this. What's that? What's your team look like and what are the skill sets that you've got that you're deploying? >>Yeah, I mean, as you can imagine, it's a challenge and it's a, a highly specialized skill set that you need. And you talk about cloud, you know, I, I tell developers when we bring new folks in, in the old days, you could just be like really good at Java and study that for and be good at that for decades. But in the cloud world, you have to be wide in, in your breadth. And so you have to understand those 200 services, right? And so one of the things that really has helped us is we've had a partner. So UST Global is a digital services company and they've really kind of been on the journey up the same timeline that we were. And I had worked with them on the cloud team, you know, before I came to commerce. And when I came to, to the commerce team, we were really struggling, especially from that operational perspective. >>The, the team was just not adapting to that new cloud reality. They were used to the on-prem world, but we brought these folks in because not only were they really able to understand the stuff, but they had built a lot of the platforms that we were gonna be leveraging for commerce with us on the cloud team. So for example, we have built, T-Mobile operates our own customized Kubernetes platform. We've done some stuff for serverless development, C I C D, cloud security. And so not only did these folks have the right skill sets, but they knew how we were approaching it from a T-mobile cloud perspective. And so it's kind of kind of fun to see, you know, when they came on board with this journey with us, we were both, both companies were relatively new and, and learning. Now I look and, you know, I I think that they're like a, a platinum sponsor these days here of aws and so it's kind of cool to see how we've all grown together, >>A lot of evolution, a lot of maturation. Glen, I wanna know from you when we're almost out of time here, but tell me the what the digital commerce domain, you kind of talked about this in the beginning, but I wanna know what's the value in it for me as a customer? All of this under the hood plumbing? Yeah, the maturation, the transformation. How does it benefit mean? >>Great question. So as a customer, all they care about is coming into, going to the website, walking into a store, and without spending too much time completed that transaction and walkout, they don't care about what's under the hood, right? So this transformational journey from, you know, like I talked about, we started with easy twos back in the day. It was what we call the wild west in the, on a cloud native platform to where we have reached today. You know, the journey we have collectively traversed with the USD has allowed us to basically build a system that allows a customer to walk into a store and not spend a whole hour dealing with a sales rep that's trying to sell them things. They can walk in and out quickly, they go to the website, literally within a couple minutes they can complete the transaction and leave. That's what customers want. It is. And that has really sort of helped us when you think about T-Mobile and the fact that we are now poised to be a leader in the US in telco at this whole concept of systems that really empower the customers to quickly complete their transaction has been one of the key components of allowing us to kind of make that growth. Right. So >>Right. And a big driver of revenue. >>Exactly. >>I have one final question for each of you. We're making a Instagram reel, so think about if you had 30 seconds to describe T-Mobile as a technology company that sells phones or a technology company that delights people, what, what would you say if you had a billboard, what would it say about that? Glen, what do you think? >>So T-Mobile, from a technology company perspective, the, the whole purpose of setting up T-mobile's, you know, shopping experience is about bringing customers in, surprising and delighting them with the frictionless shopping experiences that basically allow them to come in and complete the transaction and move on with their lives. It's not about keeping them in the store for too long when they don't want to do it. And essentially the idea is to just basically surprise and delight our customers. >>Perfect. Nick, what would you say, what's your billboard about T-Mobile as a technology company that's delivering great services to its customers? >>Yeah, I think, you know, Glen really covered it well. What I would just add to that is I think the way that we are approaching it these days, really starting from that 2016 period is we like to say we don't think of ourselves as a telco company anymore. We think of ourselves as a technology company that happens to do telco among other things, right? And so we've approached this from a point of view of we're here to provide the best possible experience we can to our customers and we take it personally when, when we don't reach that high bar. And so what we've done in the last few years as a transformation is really given us the toolbox that we need to be able to meet that promise. >>Awesome. Guys, it's been a pleasure having you on the program, talking about the transformation of T-Mobile. Great to hear what you're doing with aws, the maturation, and we look forward to having you back on to see what's next. Thank you. >>Awesome. Thank you so much. >>All right, for our guests and Dave Ante, I'm Lisa Martin, you watching The Cube, the leader in live enterprise and emerging tech coverage.

>> Live from Las Vegas. It's the Cube covering Adobe Summit twenty nineteen brought to you by extension for interactive. >> Hey, welcome back it when Cube live coverage here in Las Vegas are doubly. Summit twenty nineteen. I'm John Murray with Jeffrey, my coz this week. Two days of wall to wall coverage. Our next guest is Glenn Hartman. North America lied for a censure Interactive. Thanks for joining us. Hey, thanks for having me. Great beer. So you guys are doing some great stuff around the creativity, peace, doing great customer experience. Implement taste. We have a great walk through from a lot of the folks from your organization. You're designing up ideas and products, delivering them and then operating them. Nice model. Yeah, thanks. Is that your business model that has network, What century? Interactives business model. I mean center and >> really about one thing. It's about creating, delivering and running the best experiences on the planet. We help our clients do that for their own customers. And when >> we talk about experience, a lot of people have different definitions for that, especially at the conference here. It's not necessarily an experience of a website or a mobile app or u X is people use it. It's really anyway, a brand engages with a customer. It's not just marketing. Either it could be sales or customer service loyalty. Anything anyway. A brand promise can be delivered. Teo to customers. One of things that I've noticed with you guys is that it's a talk track and thought leaders around a new creative and new creativity. And Toby honestly with software that they haven't now the cloud you're seeing in the marketplace. We saw this at Sundance two years ago. A new kind of creative is, um, organically coming into the marketplace with more channels to direct to consumer, whether it be to be or be to see you and now have new kinds of mechanisms to take product, whether it's APS or content or movies, You start to see this democratization really starting to happen. How is that changing? How you guys helped us is because now they now have new capability. They can tell their story in a different way. They have access to new kinds of channels that weren't there before. How is that changing the business, in your opinion, die in a profound way? So I mean, everybody knows that marking is inextricably linked >> to technology and data. Everybody knows that. But when >> we are thinking about the new, creative and own ways that you can tell stories and create experiences. We look at experience very differently. I mentioned before that it was all about all the different touch points in the ways people interact with the brand. But when we look at experience, we call it the Big E. And the biggie actually stands for empathy. And it's understanding how to define what a brand experience means to that customer and defining success in terms of that customer. And you know, Jeff and John, you guys air not to find by your data set or what you bought last week. You could be very different types of people in different situations and understanding ways to empathize with you in the moment and having experiences change in the moment and having creative play, a part of that and data play. A part of that is the big hot. It's a new way of looking at things, and the last part of that, too. And the big is about emotion. So when you have a big brand that has some emotional connection, you know you love this brand for an automotive or you love this hotel chain or there's some brand a connection you have. How do you have that connection flow through every touch? Point and data and technology can enable that. But it's really empathy and emotion. That's the driver. How do you get empathy and creativity to work together? Because you now have an accelerant with data you mentioned getting to know people's empathizing with them in the moment it's contextual. I could be having a great day or a bad day or driving my kids to school of whatever's going on with me. It certainly there might be some data out. How do you get the creativity and the empathy to work together in your mind? You see that appointment? That's the nice parties. More than ever, we have different data sets that can help us do that. Just give you a couple of examples. Um, instead of understanding how to market >> to someone so they'LL buy the next product on basing. That may be on their demographic, maybe basing it on their preferences. You've hear all these terms and marketing for years, and you can understand what they bought. >> Instead of understanding that, why don't we try and look and use data which you could easily do today to understand why they bought something so it could be something as simple as like a I don't know, Cpg example. Maybe have shampoo and you could say, Well, they bought these kinds of things before, So maybe they'LL buy that champion. But if they know that, you know, maybe Jeff is equal friendly are John. Maybe you're more into things that maybe you buy that shampoo because you care about animals and they know they don't test them on animals. Or maybe it's something more about experience that that particular shampoo won't make your daughter cry when you ship your her hair. And it helps that experience that that's the reason why it actually helps me. You're empathizing in the moment with something that is meaningful that you care >> about. It's not about a better deal or >> better price or some kind of feature. It's something actually about you more meaningful, much very meaningful interaction. Data set is that because there's no data before, is it more? There's more signals, potentially to get exposed to that, because that's a hard data points to get. I mean, to find the why is a holistic kind of perspective. It's true, but I mean, I think it's more of a mind set. The date is there, but the mind set has been different. Over time, people were looking to technology every buzz word in the world use big data and personalization. Aye, aye. And then now it's a I and machine learning and like, Well, that's great. And they're all wonderful enablers, as I said, but it has to be driven by empathy first. So it all starts. >> I mean, we've been saying this forever customer, centrist city and a customer >> concerts. But really, I mean, for real. If you start to use those data sets, aunt have the mindset has to be a c M. O. R. A brand manager is someone who actually it's advocating for the customer, and they're willing to say, No, I don't need big data. I don't need all the data. I need this gold nugget part so I can speak to John. >> It's interesting plate, as you say, the emotional part of the biggie. Also, I think it is the old Coke commercial, right raw one world together, and we all cry and there's some great McDonald's commercials, right when you talk about you think beyond that to the to the empathy. I can't help but think of kind of the whole purpose. Purpose driven, mission driven companies. You know, kids coming out of college want to work for mission driven, cos we heard it over and over. And the key notes, you know, we're not a product company, not even really a service company. But we're committed to two, an ideal to the mission. Beep. Be partners with us, be our customer. Let's have a relationship that goes so much deeper and longer than any particular transaction is that kind of that tied it, that that is a part of it? >> Absolutely. Now, the interesting thing of what you said is that people are >> tied to a purpose and maybe something that's meaningful in a broad sense, absolutely. And that's a wonderful place to start. And you can start to align products and services in that way, >> the way he talked about, like, shampoo and, you know, animal testing, right? Well, >> it's a good one, but the next one is really getting a little bit Mohr down to you. So I think all that is great, but really understand what you need in the moment, because what happens is it. Some of those things may change if you are shopping at a grocery store every >> Saturday for your family and you're used to doing down your attitude. There might be different then when >> you're shopping, when your kid is sick and you got pulled out of work and you got to get there to get the prescription, you're into speed and your stress in the moment they're versus. Maybe >> on Saturday you're like all try some new coupons and try some new things and go by. The little tasting >> station is actually behaviors that you want to understand in the moment. That is a big part of that as well. But the key thing to hear is, let's think of this when you deal with empathy. It's not just getting to know all those things. Even if it gets to that level, it's actually changing the way marketers think about talking to communicating and relating to customers even the language that they use. I mean, think of it >> today. I mean, >> still, people use marketers are their marketing to people. It's, uh, that's acquire customers. Let's convert wind developers over good. We don't win their helpers over. And what was the last time you guys were real excited about getting converted? Okay, it's not a fun experience, right? So if you even changed, I might send you say, Let's market with someone or let's let's help them. You actually create experiences that are useful and helpful not about conversion and not a business metric. But success is defined by the customer. How are you guys playing this? Because this is really kind of ties on multiple threads. I mean, the whole nother community angle to people belong to communities in context to their life. And they engage. And when they engage his emotional connection to a group of this and some cohorts is the worker. Thank you. Okay, groups. But they're friends and colleagues, or whatever could be you guys were. The point is, with customers, take us through a use case day in the life of empathy, deployed into how you guys do business with the big Branson and one of the success. How do you make it happen? What's the engagement look like? How does someone do this? So they just wake up one morning, say Okay, I'm gonna have more empathy. They also call you guys up. What happens? Like what? Take us through What? Certain evidence was made in the slide. I could give you >> a little bit of it. An example of how stars we've been talking a little bit more dramatically about sick kids and testing on pets and animals and things like that. Not testing the pets. Can you imagine? I'd really be >> horrible. But single graphic of the users individual personal things is hard, right? So? Well, I mean, the whole point of this is that when you really get into the mechanics of how this works, I'll give you an >> example. It's a little bit less dramatic. OK, so it's a telecommunications company. Telco company. That's selling. You guys know what? Triple play? Yeah. Okay, So you have It's >> a cable and a phone, right? All it's it's like a commodity product writers. There's no emotional thing necessarily. But in that game, if you can just optimize certain parts of the journey, you can make a big difference, right so way got a benign request from a marketer. Teo say, Listen, we do a lot of paid search. Can you help us with this one product? Just if you move it even like one percent, it would be significant to the company. But when? So okay, we'LL do that, >> and it worked out. We go in and help them do their >> search. But because we're thinking about experience in a broader sense for so well, let's let's do it more. Let's make them be able to transact or engage in multiple ways. Well, you could. You could sign up for the service to email. Or maybe there's click to call or click to chat. Or you could even walk into a branch and do it there. Maybe through the call centre, right? It's what's all that's working together on the channel, though fun words you want easy. You're leveraging >> different technologies to do it and people, >> so the way this worked was you were coming in through search and then eventually a lot of them were converting in the call center. So it was all working. And you think that's great? Well, it wasn't great to the company at all. They were very upset. The people that we're buying the media, we're really bummed because they couldn't get the attribution of the credit for the thing was in the call center. So they came over the great idea. They said, OK, take the phone number off and take the the click to call off and will force the >> customer to convert in. Our channel, of >> course, is a brilliant company with great people and rational thinking prevailed and they didn't do that. But they said, Well, what do we do? I said, Well, you're going to eat and multi attribution model to be able to help you do that, Okay, but that's not enough because you also need a new sales incentive and commission structure inside the call center because those people are getting pit on that. But since it's such a low commodity product, that's not gonna work to change. That's that's a new sales kind of thing, then you wait. They can't talk for another three seconds to that person because you'LL bring the margins on. You got to get him back. All the turtleneck just screwed up. That's right. So there's a new business process is now a new operating model whose skills to get him back into the general. So all of a sudden it's benign. Request from a marketing team taken, you optimize my paint search becomes new business transformation. Okay, now, because that brand manager had the guts to say I'm going to advocate for this customer. This customer wants to come in through this channel. They want to convert over here, and we're gonna actually change the operating model, the sales structure to call the sales lead. I had to call the CFO on the CEO, and we're gonna make this happen, Gonna change the way we do business on behalf of that customer. That is weight world, I could tell you we see this all the time and marketers all the time that there's so married to their website analytics funnel that that's all about who gets credit coded earls and the customer experience is a brutal. It's like I'm not here on other sites are all over the place. I don't really need to go the site every day, right? Somebody only go there when Otto and the thing we were talking about before. If you're grocery shopping >> and Europe's have set on Tuesday and you want to get off >> for your kid at versus the nice leisurely thing we talked about on the weekend, there's a whole nother set of outcomes in Cape Yas you have to deal with. If you went into that supermarket on Tuesday and they figured out a way to get you in and out fast and just get those two or three autumns, you need it for your kid. That's a failed trip. According to the grocery industry. You need to be in there longer they want, so you stay in the back. That's right. Totally. That breaks the whole model, but it's wonderful for you. You'LL shop there forever because of that experience, what you're getting at the Morrises air, changing the business models of cos that's the bottom line. You're at the center. That could be a driver for the transformation. That's it. Empathy. Is the driver Absolutely no. You need to have the emotional connection to all that stuff to help also internally see emos. I don't need to just be relevant and customers that need to be relevant to the enterprise. They need to be relevant to the CEO, Doc in seconds and hear the screaming and kicking and screaming right now. Glenn, that's great. But man, that's a heavy lift way could do it. How do you How do you get it? Because now I can see a cultural reaction. The antibodies will come out and attack that notion because it's scary because now, like, whoa, yeah, well, I mean, it is hard, but the good news is, is that we see, even at this conference, and a lot of our clients are coming over to do that this incremental ways to get there. But I'll make it simple. So the advantages are we said this new technology, new data that allow you to do some of this stuff right? That's great. And you can see a lot of them consolidating, right? A lot of the stacks all now have content and analytics and commerce and all that, and in this nice ways that they come together and that consolidation can help, and there's other ones that can handle different data sets, and that helps to his automation. And but the thing is that what people miss is one of the ways to accelerate this is add a human centered approach to how you actually create the experience internally. And what I mean is, it's not enough to consolidate the data and figure out that Gold nugget and not enough to do with technology have to do with humans. It's a human centered approach, so we're bringing in integrated teams of humans that are pulling all the stuff together. It's someone who understands strategy. Maybe someone understands creative when it hit me in the club, basically their prime in the pump getting it. But they will sit together. They sent to get the analytics. People sit next to the creative people. It's in next you people. They work on it as they designed the experience. You don't do a strategy project and then do a A road map and then do an R P for technology enabled Waterfall does not work, but it's beyond even waterfall versus algebra. This is actually taking humans and consolidate that thinking New skill sets at the center in like an incubator way to do pilots to do prototyping to do things. If you want to create that new experience that we're talking about in any of these cases, you gotta hand CMO some kind of thing that can bring to the team and say, Look, here's an app that would enable this or he has a pilot We could try without boiling the ocean toe, actually create an experience that would move the needle or whatever. Lame corporate analogy. Just make more money and get some decent results in Get a beachhead, just small eatery through it. Glenn. Great Insights. This is a great time. We'd love to get you back on the Cuban girl down, and it's kind of design thinking, combined with execution on the front lines with customers. Center the value proposition. Great conversation before we and just give a quick plug for the business. What's going on with a sensor Interactive? How's the business going? What are your goals? How many people are working there? What's the geography is looked like? Give us enough. Thanks for asking So essential Interactive is enjoying its third year as being listed by at H magazine, is the largest digital agency in the world on the fastest growing Wei have coverage and a truly integrated global delivery model that hits every part of the of every market. And we're so excited. Tio have this growth because it's a way to show that the market is truly interested and being experienced lead and the way we're defining experience. We're seeing more and more clients moving from some of these incremental changes to really >> try to put the customer at the center of what they're doing. And, you know, X Ensure Interactive believes in this model it is. It's very much in some ways way. Call it a new kind of provider, like an experience agency for lack of a better term. TTO help companies drive that transformation, and it's >> done with people and technology, and we're been on a tear recently. Most are growth is organic, but we also do lots and lots of acquisitions to make these capabilities come together. All the creativity and the design and the strategy and the techniques and the run of it is all in one integrated team, and that is very, very helpful when you're trying to do some of the things we've been talking about and you're you guys. I think I'm the right way. This customer wave is really, really because with digital customers air in charge, they control their data. They're now going to shift is happening. We're starting to see some visibility into it. It was going to impact the economics process and business models, so I think it's just beginning. Congratulations really is thanks. And we're so excited because some of the >> client successes it's truly transformational somethings. You got it. Carnival or Marriott or or Subway. I >> mean, it's a whole different >> kind of way of looking at experience, and it's >> really helping people. It's not just for its good for the business, but we're really changing people's lives and helping have experiences be meaningful. It's been wonderful and fun for us. Glenn, Thanks so much for sharing this insights here in the Cube. Hey, thanks for going the data Here live adobe summat. Twenty nineteen, Jumper, jefe Rick, Stay tuned for more coverage after this short break

>>Welcome back to the cubes coverage of ADA reinvent 2021 here, the cube coverage. I'm Judd for a, your host we're on the ground with two sets on the floor, real event. Of course, it's hybrid. It's online as well. You can check it out there. All the on-demand replays are there. We're here with Sandy Carter, worldwide vice president, public sector partners and programs. And we've got Fred Swanick, her founder, and chief curator of the room. We're talking about getting the best talent programming and in the cloud, doing great things, innovation all happening, Sandy. Great to see you. Thanks for coming on the cube, but appreciate it. Thanks for halfway to see. Okay. So tell us about the room. What is the room what's going on? >>Um, well, I mentioned in the room is to help the world's most extraordinary do us to fulfill their potential. So, um, it's a community of exceptional talent that we are building throughout the world, um, and connecting this talent to each other and connecting them to the organizations that are looking for people who can really move the needle for those organizations. >>So what kind of results are you guys seeing right now? Give us some stats. >>Well, it's a, it's a relatively new concept. So we're about 5,000 members so far, um, from 77 different countries. Um, and this is, you know, we're talking about sort of the top two to 3% of talent in different fields. Um, and, um, as we go forward, you know, we're really looking, seeing this as an opportunity to curate, um, exceptional talent. Um, and it feels like software engineering, data science, UX, UI design, cloud computing, um, and, uh, it really helped to, um, identify diverse talent as well from pockets that have typically been untapped for technology. Okay. >>I want to ask you kind of, what's the, how you read the tea leaves. How do I spot the talent, but first talk about the relationship with Amazon. What's the program together? How you guys working together? It's a great mission. I mean, we need more people anyway, coding everywhere, globally. What's the AWS connection. >>So Fred and I met and, uh, he had this, I mean the brilliant concept of the room. And so, uh, obviously you need to run that on the cloud. And so he's got organizations he's working at connecting them through the room and kind of that piece that he was needing was the technology. So we stepped in to help him with the technology piece because he's got all the subject matter expertise to train 3 million Africans, um, coming up on tech, we also were able to provide him some of the classwork as well for the cloud computing models. So some of those certs and things that we want to get out into the marketplace as well, we're also helping Fred with that as well. So >>I mean, want to, just to add onto that, you know, one of the things that's unique about the room is that we're trying to really build a long-term relationship with talent. So imagine joining the room as a 20 year old and being part of it until you're 60. So you're going to have a lot of that. You collect on someone as they progress through different stages of their career and the ability for us to leverage that data, um, and continuously learn about someone's, you know, skills and values and use, um, predictive algorithms to be able to match them to the right opportunities at the right time of their lives. And this is where the machine learning comes in and the, you know, the data lake that we're building to build to really store this massive data that we're going to be building on the top talent to the world. >>You know, that's a really good point. It's a list that's like big trend in tech where it's, it's still it's over the life's life of the horizon of the person. And it's also blends community, exactly nurturing, identifying, and assisting. But at the same day, not just giving people the answer, they got to grow on their own, but some people grow differently. So again, progressions are nonlinear sometimes and creativity can come out of nowhere. Got it. Uh, which brings me up to my number one question, because this always was on my mind is how do you spot talent? What's the secret sauce? >>Well, there is no real secret source because every person is unique. So what we look for are people who have an extra dose of five things, courage, passion, resilience, imagination, and good values, right? And this is what we're looking for. And you will someone who is unusually driven to achieve great things. Um, so of course, you know, you look at it from a combination of their, their training, you know, what they, what they've learned, but also what they've actually done in the workplace and feedback that you get from previous employers and data that we collect through our own interactions with this person. Um, and so we screened them through, you know, with the town that we had, didn't fly, we take them through really rigorous selection process. So, um, it takes, uh, for example, people go through an online assessments and then they go through an in-person interview and then we'll take them through a one to three month bootcamp to really identify, you know, people who are exceptional and of course get data from different sources about the person as well. >>Sandy, how do you see this collaboration helping, uh, your other clients? I mean, obviously talent, cross pollinates, um, learnings, what's your, you see this level of >>It has, uh, you know, AWS grows, obviously we're going to need more talent, especially in Africa because we're growing so rapidly there and there's going to be so much talent available in Africa here in just a few short years. Most of the tech talent will be in Africa. I think that that's really essential, but also as looking after my partners, I had Fred today on the keynote explaining to all my partners around the world, 55,000 streaming folks, how they can also leverage the room to fill some of their roles as well. Because if you think about it, you know, we heard from Presidio there's 3 million open cyber security roles. Um, you know, we're training 20 of mine million cloud folks because we have a gap. We see a gap around the world. And part of my responsibility with partners is making sure that they can get access to the right skills. And we're counting on the room and what Fred has produced to produce some of those great skills. You have AI, AML and dev ops. Tell us some of the areas you haven't. >>You know, we're looking at, uh, business intelligence, data science, um, full-stack software engineering, cybersecurity, um, you know, IOT talent. So fields that, um, the world needs a lot more talented. And I think today, a lot of technology, um, talent is moving from one place to another and what we need is new supply. And so what the room is doing is not only a community of top 10, but we're actually producing and training a lot more new talent. And that was going to hopefully, uh, remove a key bottleneck that a lot of companies are facing today as they try to undergo the digital trends. >>Well, maybe you can add some hosts on there. We need some cube hosts, come on, always looking for more talent on the set. You could be there. >>Yeah. The other interesting thing, John, Fred and I on stage today, he was talking about how easy to the first narrative written for easy to was written by a gentleman out of South Africa. So think about that right. ECE to talent. And he was talking about Ian Musk is based, you know, south African, right? So think about all the great talent that exists. There. There you go. There you go. So how do you get access to that talent? And that's why we're so excited to partner with Fred. Not only is he wicked impressive when a time's most influential people, but his mission, his life purpose has really been to develop this great talent. And for us, that gets us really excited because we, yeah, >>I think there's plenty of opportunities to around new business models in the U S for instance, um, my friends started upstart, which they were betting on people almost like a stock market. You know, almost like currency will fund you and you pay us back. And there's all kinds of gamification techniques that you can start to weave into the system. Exactly. As you get the flywheel going, exactly, you can look at it holistically and say, Hey, how do we get more people in and harvest the value of knowledge? >>That's exactly. I mean, one of the elements of the technology platform that we developed to the Amazon with AWS is the room intelligence platform. And in there is something called legacy points. So every time you, as a member of the room, give someone else an opportunity. You invest in their venture, you hire them, you mentor them, you get points and you can leverage those points for some really cool experiences, right? So you want to game-ify um, this community that is, uh, you know, essentially crowdsourcing opportunities. And you're not only getting things from the room, but you're also giving to others to enable everyone to grow. >>Yeah, what's the coolest thing you've seen. And this is a great initiative. First of all, it's a great model. I think it's, this is the future. Cause I'm a big believer that communities groups, as we get into this hybrid world is going to open up the virtualization. What the virtual world has shown us is virtualization, which is a cloud technology when Amazon started with Zen, which is virtualization technology, but virtualization, conceptually is replicating things. So if you think hybrid world, you can blend the connect people together. So now you have this social construct, this connective tissue between relationships, and it's always evolving, you know, this and you've been involved in community from, from, from the early days when you have that social evolution, it's not software as a mechanism. It's a human thing. Exactly. It's organism, it evolves. And so if you can get the software to think like that and the group to drive the behavior, it's not community software. >>Exactly. I mean, we say that the room is not an online community. It's really an offline community powered by technology. So our vision is to actually have physical rooms in different cities around the world, whether it's talent gathers, but imagine showing up at a, at a room space and we've got the technology to know what your interests are. We know that you're working on a new venture and there's this, there's a venture capitalists in that area, investing that venture, we can connect you right then that space powered by the, >>And then you can have watch parties. For instance, there's an event going on in us. You can do some watch parties and time shifted and then re replicated online and create a localization, but yet have that connection in >>Present. Exactly, exactly. Exactly. So what are the >>Learnings, what's your big learning share with the audience? What you've learned, because this is really kind of on the front edge of the new kind of innovation we're seeing, being enabled with software. >>I mean, one thing we're learning is that, uh, talent is truly, uh, evenly distribute around the world, but what is not as opportunity. And so, um, there's some truly exceptional talent that is hidden and on tap today. And if we can, you know, and, and today with the COVID pandemic companies or around the world, a lot more open to hiring more talent. So there's a huge opportunity to access new talent from, from sources that haven't been tapped before. Well, but also learnings the power of blending, the online and offline world. So, um, you know, the room is, as I mentioned, brings people together, normally in line, but also offline. And so when you're able to meet talent and actually see someone's personality and get a sense of the culture fit the 360 degree for your foot, some of that, you can't just get on a LinkedIn. Yes. That I built it to make a decision, to hire someone who is much better. And finally, we're also learning about the importance of long-term relationships. One of my motives in the room is relationships not transactions where, um, you actually get to meet someone in an environment where they're not pretending in an interview and you get to really see who they are and build relationships with them before you need to hide them. And these are some really unique ways that we think we can redefine how talent finds opportunity in the 21st. So >>You can put a cube in every room, we pick >>You up because, >>And the cube, what we do here is that when people collaborate, whether they're doing an interview together, riffing and sharing content is creating knowledge, but that shared experience creates a bonding. So when you have that kind of mindset and this room concept where it's not just resume, get a job, see you later, it's learning, having peers and colleagues and people around you, and then seeing them in a journey, multiple laps around the track of humans >>And going through a career, not just a job. >>Yes, exactly. And then, and then celebrating the ups and downs in learning. It's not always roses, as you know, it's always pain before you accelerate. >>Exactly. And you never quite arrive at your destination. You're always growing, and this is where technology can really play. >>Okay. So super exciting. Where's this go next, Sandy. And next couple of minutes left in. >>So, um, one of the things that we've envisioned, so this is not done yet, but, um, Fred and I imagined like, what if you could have an Alexa set up and you could say, Hey, you know, Alexa, what should be my next job? Or how should I go train? Or I'm really interested in being on a Ted talk. What could I do having an Alexa skill might be a really cool thing to do. And with the great funding that Fred Scott and you should talk about the $400 million to that, he's already raised $400 million. I mean, there, I think the sky's the limit on platforms. Like >>That's a nice chunk of change. There it is. We've got some fat financing as they say, >>But, well, it's a big mission. So to request significant resources, >>Who's backing you guys. What's the, who's the, where's the money coming from? >>It's coming from, um, the MasterCard foundation. They, our biggest funder, um, as well as, um, some philanthropists, um, and essentially these are people who truly see the potential, uh, to unlock, um, opportunity for millions of people global >>For Glen, a global scale. The vision has global >>Executive starting in Africa, but truly global. Our vision is eventually to have a community of about 10 to 20 million of the most extraordinary doers in the world, in this community, and to connect them to opportunity >>Angela and diverse John. I mean, this is the other thing that gets me excited because innovation comes from diversity of thought and given the community, we'll have so many diverse individuals in it that are going to get trained and mentored to create something that is amazing for their career as well. That really gets me excited too, as well as Amazon website, >>Smart people, and yet identifying the fresh voices and the fresh minds that come with it, all that that comes together, >>The social capital that they need to really accelerate their impact. >>Then you read the room and then you get wherever you need. Thanks so much. Congratulations on your great mission. Love the room. Um, you need to be the in Cuban, every room, you gotta get those fresh voices out there. See any graduates on a great project, super exciting. And SageMaker, AI's all part of, it's all kind of, it's a cool wave. It's fun. Can I join? Can I play? I tell you I need a room. >>I think he's top talent. >>Thanks so much for coming. I really appreciate your insight. Great stuff here, bringing you all the action and knowledge and insight here at re-invent with the cube two sets on the floor. It's a hybrid event. We're in person in Las Vegas for a real event. I'm John ferry with the cube, the leader in global tech coverage. Thanks for watching.

>> It's The Cube covering Fortinet Security Summit brought to you by Fortinet. >> Hey and welcome back to the cubes coverage of Fortinets championship series. Cybersecurity summit here in Napa valley Fortinet is sponsoring the PGA tour event, kicking off the season here, and the cubes here as part of the coverage. And today is cybersecurity day where they bring their top customers in. We got Glenn Katz SVP, general manager, Comcast Enterprise Solutions. Glenn, thanks for coming on The Cube. Thanks for taking time out of your day. - Thank you no This is great. This is great. >> Interviewer: Tell me to explain what you guys do in the Comcast business enterprise group. >> That's our Comcast business. We're a part of Comcast overall. I always like to explain what Comcast really is. If you look at Comcast, it's a technology innovation company by itself that happens to focus on communications and media type of, of markets, right? And if you look at the Comcast side there on the communication side, it's really everything residential with customers. Then there's the us Comcast business and we're the fastest growing entity over the last 15 years within Comcast. And we started in small business, voice, video, and data to small businesses. Then we moved up to provide fiber ethernet type of a transport to mid-market. And then my group started in 2014. And what we do is focus on managed services. It doesn't matter who the transport layer is for enterprise Fortune 1000 type companies. And then when you layer in all these managed wider network services. So that's my business unit. >> Interviewer: Well, we appreciate it we're a customer by the way in Palo Alto >> Glen: Oh great >> So give a shout out to you guys. Let's get into the talk you're giving here about cybersecurity, because I mean, right now with the pandemic, people are working at home. Obviously everyone knows the future of work is hybrid now you're going to see more decentralized defy and or virtual spaces where people are going to want to work anywhere and businesses want to have that extension, right? What people are talking about, and it's not new, but it's kind of new in the sense of reality, right? You've got to execute. This is a big challenge. >> Glen: It is - What's your thoughts on that, >> Well it's a big challenge. And one of the things that I'll try to, I'll speak to this afternoon here, which is at least from the enterprise perspective, which includes the headquarters, the enterprise, the branch locations, the digital commerce, everywhere else, commerce is being done. It's not just at a store anymore. It's everywhere. Even if you only have a store and then you have the remote worker aspect. I mean, they do that to your point earlier. We're not in that fortress sort of security mentality anymore. There's no more DMZs it's done. And so you've got to get down to the zero trust type of network architecture. And how do you put that together? And how does that work? Not just for remote workers that have to access the enterprise applications, but also for simple, you know, consumers or the business customers of these, of these enterprises that have to do business from over the phone or in the store. >> Interviewer: What are the some of the challenges you hear from your customers, obviously, business of the defend themselves now the, the, the attacks are there. There's no parameters. You mentioned no fortress. There's more edge happening, right? Like I said, people at home, what are the top challenges that you're hearing from customers? >> So the biggest challenge, and this is, I would think this is, this is mostly focused on the enterprise side of it is that the is two interesting phenomenons going on. This is sort of beginnings before the pandemic. And then of course the pandemic, the role of the CIO has been elevated to now, they have a real seat at the table. Budgets are increasing to a point, but the expertise needed in these, in these it departments for these large enterprises, it's, it's impossible to do what you were just talking about, which is create a staff of people that can do everything from enterprise applications, e-commerce analytics, the network. How do you secure that network all the way down to the end users? Right? So it's that middle portion. That's the biggest challenge because that takes a lot of work and a lot of effort. And that's where folks like Comcast can come in and help them out. That's their biggest challenge. They can handle the enterprise, they can handle the remote workers. They can handle their own applications, which are continually trying to be, you know, have to be it's competitive out there. It's that middle area, that communications layer that their challenged with. >> Interviewer: Yeah. And John Madison's EVP, CMO Ford. It's always talking about negative unemployment in cybersecurity. Nevermind just the staff that do cyber >> Glen: That's exactly right, that's given. If you're a business, you can't hire people fast enough and you might not have the budget for you want to manage service. So how do you get cyber as a service? >> Glen: Well, so it's even bigger than that. It's not just the cyber as a service because it's now a big package. That's what SASE really is SASE is Secure Access Service Edge. But think of it where I think of it is you've got remote users, remote workers, mobile apps on one side, you've got applications, enterprise or commercial that are now moved into different cloud locations. And in the middle, you've got two real fundamental layers, the network. And, and that includes uh, the actual transport, the software defined wide area, networking components, everything that goes with that, that's the network as a service. And then you've got the secure web gateway portion, which includes everything to secure all the data, going back and forth between your remote laptop, the point of sales. And let's say the cloud based applications, right? So that's really the center stage right there. >> Interviewer: And the cloud has brought more service at the top of the stack. I mean, people thought down stack up stack is kind of like a geeky terms. You're talking about innovation. If you're down stack with network and transport, those are problems that you have to solve on behalf of your customers And make that almost invisible. And that's your job >> That's our job. That's our job is to service provider What's interesting is though back in the day, I mean, when, I mean, back in the day, it could have been 10 years ago in 20. You really, you know, you had stable networks, they were ubiquitous, they were expensive and they were slow. That's kind of the MPLS legacy TDM. Yeah. So you just put them in and you walked away and you still did all your enterprise. You still did all of your applications, but you had your own private data centers. Everything was nicer. It was that fortress mentality right now. It's different. Now everybody needs broadband. Well guess what? Comcast is a big company, but we don't have broadband everywhere. ATT doesn't have it. Verizon doesn't have it Charter doesn't have it. Right. So you need, so now to think about that from enterprise, I'm going to go, I'll give you an example. All of our customers to fulfill a nationwide network, just for the broadband infrastructure, that's, you know, redundant. If you want to think of it that way we, we source probably 200 to 300 different providers to provide an ubiquitous network nationwide for broadband. Then we wrap a layer of the SD wan infrastructure for that, as an example, over the top of that, right? You can't do that by yourself. I mean, people try and they fail. And that's the role of a managed service provider like us is to pull all that together. Take that away. We have that expertise. >> Interviewer: I think this is a really interesting point. Let's just unpack that just for a second. Yeah. In the old days, we want to do an interconnect. You had an agreement. You did, you have your own stuff, do an interconnected connect. >> Glen: Yep. >> Now this, all this mishmash, you got to traverse multiple hops, different networks. >> Glen: That's right >> Different owners, different don't know what's on that. So you guys have to basically stitch this together, hang it together and make it work. And you guys put software on the top and make sure it's cool is that how it works? >> Glen: Yeah. Software and different technology components for the SD wan. And then we would deliver the shore and manager all that. And that's, that's where I really like what's happening in the industry, at least in terminology, which is they try, you have to try to simplify that because it's very, very complicated, but I'm going to give you the network as a service mean, I'm going to give you all the transport and you have to don't have to worry about it. I'm going to rent you the, the SD wan technology. And then I'm going to have in my gateways all these security components for a firewall as a service, zero trust network access, cloud brokerage services. So I will secure all of your data as you go to the cloud and do all of that for you. That's really what we, that's what we bring to the table. And that's what is really, really hard for enterprises to do today. Just because they can't, the expertise needed to do that is just not there. >> Interviewer: Well, what's interesting is that first you have to do it now because the reality of your business now is you don't do it. You won't have customers, but you're making it easier for them. So they don't have to think about it. - [Glen] That's right. >> But now you bring in hybrid networking hybrid cloud, they call it or multi-cloud right. It's essentially a distributed computing and essentially what you're doing, but with multiple typologies, >> Glen: that's right. >> Interviewer: I got an edge device. - [Glen] That's right. If I'm a business. - [Glen] That's right. >> That's where it could be someone working at home >> Glen: That's right. - Or it could be my retail >> Or whatever it could be. So edge is just an extension of what you guys already do. And is that right? Am I getting that right? >> Glen: Yeah that's exactly right. And, and, but the point is, is to make it economic and to make it really work for the end user. If you're a branch, you may have a, a application that's still being run via VPN, but you also need wifi internet for your customers because you want to use your mobile device. They've entered into your store and you want to be able to track that right. And push something to them. And then you've got the actual store applications could be point of sales could be back of house comparing that's going up to AWS. Azura whatever. Right. And that all has to be, it all has to come from one particular branch and someone has to be able to manage that capability. >> Interviewer: It's funny, - Its so different >> Interviewer: just as you're talking, I'm just thinking, okay. Facial recognition, high, high bandwidth requirements, >> Glen: Huge high bandwidth requirements >> Processing at the edge becomes huge. >> Glen: It does. >> So that becomes a new dynamic. >> Glen: It does. It's got to be more dynamic. It's not a static IP end point. >> Glen: Well, I'll give you another an example. Let's say it's, it seems silly, but it's so important from a business perspective, your quick service restaurant, the amount of digital sales from applications are just skyrocketing. And if you yourself, and particularly in the pandemic, you order something, or that goes up to the cloud, comes back through, goes to the point of sales. And then the, the back of house network in a particular restaurant, if that doesn't get there, because one line of you only have one internet connection and it's down, which sometimes happens, right? You lose business, you lose that customer. It's so important. So what's being pushed down to the edge is, you know, reliable broadband hybrid networks, where you have a primary wire line and a secondary wire line, maybe a tertiary wireless or whatever. And then a box, a device that can manage between those two so that you can keep that 99.9, 9% availability at your branch, just for those simple types of applications. >> Interviewer: You know Glenn, you as you're talking most people, when we talk tech, like this is mostly inside the ropes, Hey, I can get it. But most people can relate with the pandemic because they've ordered with their phone on - [Glen] Exactly right >> With the QR code. - [Glen] That's exactly right >> They see the menu - [Glen] That's right >> They get now what's happening - [Glen] That's right that their phone is now connected to the service. >> Glen: That's right >> This is not going away. The new normal. >> Glen: No, it's absolutely here. And what I've seen are there are many, many companies that already knew this and understood this pre pandemic. And they were, they had already changed their infrastructure to really fit what I was calling that network as a service in the SASE model, in different ways. Then there were a bunch that didn't, and I'm not going to name names, but you can look at those companies and you can see how they're, they're struggling terribly. But then there was this. Now there's a, a much bigger push and privatization again, see, I was sending, Hey, I asked for this before. It's not like the CIO didn't know, but management said, well, maybe it wasn't important. Now it is. And so you're seeing this actual amazing surge in business requests and requirements to go to the model that we're all talking about here, which is that SASE type of implementation high-speed broadband. That's not going away for the same reason. And you need a resilient network, right? Yes. >> Interesting. Best practice. Let's just take that advice to the, to the audience. I want to get your thoughts because people who didn't do any R and D or experimentation prior to the pandemic, didn't have cloud. Wasn't thinking about this new architecture got caught flat-footed. -Exactly. >> And they're hurting and or out of business. >> Correct. >> If people who were on the right side of that took advantage as a tailwind and they got lifts. >> That's exactly right. >> So what is the best practice? How should a business think about putting their toe in the water a little bit or jumping in and getting immersed in the new, new architecture? What advice would you give? Because people don't want to be in the wrong side of history. >> No, they don't. >> What's your guy's best practice? >> I may sound biased, but I'm really not trying to be biased. And this'll be some of the I'll speak about here later today. You have to try it. You, as the end user, the enterprise customer, to, to fulfill these types of needs, you've got to really probe your managed service providers. You've got to understand which ones, not just can give you a nice technology presentation and maybe a POC, but who's going to be there for the longterm who has the economic wherewithal to be able to give the resources needed to do what I was talking about, which is you're going to outsource your entire network to me and your sh, and a good portion of your security for the network to a service provider. that service provider has to be able to provide all that has to be able to have the financial capabilities, to be able to provide you with an operating type of model, not you have to buying equipment all the time. That service provider has to be able to have teams that can deliver all of that 200 to 300 different types of providers aggregate all that, and then be there for day two. Simple thing. Like if you know, most companies, if you're not a really large location, you can't afford to, you know, double types of routers that are connected. And if one fails you have fail over, right, most of them will have one router and they'll have, but they'll have two backup paths. Well, what happens is that router or switch, single switch fails? You need to have a meantime to repair a four hours. I mean, that's kind of basic and well do that. How do you do that? You've got to have depots around the entire country. These are the types of questions that any enterprise customers should be probing their managed service provider, right? It's not just about the technology. It's about how can you deliver this and assure this going forward. >> And agility too cause when, if, if things do change rapidly, being agile... >> Exactly >> means shifting and being flexible with your business. >> That's exactly right. And that's important. That's a really important question. And the agility comes from this financial agility, right? Like new threat, new box. I want, I want this old one. I'm going to upgrade to a different type of service. The service providers should be able to do that without me having to force you to go get some more CapEx and buy some more stuff. Cause that's number one. But the other agility is every enterprise is different. Every enterprise believes that its network is the only network in the world and they have opinions and they've tested different technologies. And you're going to have to adapt a little bit to that. And if you don't, you're not going to get out of this. >> It's funny. The old days non-disruptive operations was like a benefit, we have non-disrupt- now it's a table stakes. You can't disrupt businesses. - You can't. You can't at the branch at the remote worker. If you're on a zoom call or whatever, or you're on a teams call, we've all been there. We're still doing it. If it breaks in the middle of a presentation to a customer that's problem. >> Glenn thanks for coming on the cube with great insight. >> Oh great. This was fun. >> Are you exciting and plays golf? You're going to get out there on the range? >> I played, I played golf a lot when I was younger, but I haven't. And so I have a few other things I do, but I guess I'm going to have to learn now that we're also a sponsor of PGA, so yeah, for sure. >> Great. Well, great to have you on - All right thank you and great talk. Thanks for coming on and sharing your insight. >> This was great. I appreciate okay. >> Keep coverage here. Napa valley with Fortinet's Cybersecurity Summit as part of their PGA tour event, that's happening this weekend. I'm John for the Cube. Thanks for watching.

>>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi guys. Welcome back. So we have discussed the current state and the near future state of DevOps and how it's going to evolve from three unique perspectives. In this last segment, we're going to open up the floor and see if we can come to a shared understanding of where dev ops needs to go in order to be successful next year. So our guests today are, you've seen them all before Jeffrey Hammond is here. The VP and principal analyst serving CIO is at Forester. We've also got Serge Lucio, the GM of Broadcom's enterprise software division and Glenn Martin, the head of QA transformation at BT guys. Welcome back. Great to have you all three together >>To be here. >>All right. So we're very, we're all very socially distanced as we've talked about before. Great to have this conversation. So let's, let's start with one of the topics that we kicked off the forum with Jeff. We're going to start with you spiritual co-location that's a really interesting topic that we've we've uncovered, but how much of the challenge is truly cultural and what can we solve through technology? Jeff, we'll start with you then search then Glen Jeff, take it away. >>Yeah, I think fundamentally you can have all the technology in the world and if you don't make the right investments in the cultural practices in your development organization, you still won't be effective. Um, almost 10 years ago, I wrote a piece, um, where I did a bunch of research around what made high performance teams, software delivery teams, high performance. And one of the things that came out as part of that was that these teams have a high level of autonomy. And that's one of the things that you see coming out of the agile manifesto. Let's take that to today where developers are on their own in their own offices. If you've got teams where the team itself had a high level of autonomy, um, and they know how to work, they can make decisions. They can move forward. They're not waiting for management to tell them what to do. >>And so what we have seen is that organizations that embraced autonomy, uh, and got their teams in the right place and their teams had the information that they needed to make the right decisions have actually been able to operate pretty well, even as they've been remote. And it's turned out to be things like, well, how do we actually push the software that we've created into production that would become the challenge is not, are we writing the right software? And that's why I think the term spiritual co-location is so important because even though we may be physically distant, we're on the same plane, we're connected from a, from, from a, a, a shared purpose. Um, you know, surgeon, I worked together a long, long time ago, surgery it's been what almost 15, 16 years since we were at the same place. And yet I would say there's probably still a certain level of spiritual co-location between us, uh, because of the shared purposes that we've had in the past and what we've seen, uh, in the industry. And that's a really powerful tool, uh, to build on. So what do tools play as part of that, to the extent that tools make information available, to build shared purpose on to the extent that they enable communication so that we can build that spiritual co-location to the extent that they reinforce the culture that we want to put in place, they can be incredibly valuable, especially when, when we don't have the luxury of physical locate, physical colocation. Hope. That makes sense. >>It does. I should have introduced us. This last segment is we're all spiritually co-located or it's a surge, clearly you're still spiritually co located with junk. Talk to me about what your thoughts are about spiritual of co-location the cultural impact and how technology can move it forward. >>Yeah. So I think, well, I'm going to sound very similar to Jeff in that respect. I think, you know, it starts with kind of a shared purpose and the other, I, Oh, individuals teams, uh, contributed to kind of a business outcome. What is our shared goal or shared vision? What's what is it we're trying to achieve collectively and, uh, keeping it aligned to that. Um, and so, so it's really starts with that now, now the big challenge, always these over the last 20 years, especially in large organizations, there's the specialization of roles and functions. And so we, we all that started to basically measure which we do, uh, on a daily basis using metrics, which oftentimes are completely disconnected from kind of a business outcome or purpose. We, we kind of revert back to, okay, what is my database all the time? What is my cycle time like? >>And, and I think, you know, which we can do or where we really should be focused as an industry is to start to basically provide a lens for these different stakeholders to look at what they're doing in the context of kind of these business outcomes. So, um, you know, probably one of my, um, theories of experience was to actually weakness at one of a large financial institution, um, you know, to stakeholders and quote development and operations staring at the same data, right. Which was related to, you know, in calming changes, um, testing, execution results, you know, covert coverage, um, official liabilities and all the all ran. It could have a direction leveling. So that's when you start to put these things in context and represent that in a way that these different stakeholders can, can look at from their different lens. And, uh, and it can start to basically communicate and understand of they jointly are competing to, uh, to, to that kind of common view or objective. >>And Glen, we talked a lot about transformation with you last time. What are your on spiritual co-location and the cultural part, the technology impact? >>Yeah, I mean, I agree with Jeffrey that, you know, um, the people and culture, the most important thing, actually, that's why it's really important when you're transforming to have partners who have the same vision as you, um, who, who you can work with, have the same end goal in mind. And I've certainly found that with our, um, you know, continuing relationship with Broadcom, what it also does though, is although, you know, tools can accelerate what you're doing and can join consistency. You know, we've seen within simplify, which is BTS flagship transformation program, where we're trying to, as it says, simplify the number of systems stacks that we have, the number of products that we have actually at the moment, we've got different value streams within that program who have got organizational silos who were trying to rewrite, rewrite the wheel, um, who are still doing things manually. >>So in order to try and bring that consistency, we need the right tools that actually are at an enterprise grade, which can be flexible to work with in BT, which is such a complex and very different environments. But in all areas, BT you're in whether it's a consumer, whether it's a mobile area, whether it's large global or government organizations, you know, we found that we need tools that can drive that consistency, but also flex to Greenfield brownfield kind of technologies as well. So it's really important that as I say, for a number of different aspects, that you have the right partner, um, to drive the right culture, I've got the same vision, but also who have the tool sets to help you accelerate. They can't do that on their own, but they can help accelerate what it is you're trying to do in it. And a really good example of that is we're trying to shift left, which is probably a, quite a bit of a buzz phrase in there kind of testing world at the moment. >>But, you know, I could talk about things like continuous delivery director, one of Broadcom's tools, and it has many different features to it, but very simply on its own, it allows us to give the visibility of what the teams are doing. And once we have that visibility, then we can talk to the teams, um, around, you know, could they be doing better component testing? Could they be using some virtualized services here or there? And that's not even the main purpose of continuous delivery director, but it's just a reason that tools themselves can just give greater visibility of have much more intuitive and insightful conversations with other teams and reduce those organizational silos. >>Thanks, Ben. So we'd kind of sum that up. Autonomy collaboration tools that facilitate that. So let's talk now about metrics from your perspectives. What are the metrics that matter, Jeff? >>Well, I'm going to go right back to what Glenn said about data that provides visibility that enables us to, to make decisions, um, with shared purpose. And so business value has to be one of the first things that we at. Um, how do we assess whether we have built something that is valuable, you know, that could be sales revenue, it could be net promoter score. Uh, if you're not selling what you've built, it could even be what the level of reuse is within your organization or other teams picking up the services, uh, that you've created. Um, one of the things that I've begun to see organizations do is to align value streams with customer journeys and then to align teams with those value streams. So that's one of the ways that you get to a shared purpose, cause we're all trying to deliver around that customer journey, the value associated with it. >>And we're all measured on that. Um, there are flow metrics which are really important. How long does it take us to get a new feature out from the time that we conceive it to the time that we can run our first experiments with it? There are quality metrics, um, you know, some of the classics or maybe things like defect, density, or meantime to response. Um, one of my favorites came from a, um, a company called ultimate software where they looked at the ratio of defects found in production to defects found in pre production and their developers were in fact measured on that ratio. It told them that guess what quality is your job to not just the test? Uh, department's a group. The fourth level that I think is really important, uh, in, in the current, uh, uh, situation that we're in is the level of engagement in your development organization. >>We used to joke that we measured this with the parking lot metric. How full was the parking lot at nine? And how full was it at five o'clock? I can't do that anymore since we're not physically co-located, but what you can do is you can look at how folks are delivering. You can look at your metrics in your SCM environment. You can look at, uh, the relative rates of churn. Uh, you can look at things like, well, are our developers delivering, uh, during longer periods earlier in the morning, later in the evening, are they delivering, uh, you know, on the weekends as well? Are those signs that we might be heading toward a burnout because folks are still running at sprint levels instead of marathon levels. Uh, so all of those in combination, uh, business value, uh, flow engagement in quality, I think form the backbone of any sort of, of metrics, uh, uh, a program. >>The second thing that I think you need to look at is what are we going to do with the data and the philosophy behind the data is critical. Um, unfortunately I see organizations where they weaponize the data and that's completely the wrong way to look at it. What you need to do is you need to say, you need to say, how is this data helping us to identify the blockers? The things that aren't allowing us to provide the right context for people to do the right thing. And then what do we do to remove those blockers, uh, to make sure that we're giving these autonomous teams the context that they need to do their job, uh, in a way that creates the most value for the customer. >>Great advice stuff, Glenn, over to your metrics that matter to you that really make a big, and also, >>How do you measure quality kind of following onto the advice that Jeff provided? I mean, Jeff provided some great advice. Actually, he talks about value. He talks about flow. Both of those things are very much on my mind at the moment. Um, but there was this, I listened to a speaker called me Kirsten a couple of months ago. It talked very much about how important flight management is and removing, you know, and using that to remove waste, to understand in terms of, you know, making software changes, um, what is it that's causing us to do it longer than we need to. So where are those areas where it takes too long? So I think that's a very important thing for us. It's, um, even more basic than that at the moment, we're on a journey from moving from kind of a waterfall to agile. Um, and the problem with moving from waterfall to agile is with waterfall, the, the business had a kind of comfort that, you know, everything was tested together and therefore it's safer. >>Um, and with agile, there's that kind of, how do we make sure that, you know, if we're doing things quick and we're getting stuff out the door that we give that confidence, um, that that's ready to go, or if there's a risk that we're able to truly articulate what that risk is. So there's a bit about release confidence, um, and some of the metrics around that and how healthy those releases are, and actually saying, you know, we spend a lot of money, um, um, an investment setting up, Pat, our teams training our teams, are we actually seeing them deliver more quickly and are we actually seeing them deliver more value quickly? So yeah, those are the two main things for me at the moment, but I think it's also about, you know, generally bringing it all together, the dev ops, you know, we've got the kind of value ops AI ops, how do we actually bring that together to so we can make quick decisions and making sure that we are delivering the biggest bang for our buck, absolutely biggest bang for the buck, surge, your thoughts. >>Yeah. So I think we all agree, right? It starts with business metrics, flow metrics. Um, these are kind of the most important metrics. And ultimately, I mean, one of the things that's very common across a highly functional teams is engagements, right? When, when you see a team that's highly functioning, that's agile, that practices DevOps every day, they are highly engaged. Um, that that's, that's definitely true. Now the, you know, back to, I think, uh, GemCis point on weaponization of metrics. One of the key challenges we see is that, um, organizations traditionally have been kind of, uh, you know, setting up benchmarks, right? So what is a good cycle time? What is a good lead time? What is a good meantime to repair? The, the problem is that this is very contextual, right? It varies. It's going to vary quite a bit, depending on the nature of application and system. And so one of the things that we really need to evolve, um, as an industry is to understand that it's not so much about those flow metrics is about, are these four metrics ultimately contribute to the business metric to the business outcome. So that's one thing, the second aspect, I think that's oftentimes misunderstood. >>Yeah. >>So that cycle time, or, or, or what you perceive as being a buy cycle time or better quality, the problem is oftentimes like all, do you go and explore why, right. What is the root cause of this? And I think one of the key challenges is that we tend to focus a lot of time on metrics and not on the eye type patterns, which are pretty common across the industry. Um, you know, you look at, for instance, things like, you know, lead time, for instance, it's very common that, uh, organizational boundaries are going to be a key contributor to badly time. And so I think that there is, you know, the metrics there is, I think a lot of, uh, work that we need to do in terms of classifying this antibiograms, um, you know, back to you, Jeff, I think you're one of the cool offers of waterscrumfall as a, as a, as a key patterning industry or anti-fat. Um, but what our scrum fall right, is a key one, right. And you will detect that through defect, arrival rates. That's where that looks like an escort. And so I think it's beyond kind of the metrics is what do you do with those metrics? >>Right? I'll tell you a search. One of the things that is really interesting to me in that space is I think those of us had been in industry for a long time. We know the anti-patterns cause we've seen them in our career maybe in multiple times. And one of the things that I think you could see tooling do is perhaps provide some notification of anti-patterns based on the telemetry that comes in. I think it would be a really interesting place to apply, uh, machine learning and reinforcement learning techniques. Um, so hopefully something that we'd see in the future with dev ops tools, because, you know, as a manager that, that, you know, may be only a 10 year veteran or 15 year veteran, you may be seeing these anti-patterns for the first time. And it would sure be nice to know what to do, uh, when they start to pop up, >>That would right. Insight, always helpful. All right, guys, I would like to get your final thoughts on the fit. The one thing that you believe our audience really needs to be on the lookout for and to put on our agendas for the next 12 months, Jeff, we'll go back to you. >>I would say, look for the opportunities that this disruption presents. And there are a couple that I see, first of all, as we shift to remote central working, uh, we're unlocking new pools of talent, uh, we're, it's possible to implement, uh, more geographic diversity. So, so look to that as part of your strategy. Number two, look for new types of tools. We've seen a lot of interest in usage of low-code tools to very quickly develop applications. That's potentially part of a mainstream strategy as we go into 2021. Finally, make sure that you embrace this idea that you are supporting creative workers that agile and dev ops are the peanut butter and chocolate to support creative, uh, workers with algorithmic capabilities, >>Peanut butter and chocolate Glen, where do we go from there? What are, what's the one silver bullet that you think folks should be on the lookout for? >>I certainly agree that, um, low, low code is, uh, next year. We'll see much more low code we'd already started going, moving towards a more of a SAS based world, but Loco also, um, I think as well for me, um, we've still got one foot in the kind of cow camp. Um, you know, we'll be fully trying to explore what that means going into the next year and, and exploiting the capabilities of cloud. But I think the last, um, the last thing for me is how do you really instill quality throughout the kind of, um, the life cycle, um, where, when I heard the word scrum for it kind of made me shut it because I know that's a problem. That's where we're at with some of our things at the moment. So we need to get beyond that. We need to be releasing, um, changes more frequently into production and actually being a bit more brave and having the confidence to actually do more testing in production in going straight to production itself. So expect to see much more of that next year. Um, yeah. Thank you. I haven't got any food analogies. Unfortunately >>We all need some peanut butter and chocolate. All right. It starts to take us on that's what's that nugget you think everyone needs to have on their agendas. >>That's interesting. Right. So a couple of days ago we had kind of a latest state of the DevOps report, right? And if you read through the report, it's, it's all about the lost city, right? It's all about, we still are receiving DevOps as being all about speed. And so to me, the key advice is in order to create kind of that spiritual collocation in order to foster engagement, we have to go back to what is it we're trying to do collectively. We have to go back to tie everything to the business outcome. And so for me, it's absolutely imperative for organizations to start to plot their value streams, to understand how they're delivering value and to align everything they do from a metrics to deliver it, to flow to those metrics. And only with that, I think, are we going to be able to actually start to really start to align kind of all these roles across the organizations and drive, not just speed, but business outcomes, >>All about business outcomes. I think you guys, the three of you could write a book together. So I'll give you that as food for thought. Thank you all so much for joining me today and our guests. I think this was an incredibly valuable fruitful conversation, and we appreciate all of you taking the time to spiritually co-located with us today, guys. Thank you. Thank you, Lisa. Thank you for Jeff Hammond serves Lucio and Glen Martin. I'm Lisa Martin. Thank you for watching the broad cops Broadcom dev ops virtual forum.

>>from around the globe. It's >>the Cube with digital coverage of Dev >>Ops Virtual Forum Brought to You by Broadcom. Welcome to Broadcom, Step Ups, Virtual Forum I and Lisa Martin and I'm joined by another Martin very socially. Distance from me all the way. Coming from Birmingham, England, is Glynn Martin, head of Q. A transformation at BT Glenn. It's great to have you on the program. >>Thank you, Lisa. I'm looking forward, Toa. >>As we said before, we went live to Martin's for the price of one in one segment. So this is gonna be an interesting segment, Guesses. What we're gonna do is Glen's gonna give us a really kind of deep inside out view of Dev ops. From an evolution perspective, Soglo's Let's start transformation is at the heart of what you dio. It's obviously been a very transformative year. How have the events of this year affected the transformation that you are so responsible for driving? >>Yeah. Thank you, Leigh. So I mean, yeah, it has been a difficult year Bond, although working for BT, which is ah, global telecommunications company. Relatively resilient, I suppose, as an industry through covert, it obviously still has been affected and has got its challenges on bond. If anything is actually caused us to accelerate of our transformation journey, you know, we had to do some great things during this time around. You know, in the UK for our emergency and health workers give them unlimited data and for vulnerable people to support them and that spent that we've had to deliver changes quickly. Um, but what? We want to be able to do it, deliver those kind of changes quickly, but sustainably for everything that we do, not just because there's an emergency eso we were already on the kind of journey to by John, but ever so ever more important now that we are what we're able to do, those that kind of work, do it more quickly on. But it works because the implications of it not working is could be terrible in terms of, you know, we've been supporting testing centers, new hospitals to treat covert patients, so we need to get it right and therefore the coverage of what we do, the quality of what we do and how quickly we do. It really has taken on a new scowling what was already a very competitive market within the telco industry within the UK. Um, you know, what I would say is that you know, we are under pressure to deliver more value, but we have small cost challenges. We have to obviously deal with the fact that you know, Cove in 19 has hit most industries kind of revenues and profits. So we've got this kind of paradox between having less cost, but they're having to deliver more value quicker on bond, you know, to higher quality. So, yeah, certainly the finances is on our minds. And that's why we need flexible models, cost models that allow us to kind of do growth. But we get that growth by showing that we're delivering value, especially in, you know, these times when there are financial challenges on companies. >>So one of the things that I want to ask you about again looking at, develops from the inside out on the evolution that you've seen you talked about the speed of things really accelerating in this last nine months or so. When we think Dev ops, we think speed. But one of the things I love to get your perspective on we've talked about in a number of the segments that we've done for this event is cultural change. What are some of the things that scene there as as needing to get, as you said, get things right but done so quickly to support essential businesses, essential workers? How have you seen that cultural shift? >>Yeah, I think you know, before, you know, test test team saw themselves of this part of the software delivery cycle. Andi, actually, now, really, our customers were expecting their quality and to deliver for our customers what they want. Quality has to be ingrained throughout the life cycle. Obviously that you know, there's lots of buzzwords like shift left. How do you do? Shift left testing. But for me, that's really instilling quality and given capabilities shared capabilities throughout the life cycle. That Dr you know, Dr Automation drive improvements. I always say that you know, you're only as good as your lowest common denominator on one thing that we're finding on our Dev Ops Journey Waas that we were you know, we would be trying thio do certain things quicker and had automated build automated tests. But if we were taking weeks to create test scripts or we were taking weeks to manly craft data, and even then when we had taken so long to do it that the coverage was quite poor and that led to lots of defects later in the lifecycle or even in in our production environment, we just couldn't afford to do that. And actually, you know, focusing on continuous testing over the last 9 to 12 months has really given us the ability Thio delivered quickly across the the whole life cycle and therefore actually go from doing a kind of semi agile kind of thing where we did you use the stories we did a few of the kind of, you know, as our ceremonies. But we weren't really deploying any quicker into production because, you know, our stakeholders were scared that we didn't have the same control that we had when we had more water for releases. And, you know, when way didn't think ourselves. So we've done a lot of work on every aspect, especially from a testing point of view, every aspect of every activity, rather than just looking at automated test, you know, whether it is actually creating the test in the first place, Whether it's doing security testing earlier in the light and performance testing. Learn the life cycle, etcetera. So, yeah, it Z It's been a riel key thing that for for C T for us to drive, develops, >>talk to me a little bit about your team. What are some of the shifts in terms of expectations that you're experiencing and how your team interacts with the internal folks from pipeline through life cycle? >>Yeah, we've done a lot of work on this, you know, there's a thing. I think people were pretty quiet. Customer experience. Gap. It reminds me of a cart, a Gilbert cartoon where, you know, we start with the requirements here on Do you know, we almost like a Chinese whisper effects and what we deliver eyes completely, completely different. So we think the testing team or the the delivery team, you know, you know, you think they've done a great job. This is what it said in the acceptance criteria, but then our customers the same Well, actually, that's not working. This isn't working, you know, on there's this kind of gap Way had a great launched this year of actual Requirement Society, one of the board common tools Onda that for the first time in in since I remember actually working within B. T, I had customers saying to may, Wow, you know, we want more of this. We want more projects, um, to have a actual requirements design on it because it allowed us to actually work with the business collaboratively. I mean, we talk about collaboration, but how do you actually, you know, do that have something that both the business on technical people can understand? And we've actually been working with the business using at our requirement. Designer Thio, you know, really look about what the requirements are. Tease out requirements to the hadn't even thought off and making sure that we've got high levels of test coverage. And so what we actually deliver at the end of it, not only have you been able Thio generate test more quickly, but we've got much higher test coverage and also can more smartly, you're using the kind of AI within the tour and with some of the other kind of pipeline tools actually deliver to choose the right tests on the bar, still actually doing a risk based testing approach. So that's been a great launched this year, but just the start of many kind of things that we're >>doing. But what I hear in that Glenn is a lot of positives that have come out of a very challenging situation. Uh, talk to me about it and I like that perspective. This is a very challenging time for everybody in the world, but it sounds like from a collaboration, perspective is you're right. We talk about that a lot critical with Dev Ops. But those challenges there you guys were able to overcome those pretty quickly. What other challenges did you face and figure out quickly enough to be able to pit it so fast? >>I mean, you talked about culture. I mean, you know, Bt is like most come countries companies. So, um, is very siloed. You know, we're still trying to work to become closer as a company. So I think there's a lot of challenges around. How do you integrate with other tools? How do you integrate with you know, the various different technologies and bt we have 58 different whitey stacks? That's not systems that stacks all of those stacks of can have, you know, hundreds of systems on we're trying to. We're gonna drive at the moment a simplified program where we're trying Thio, you know, reduce that number 2 14 stacks. And even then they'll be complexity behind the scenes that that we will be challenged. Maurin Mawr As we go forward, how do you actually hired that to our users on as an I T organization? How do we make ourselves Lena so that even when we you know, we've still got some of that legacy and we'll never fully get rid of it on that's the kind of trade off that we have to make. How do we actually deal with that and and hide that for my users a say and and and drive those programs so we can actually accelerate change. So we take, you know, reduce that kind of waste, and that kind of legacy costs out of our business. You know, the other thing is, well, beating. And I'm sure you know telecoms probably no difference to insurance or finance we've got You know, when you take the number of products that we do and then you combine them, the permutations are tens and hundreds of thousands of products. So we as a business to trying to simplify. We are trying Thio do that in a natural way and haven't trying to do agile in the proper way, you know, and really actually work it paste really deliver value. So I think what we're looking Maura, Maura, at the moment is actually, um is more value focus? Before we used to deliver changes, sometimes into production, someone had a great idea or it was a great idea nine months ago or 12 months ago. But actually, then we end up deploying it. And then we look at the the the users, you know, the usage of that product of that application or whatever it is on. It's not being used for six months, so we're getting much we haven't got, you know, because of the last 12 months, we certainly haven't got room for that kind of waste and you know, the for not really understanding the value of changes that we we are doing. So I think that's the most important thing at the moment is really taken that waste out. You know, there's lots of focus on things like flow management. What bits of the our process are actually taking too long, and we've We've started on that journey, but we've got a hell of a long way to go, you know, But that that involves looking every aspect off the kind of software delivery cycle. >>What are some? Because that that going from, what, 58 i t stocks down to 14 or whatever it's going to be go simplifying is sounds magical. Took everybody. It's a big challenge. What are some of the core technology capabilities that you see really as kind of essential for enabling that with this new way that you're working? >>Yeah. I mean, I think we've started on a continuous testing journey, and I think that's just the start. I mean, that's really, as I say, looking at every aspect off, you know, from a Q, a point of view. It's every aspect of what we dio. But it's also looking at, you know, we're starting to branch into more like a AI ops and, you know, really, the full life cycle on. But, you know, that's just a stepping stone onto, you know, I think oughta Nomics is the way forward, right? You know all of this kind of stuff that happens um, you know, monitoring, you know, monitoring systems, what's happening in production had to be feed that back. How do you get to a point where actually we think about a change on then suddenly it's in production safely. Or if it's not going to safety, it's automatically backing out. So, you know, it's a very, very long journey. But if we want Thio, you know, in a world where the pace is ever increasing the demands of the team and you know, with the pressures on at the moment where with we're being asked to do things, you know more efficiently Ondas leaving as possible. We need to be, you know, thinking about every part of the process. And how do we put the kind of stepping stones in players to lead us to a more automated kind of, you know, their future? >>Do you feel that that plant outcomes are starting to align with what's delivered? Given this massive shift that you're experiencing, >>I think it's starting to, and I think you know, Azzawi. Look at more of a value based approach on. Do you know a Zeiss? A princess was a kind of flight management. I think that's that will become ever evermore important. So I think it's starting to people. Certainly realized that, you know, people teams need to work together. You know, the kind of the cousin between business and ICT, especially as we go Teoh Mawr kind of sad space solutions, low cold solutions. You know there's not such a gap anymore. Actually, some of our business partners expects to be much more tech savvy. Eso I think you know, this is what we have to kind of appreciate. What is I ts role? How do we give the capabilities become more for centers of excellence rather than actually doing Mount amount of work And for May and from a testing point of view, you know, amount, amount of testing, actually, how do we automate that? How do we actually generate that instead of created? I think that's the kind of challenge going forward. >>What are some? As we look forward, what are some of the things that you would like to see implemented or deployed in the next say, 6 to 12 months as we hopefully round a corner with this pandemic? >>Yeah, I think you know, certainly for for where we are as a company from a Q A perspective. We are. Yeah, there's certain bits that we do Well, you know, we've started creating continuous delivery. A day evokes pipelines. Um, there's still manual aspects of that. So, you know, certainly for May I I've challenged my team with saying, How do we do an automated journey? So if I, you know, I put a requirement injera or value whoever it is, that's why. Then click a button on bond, you know, with either zero touch of one touch, then put that into production and have confidence that that has been done safely on that it works. And what happens if it doesn't work? So you know, that's that's the next in the next few months, that's what our concentration is about. But it's also about decision making, you know, how do we actually understand those value judgements? And I think there's lots of the things Dev ops, ai ops, kind of always that aspects of business operations. I think it's about having the information in one place to make those kind of decisions. How does it all tied together, as I say, even still with kind of Dev ops, we've still got elements within my company where we've got lots of different organizations doing some doing similar kind of things but the walking of working in silos Still. So I think, having a eye ops Aziz becomes more and more to the fore as we go to the cloud. And that's what we need to. You know, we're still very early on in our cloud journey, you know. So we need to make sure the technologies work with Cloud as well as you kind of legacy systems. But it's about bringing that all together and having a full visible pipeline. Everybody can see and make decisions against >>you said the word confidence, which jumped out at me right away. Because absolutely, you've gotta have be able to have confidence in what your team is delivering and how it's impacting the business and those customers. Last question for you is how would you advise your peers in a similar situation to leverage technology automation, for example, dev ops to be able to gain the confidence that they're making the right decisions for their business? >>Yeah, I mean, I think the the approach that we've taken actually is not started with technology we've actually taken human centered design a za core principle of what we dio within the i t part of BT. So by using humans tend to design. That means we talked to our customers. We understand their pain points, we map out their current processes on. But when we mapped out, those processes also understand their aspirations as well, you know, Where do they want to be in six months? You know, Do they want to be more agile and you know, or do they want Teoh? Is this apart their business that they want thio run better? We have to Then look at why that's not running well and then see what solutions are out there. We've been lucky that, you know, with our partnership with Broadcom within the P l. A. A lot of the tortures and the P l. A have directly answered some of the businesses problems. But I think by having those conversations and actually engaging with the business, um, you know, especially if the business hold the purse strings, which is you know, in some companies, including as they do there is that kind of, you know, almost by understanding their their pain points and then saying This is how we can solve your problem We've tended to be much more successful than trying Thio impose something and say We're here to technology that they don't quite understand doesn't really understand how it could have resonate with their problems. So I think that's the heart of it is really about, you know, getting looking at the data, looking at the processes, looking at where the kind of waste is on. Then actually then looking at the right solutions. And as I say, continuous testing is a massive for us. We've also got a good relationship with capitals looking at visual ai on. Actually, there's a common theme through that, and I mean, AI is becoming more and more prevalent, and I know yeah, sometimes what is A I and people have kind of the semantics of it. Is it true, ai or not? But yes, certainly, you know, AI and machine learning is becoming more and more prevalent in the way that we work, and it's allowing us to be much more effective, the quicker and what we do on being more accurate. You know, whether it's finding defects, running the right tests or, you know, being able to anticipate problems before they're happening in a production environment. >>Welcome. Thank you so much for giving us this sort of insight. Outlook at Dev Ops, sharing the successes that you're having taking those challenges, converting them toe opportunities and forgiving folks who might be in your shoes or maybe slightly behind advice. I'm sure they appreciate it. We appreciate your time. >>It's been an absolute pleasure, Really. Thank you for inviting me of Extremely enjoyed it. So thank you ever so much. >>Excellent. Me too. I've learned a lot for Glynn Martin and Lisa Martin. You're watching the Cube?

>>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi, Lisa Martin here covering the Broadcom dev ops virtual forum. I'm very pleased to be joined today by a cube alumni, Jeffrey Hammond, the vice president and principal analyst serving CIO is at Forester. Jeffrey. Nice to talk with you today. >>Good morning. It's good to be here. Yeah. >>So a virtual forum, great opportunity to engage with our audiences so much has changed in the last it's an understatement, right? Or it's an overstated thing, but it's an obvious, so much has changed when we think of dev ops. One of the things that we think of is speed, you know, enabling organizations to be able to better serve customers or adapt to changing markets like we're in now, speaking of the need to adapt, talk to us about what you're seeing with respect to dev ops and agile in the age of COVID, what are things looking like? >>Yeah, I think that, um, for most organizations, we're in a, uh, a period of adjustment, uh, when we initially started, it was essentially a sprint, you know, you run as hard as you can for as fast as you can for as long as you can and you just kind of power through it. And, and that's actually what, um, the folks that get hub saw in may when they ran an analysis of how developers, uh, commit times and a level of work that they were committing and how they were working, uh, in the first couple of months of COVID was, was progressing. They found that developers, at least in the Pacific time zone were actually increasing their work volume, maybe because they didn't have two hour commutes or maybe because they were stuck away in their homes, but for whatever reason, they were doing more work. >>And it's almost like, you know, if you've ever run a marathon the first mile or two in the marathon, you feel great and you just want to run and you want to power through it and you want to go hard. And if you do that by the time you get to mile 18 or 19, you're going to be gassed. It's sucking for wind. Uh, and, and that's, I think where we're starting to hit. So as we start to, um, gear our development chops out for the reality that most of us won't be returning into an office until 2021 at the earliest and many organizations will, will be fundamentally changing, uh, their remote workforce, uh, policies. We have to make sure that the agile processes that we use and the dev ops processes and tools that we use to support these teams are essentially aligned to help developers run that marathon instead of just kind of power through. >>So, um, let me give you a couple of specifics for many organizations, they have been in an environment where they will, um, tolerate Rover remote work and what I would call remote work around the edges like developers can be remote, but product managers and, um, you know, essentially scrum masters and all the administrators that are running the, uh, uh, the SCM repositories and, and the dev ops pipelines are all in the office. And it's essentially centralized work. That's not, we are anymore. We're moving from remote workers at the edge to remote workers at the center of what we do. And so one of the implications of that is that, um, we have to think about all the activities that you need to do from a dev ops perspective or from an agile perspective, they have to be remote people. One of the things I found with some of the organizations I talked to early on was there were things that administrators had to do that required them to go into the office to reboot the SCM server as an example, or to make sure that the final approvals for production, uh, were made. >>And so the code could be moved into the production environment. And so it actually was a little bit difficult because they had to get specific approval from the HR organizations to actually be allowed to go into the office in some States. And so one of the, the results of that is that while we've traditionally said, you know, tools are important, but they're not as important as culture as structure as organization as process. I think we have to rethink that a little bit because to the extent that tools enable us to be more digitally organized and to hiring, you know, achieve higher levels of digitization in our processes and be able to support the idea of remote workers in the center. They're now on an equal footing with so many of the other levers, uh, that, that, um, uh, that organizations have at their disposal. Um, I'll give you another example for years. >>We've said that the key to success with agile at the team level is cross-functional co located teams that are working together physically co located. It's the easiest way to show agile success. We can't do that anymore. We can't be physically located at least for the foreseeable future. So, you know, how do you take the low hanging fruits of an agile transformation and apply it in, in, in, in the time of COVID? Well, I think what you have to do is that you have to look at what physical co-location has enabled in the past and understand that it's not so much the fact that we're together looking at each other across the table. It's the fact that we're able to get into a shared mindspace, uh, from, um, uh, from a measurement perspective, we can have shared purpose. We can engage in high bandwidth communications. It's the spiritual aspect of that physical co-location that is actually important. So one of the biggest things that organizations need to start to ask themselves is how do we achieve spiritual colocation with our agile teams? Because we don't have the, the ease of physical co-location available to us anymore? >>Well, the spiritual co-location is such an interesting kind of provocative phrase there, but something that probably was a challenge here, we are seven, eight months in for many organizations, as you say, going from, you know, physical workspaces, co-location being able to collaborate face to face to a, a light switch flip overnight. And this undefined period of time where all we were living with with was uncertainty, how does spiritual, what do you, when you talk about spiritual co-location in terms of collaboration and processes and technology help us unpack that, and how are you seeing organizations adopted? >>Yeah, it's, it's, um, it's a great question. And, and I think it goes to the very root of how organizations are trying to transform themselves to be more agile and to embrace dev ops. Um, if you go all the way back to the, to the original, uh, agile manifesto, you know, there were four principles that were espoused individuals and interactions over processes and tools. That's still important. Individuals and interactions are at the core of software development, processes and tools that support those individual and interact. Uh, those individuals in those interactions are more important than ever working software over comprehensive documentation. Working software is still more important, but when you are trying to onboard employees and they can't come into the office and they can't do the two day training session and kind of understand how things work and they can't just holler over the cube, uh, to ask a question, you may need to invest a little bit more in documentation to help that onboarding process be successful in a remote context, uh, customer collaboration over contract negotiation. >>Absolutely still important, but employee collaboration is equally as important if you want to be spiritually, spiritually co-located. And if you want to have a shared purpose and then, um, responding to change over following a plan. I think one of the things that's happened in a lot of organizations is we have focused so much of our dev ops effort around velocity getting faster. We need to run as fast as we can like that sprinter. Okay. You know, trying to just power through it as quickly as possible. But as we shift to, to the, to the marathon way of thinking, um, velocity is still important, but agility becomes even more important. So when you have to create an application in three weeks to do track and trace for your employees, agility is more important. Um, and then just flat out velocity. Um, and so changing some of the ways that we think about dev ops practices, um, is, is important to make sure that that agility is there for one thing, you have to defer decisions as far down the chain to the team level as possible. >>So those teams have to be empowered to make decisions because you can't have a program level meeting of six or seven teams and one large hall and say, here's the lay of the land. Here's what we're going to do here are our processes. And here are our guardrails. Those teams have to make decisions much more quickly that developers are actually developing code in smaller chunks of flow. They have to be able to take two hours here or 50 minutes there and do something useful. And so the tools that support us have to become tolerant of the reality of, of, of, of how we're working. So if they work in a way that it allows the team together to take as much autonomy as they can handle, um, to, uh, allow them to communicate in a way that, that, that delivers shared purpose and allows them to adapt and master new technologies, then they're in the zone in their spiritual, they'll get spiritually connected. I hope that makes sense. >>It does. I think we all could use some of that, but, you know, you talked about in the beginning and I've, I've talked to numerous companies during the pandemic on the cube about the productivity, or rather the number of hours of work has gone way up for many roles, you know, and, and, and times that they normally late at night on the weekends. So, but it's a cultural, it's a mind shift to your point about dev ops focused on velocity, sprints, sprints, sprints, and now we have to, so that cultural shift is not an easy one for developers. And even at this folks to flip so quickly, what have you seen in terms of the velocity at which businesses are able to get more of that balance between the velocity, the sprint and the agility? >>I think, I think at the core, this really comes down to management sensitivity. Um, when everybody was in the office, you could kind of see the mental health of development teams by, by watching how they work. You know, you call it management by walking around, right. We can't do that. Managers have to, um, to, to be more aware of what their teams are doing, because they're not going to see that, that developer doing a check-in at 9:00 PM on a Friday, uh, because that's what they had to do, uh, to meet the objectives. And, um, and, and they're going to have to, to, um, to find new ways to measure engagement and also potential burnout. Um, friend of mine once had, uh, had a great metric that he called the parking lot metric. It was helpful as the parking lot at nine. And how full was it at five? >>And that gives you an indication of how engaged your developers are. Um, what's the digital equivalent equivalent to the parking lot metric in the time of COVID it's commit stats, it's commit rates. It's, um, you know, the, uh, the turn rate, uh, that we have in our code. So we have this information, we may not be collecting it, but then the next question becomes, how do we use that information? Do we use that information to say, well, this team isn't delivering as at the same level of productivity as another team, do we weaponize that data or do we use that data to identify impedances in the process? Um, why isn't a team working effectively? Is it because they have higher levels of family obligations and they've got kids that, that are at home? Um, is it because they're working with, um, you know, hardware technology, and guess what, they, it's not easy to get the hardware technology into their home office because it's in the lab at the, uh, at the corporate office, uh, or they're trying to communicate, uh, you know, halfway around the world. >>And, uh, they're communicating with a, with an office lab that is also shut down and, and, and the bandwidth just doesn't enable the, the level of high bandwidth communications. So from a dev ops perspective, managers have to get much more sensitive to the, the exhaust that the dev ops tools are throwing off, but also how they're going to use that in a constructive way to, to prevent burnout. And then they also need to, if they're not already managing or monitoring or measuring the level of developer engagement, they have, they really need to start whether that's surveys around developer satisfaction, um, whether it's, you know, more regular social events, uh, where developers can kind of just get together and drink a beer and talk about what's going on in the project, uh, and monitoring who checks in and who doesn't, uh, they have to, to, um, work harder, I think, than they ever have before. >>Well, and you mentioned burnout, and that's something that I think we've all faced in this time at varying levels and it changes. And it's a real, there's a tension in the air, regardless of where you are. There's a challenge, as you mentioned, people having, you know, coworker, their kids as coworkers and fighting for bandwidth, because everyone is forced in this situation. I'd love to get your perspective on some businesses that are, that have done this well, this adaptation, what can you share in terms of some real-world examples that might inspire the audience? >>Yeah. Uh, I'll start with, uh, stack overflow. Uh, they recently published a piece in the journal of the ACM around some of the things that they had discovered. Um, you know, first of all, just a cultural philosophy. If one person is remote, everybody is remote. And you just think that way from an executive level, um, social spaces. One of the things that they talk about doing is leaving a video conference room open at a team level all day long, and the team members, you know, we'll go on mute, you know, so that they don't have to, that they don't necessarily have to be there with somebody else listening to them. But if they have a question, they can just pop off mute really quickly and ask the question. And if anybody else knows the answer, it's kind of like being in that virtual pod. Uh, if you, uh, if you will, um, even here at Forrester, one of the things that we've done is we've invested in social ceremonies. >>We've actually moved our to our team meetings on, on my analyst team from, from once every two weeks to weekly. And we have built more time in for social Ajay socialization, just so we can see, uh, how, how, how we're doing. Um, I think Microsoft has really made some good, uh, information available in how they've managed things like the onboarding process. I think I'm Amanda silver over there mentioned that a couple of weeks ago when, uh, uh, a presentation they did that, uh, uh, Microsoft onboarded over 150,000 people since the start of COVID, if you don't have good remote onboarding processes, that's going to be a disaster. Now they're not all developers, but if you think about it, um, everything from how you do the interviewing process, uh, to how you get people, their badges, to how they get their equipment. Um, security is a, is another issue that they called out typically, uh, it security, um, the security of, of developers machines ends at, at, at the corporate desktop. >>But, you know, since we're increasingly using our own machines, our own hardware, um, security organizations kind of have to extend their security policies to cover, uh, employee devices, and that's caused them to scramble a little bit. Uh, so, so the examples are out there. It's not a lot of, like, we have to do everything completely differently, but it's a lot of subtle changes that, that have to be made. Um, I'll give you another example. Um, one of the things that, that we are seeing is that, um, more and more organizations to deal with the challenges around agility, with respect to delivering software, embracing low-code tools. In fact, uh, we see about 50% of firms are using low-code tools right now. We predict it's going to be 75% by the end of next year. So figuring out how your dev ops processes support an organization that might be using Mendix or OutSystems, or, you know, the power platform building the front end of an application, like a track and trace application really, really quickly, but then hooking it up to your backend infrastructure. Does that happen completely outside the dev ops investments that you're making and the agile processes that you're making, or do you adapt your organization? Um, our hybrid teams now teams that not just have professional developers, but also have business users that are doing some development with a low-code tool. Those are the kinds of things that we have to be, um, willing to, um, to entertain in order to shift the focus a little bit more toward the agility side, I think >>Lot of obstacles, but also a lot of opportunities for businesses to really learn, pay attention here, pivot and grow, and hopefully some good opportunities for the developers and the business folks to just get better at what they're doing and learning to embrace spiritual co-location Jeffrey, thank you so much for joining us on the program today. Very insightful conversation. >>My pleasure. It's it's, it's an important thing. Just remember if you're going to run that marathon, break it into 26, 10 minute runs, take a walk break in between each and you'll find that you'll get there. >>Digestible components, wise advice. Jeffery Hammond. Thank you so much for joining for Jeffrey I'm Lisa Martin, you're watching Broadcom's dev ops virtual forum >>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom, >>Continuing our conversations here at Broadcom's dev ops virtual forum. Lisa Martin here, please. To welcome back to the program, Serge Lucio, the general manager of the enterprise software division at Broadcom. Hey, Serge. Welcome. Thank you. Good to be here. So I know you were just, uh, participating with the biz ops manifesto that just happened recently. I just had the chance to talk with Jeffrey Hammond and he unlocked this really interesting concept, but I wanted to get your thoughts on spiritual co-location as really a necessity for biz ops to succeed in this unusual time in which we're living. What are your thoughts on spiritual colocation in terms of cultural change versus adoption of technologies? >>Yeah, it's a, it's, it's quite interesting, right? When we, when we think about the major impediments for, uh, for dev ops implementation, it's all about culture, right? And swore over the last 20 years, we've been talking about silos. We'd be talking about the paradox for these teams to when it went to align in many ways, it's not so much about these teams aligning, but about being in the same car in the same books, right? It's really about fusing those teams around kind of the common purpose, a common objective. So to me, the, this, this is really about kind of changing this culture where people start to look at a kind of OKR is instead of the key objective, um, that, that drives the entire team. Now, what it means in practice is really that's, uh, we need to change a lot of behaviors, right? It's not about the Yarki, it's not about roles. It's about, you know, who can do what and when, and, uh, you know, driving a bias towards action. It also means that we need, I mean, especially in this school times, it becomes very difficult, right? To drive kind of a kind of collaboration between these teams. And so I think there there's a significant role that especially tools can play in terms of providing this complex feedback from teams to, uh, to be in that preface spiritual qualification. >>Well, and it talked about culture being, it's something that, you know, we're so used to talking about dev ops with respect to velocity, all about speed here. But of course this time everything changed so quickly, but going from the physical spaces to everybody being remote really does take it. It's very different than you can't replicate it digitally, but there are collaboration tools that can kind of really be essential to help that cultural shift. Right? >>Yeah. So 2020, we, we touch to talk about collaboration in a very mundane way. Like, of course we can use zoom. We can all get into, into the same room. But the point when I think when Jeff says spiritual, co-location, it's really about, we all share the same objective. Do we, do we have a niece who, for instance, our pipeline, right? When you talk about dev ops, probably we all started thinking about this continuous delivery pipeline that basically drives the automation, the orchestration across the team, but just thinking about a pipeline, right, at the end of the day, it's all about what is the meantime to beat back to these teams. If I'm a developer and a commit code, I don't, does it take where, you know, that code to be processed through pipeline pushy? Can I get feedback if I am a finance person who is funding a product or a project, what is my meantime to beat back? >>And so a lot of, kind of a, when we think about the pipeline, I think what's been really inspiring to me in the last year or so is that there is much more of an adoption of the Dora metrics. There is way more of a focus around value stream management. And to me, this is really when we talk about collaboration, it's really a balance. How do you provide the feedback to the different stakeholders across the life cycle in a very timely matter? And that's what we would need to get to in terms of kind of this, this notion of collaboration. It's not so much about people being in the same physical space. It's about, you know, when I checked in code, you know, to do I guess the system to automatically identify what I'm going to break. If I'm about to release some allegation, how can the system help me reduce my change pillar rates? Because it's, it's able to predict that some issue was introduced in the outpatient or work product. Um, so I think there's, there's a great role of technology and AI candidate Lynch to, to actually provide that new level of collaboration. >>So we'll get to AI in a second, but I'm curious, what are some of the, of the metrics you think that really matter right now is organizations are still in some form of transformation to this new almost 100% remote workforce. >>So I'll just say first, I'm not a big fan of metrics. Um, and the reason being that, you know, you can look at a change killer rate, right, or a lead time or cycle time. And those are, those are interesting metrics, right? The trend on metric is absolutely critical, but what's more important is you get to the root cause what is taught to you lean to that metric to degrade or improve or time. And so I'm much more interested and we, you know, fruit for Broadcom. Are we more interested in understanding what are the patterns that contribute to this? So I'll give you a very mundane example. You know, we know that cycle time is heavily influenced by, um, organizational boundaries. So, you know, we talk a lot about silos, but, uh, we we've worked with many of our customers doing value stream mapping. And oftentimes what you see is that really the boundaries of your organization creates a lot of idle time, right? So to me, it's less about the metrics. I think the door metrics are a pretty, you know, valid set metrics, but what's way more important is to understand what are the antiperspirants, what are the things that we can detect through the data that actually are affecting those metrics. And, uh, I mean, over the last 10, 20 years, we've learned a lot about kind of what are, what are the antiperspirants within our large enterprise customers. And there are plenty of them. >>What are some of the things that you're seeing now with respect to patterns that have developed over the last seven to eight months? >>So I think the two areas which clearly are evolving very quickly are on kind of the front end of the life cycle, where DevOps is more and more embracing value stream management value stream mapping. Um, and I think what's interesting is that in many ways the product is becoming the new silo. Uh, the notion of a product is very difficult by itself to actually define people are starting to recognize that a value stream is not its own little kind of Island. That in reality, when I define a product, this product, oftentimes as dependencies on our products and that in fact, you're looking at kind of a network of value streams, if you will. So, so even on that, and there is clearly kind of a new sets, if you will, of anti-patterns where products are being defined as a set of OTRs, they have interdependencies and you have have a new set of silos on the operands, uh, the Abra key movement to Israel and the SRE space where, um, I think there is a cultural clash while the dev ops side is very much embracing this notion of OTRs and value stream mapping and Belgium management. >>On the other end, you have the it operations teams. We still think business services, right? For them, they think about configure items, think about infrastructure. And so, you know, it's not uncommon to see, you know, teams where, you know, the operations team is still thinking about hundreds of thousands, tens of thousands of business services. And so the, the, there is there's this boundary where, um, I think, well, SRE is being put in place. And there's lots of thinking about what kind of metrics can be fined. I think, you know, going back to culture, I think there's a lot of cultural evolution that's still required for true operations team. >>And that's a hard thing. Cultural transformation in any industry pandemic or not is a challenging thing. You talked about, uh, AI and automation of minutes ago. How do you think those technologies can be leveraged by DevOps leaders to influence their successes and their ability to collaborate, maybe see eye to eye with the SRS? >>Yeah. Um, so th you're kind of too. So even for myself, as a leader of a, you know, 1500 people organization, there's a number of things I don't see right. On a daily basis. And, um, I think the, the, the, the technologies that we have at our disposal today from the AI are able to mind a lot of data and expose a lot of, uh, issues that's as leaders we may not be aware of. And some of the, some of these are pretty kind of easy to understand, right? We all think we're agile. And yet when you, when you start to understand, for instance, uh, what is the, what is the working progress right to during the sprint? Um, when you start to analyze the data you can detect, for instance, that maybe the teams are over committed, that there is too much work in progress. >>You can start to identify kind of, interdepencies either from a technology, from a people point of view, which were hidden, uh, you can start to understand maybe the change filler rates he's he is dragging. So I believe that there is a, there's a fundamental role to be played by the tools to, to expose again, these anti parents, to, to make these things visible to the teams, to be able to even compare teams. Right. One of the things that's, that's, uh, that's amazing is now we have access to tons of data, not just from a given customer, but across a large number of customers. And so we start to compare all of these teams kind of operate, and what's working, what's not working >>Thoughts on AI and automation as, as a facilitator of spiritual co-location. >>Yeah, absolutely. Absolutely. It's um, you know, th there's, uh, the problem we all face is the unknown, right? The, the law city, but volume variety of the data, uh, everyday we don't really necessarily completely appreciate what is the impact of our actions, right? And so, um, AI can really act as a safety net that enables us to, to understand what is the impact of our actions. Um, and so, yeah, in many ways, the ability to be informed in a timely matter to be able to interact with people on the basis of data, um, and collaborate on the data. And the actual matter, I think is, is a, is a very powerful enabler, uh, on, in that respect. I mean, I, I've seen, um, I've seen countless of times that, uh, for instance, at the SRE boundary, um, to basically show that we'll turn the quality attributes, so an incoming release, right. And exposing that to, uh, an operations person and a sorry person, and enabling that collaboration dialogue through data is a very, very powerful tool. >>Do you have any recommendations for how teams can use, you know, the SRE folks, the dev ops says can use AI and automation in the right ways to be successful rather than some ways that aren't going to be nonproductive. >>Yeah. So to me, the th there, there's a part of the question really is when, when we talk about data, there are there different ways you can use data, right? Um, so you can, you can do a lot of an analytics, predictive analytics. So I think there is a, there's a tendency, uh, to look at, let's say a, um, a specific KPI, like a, an availability KPI, or change filler rate, and to basically do a regression analysis and projecting all these things, going to happen in the future. To me, that that's, that's a, that's a bad approach. The reason why I fundamentally think it's a better approach is because we are systems. The way we develop software is, is a, is a non-leader kind of system, right? Software development is not linear nature. And so I think there's a D this is probably the worst approach is to actually focus on metrics on the other end. >>Um, if you, if you start to actually understand at a more granular level, what har, uh, which are the things which are contributing to this, right? So if you start to understand, for instance, that whenever maybe, you know, you affect a specific part of the application that translates into production issues. So we, we have, I've actually, uh, a customer who, uh, identified that, uh, over 50% of their unplanned outages were related to specific components in your architecture. And whenever these components were changed, this resulted in these plant outages. So if you start to be able to basically establish causality, right, cause an effect between kind of data across the last cycle. I think, I think this is the right way to, uh, to, to use AI. And so pharma to be, I think it's way more God could have a classification problem. What are the classes of problems that do exist and affect things as opposed to analytics, predictive, which I don't think is as powerful. >>So I mentioned in the beginning of our conversation, that just came off the biz ops manifesto. You're one of the authors of that. I want to get your thoughts on dev ops and biz ops overlapping, complimenting each other, what, from a, the biz ops perspective, what does it mean to the future of dev ops? >>Yeah, so, so it's interesting, right? If you think about DevOps, um, there's no felony document, right? Can we, we can refer to the Phoenix project. I mean, there are a set of documents which have been written, but in many ways, there's no clear definition of what dev ops is. Uh, if you go to the dev ops Institute today, you'll see that they are specific, um, trainings for instance, on value management on SRE. And so in many ways, the problem we have as an industry is that, um, there are set practices between agile dev ops, SRE Valley should management. I told, right. And we all basically talk about the same things, right. We all talk about essentially, um, accelerating in the meantime fee to feedback, but yet we don't have the common framework to talk about that. The other key thing is that we add to wait, uh, for, uh, for jeans, Jean Kim's Lascaux, um, to, uh, to really start to get into the business aspect, right? >>And for value stream mapping to start to emerge for us to start as an industry, right. It, to start to think about what is our connection with the business aspect, what's our purpose, right? And ultimately it's all about driving these business outcomes. And so to me, these ops is really about kind of, uh, putting a lens on this critical element that it's not business and it, that we in fact need to fuse business 19 that I need needs to transform itself to recognize that it's, it's this value generator, right. It's not a cost center. And so the relationship to me, it's more than BizOps provides kind of this Oliver or kind of framework, if you will. That set the context for what is the reason, uh, for it to exist. What's part of the core values and principles that it needs to embrace to, again, change from a cost center to a value center. And then we need to start to use this as a way to start to unify some of the, again, the core practices, whether it's agile, DevOps value, stream mapping SRE. Um, so, so I think over time, my hope is that we start to optimize a lot of our practices, language, um, and, uh, and cultural elements. >>Last question surgeon, the last few seconds we have here talking about this, the relation between biz ops and dev ops, um, what do you think as DevOps evolves? And as you talked to circle some of your insights, what should our audience keep their eyes on in the next six to 12 months? >>So to me, the key, the key, um, challenge for, for the industry is really around. So we were seeing a very rapid shift towards kind of, uh, product to product, right. Which we don't want to do is to recreate kind of these new silos, these hard silos. Um, so that, that's one of the big changes, uh, that I think we need to be, uh, to be really careful about, um, because it is ultimately, it is about culture. It's not about, uh, it's not about, um, kind of how we segment the work, right. And, uh, any true culture that we can overcome kind of silos. So back to, I guess, with Jeffrey's concept of, um, kind of the spiritual co-location, I think it's, it's really about that too. It's really about kind of, uh, uh, focusing on the business outcomes on kind of aligning on driving engagement across the teams, but, but not for create a, kind of a new set of silos, which instead of being vertical are going to be these horizontal products >>Crazy by surge that looking at culture as kind of a way of really, uh, uh, addressing and helping to, uh, re re reduce, replace challenges. We thank you so much for sharing your insights and your time at today's DevOps virtual forum. >>Thank you. Thanks for your time. >>I'll be right back >>From around the globe it's the cube with digital coverage of devops virtual forum brought to you by Broadcom. >>Welcome to Broadcom's DevOps virtual forum, I'm Lisa Martin, and I'm joined by another Martin, very socially distanced from me all the way coming from Birmingham, England is Glynn Martin, the head of QA transformation at BT. Glynn, it's great to have you on the program. Thank you, Lisa. I'm looking forward to it. As we said before, we went live to Martins for the person one in one segment. So this is going to be an interesting segment guys, what we're going to do is Glynn's going to give us a really kind of deep inside out view of devops from an evolution perspective. So Glynn, let's start. Transformation is at the heart of what you do. It's obviously been a very transformative year. How have the events of this year affected the >> transformation that you are still responsible for driving? Yeah. Thank you, Lisa. I mean, yeah, it has been a difficult year. >>Um, and although working for BT, which is a global telecommunications company, um, I'm relatively resilient, I suppose, as a, an industry, um, through COVID obviously still has been affected and has got its challenges. And if anything, it's actually caused us to accelerate our transformation journey. Um, you know, we had to do some great things during this time around, um, you know, in the UK for our emergency and, um, health workers give them unlimited data and for vulnerable people to support them. And that's spent that we've had to deliver changes quickly. Um, but what we want to be able to do is deliver those kinds of changes quickly, but sustainably for everything that we do, not just because there's an emergency. Um, so we were already on the kind of journey to agile, but ever more important now that we are, we are able to do those, that kind of work, do it more quickly. >>Um, and that it works because the, the implications of it not working is, can be terrible in terms of you know, we've been supporting testing centers,  new hospitals to treat COVID patients. So we need to get it right. And then therefore the coverage of what we do, the quality of what we do and how quickly we do it really has taken on a new scale and what was already a very competitive market within the telco industry within the UK. Um, you know, what I would say is that, you know, we are under pressure to deliver more value, but we have small cost challenges. We have to obviously, um, deal with the fact that, you know, COVID 19 has hit most industries kind of revenues and profits. So we've got this kind of paradox between having less costs, but having to deliver more value quicker and  to higher quality. So yeah, certainly the finances is, um, on our minds and that's why we need flexible models, cost models that allow us to kind of do growth, but we get that growth by showing that we're delivering value. Um, especially in these times when there are financial challenges on companies. So one of the things that I want to ask you about, I'm again, looking at DevOps from the inside >>Out and the evolution that you've seen, you talked about the speed of things really accelerating in this last nine months or so. When we think dev ops, we think speed. But one of the things I'd love to get your perspective on is we've talked about in a number of the segments that we've done for this event is cultural change. What are some of the things that you've seen there as, as needing to get, as you said, get things right, but done so quickly to support essential businesses, essential workers. How have you seen that cultural shift? >>Yeah, I think, you know, before test teams for themselves at this part of the software delivery cycle, um, and actually now really our customers are expecting that quality and to deliver for our customers what they want, quality has to be ingrained throughout the life cycle. Obviously, you know, there's lots of buzzwords like shift left. Um, how do we do shift left testing? Um, but for me, that's really instilling quality and given capabilities shared capabilities throughout the life cycle that drive automation, drive improvements. I always say that, you know, you're only as good as your lowest common denominator. And one thing that we were finding on our dev ops journey was that we  would be trying to do certain things quick, we had automated build, automated tests. But if we were taking a weeks to create test scripts, or we were taking weeks to manually craft data, and even then when we had taken so long to do it, that the coverage was quite poor and that led to lots of defects later on in the life cycle, or even in our production environment, we just couldn't afford to do that. >>And actually, focusing on continuous testing over the last nine to 12 months has really given us the ability to deliver quickly across the whole life cycle. And therefore actually go from doing a kind of semi agile kind of thing, where we did the user stories, we did a few of the kind of agile ceremonies, but we weren't really deploying any quicker into production because our stakeholders were scared that we didn't have the same control that we had when we had more waterfall releases. And, you know, when we didn't think of ourselves. So we've done a lot of work on every aspect, um, especially from a testing point of view, every aspect of every activity, rather than just looking at automated tests, you know, whether it is actually creating the test in the first place, whether it's doing security testing earlier in the lot and performance testing in the life cycle, et cetera. So, yeah,  it's been a real key thing that for CT, for us to drive DevOps, >>Talk to me a little bit about your team. What are some of the shifts in terms of expectations that you're experiencing and how your team interacts with the internal folks from pipeline through life cycle? >>Yeah, we've done a lot of work on this. Um, you know, there's a thing that I think people will probably call it a customer experience gap, and it reminds me of a Gilbert cartoon, where we start with the requirements here and you're almost like a Chinese whisper effects and what we deliver is completely different. So we think the testing team or the delivery teams, um, know in our teeth has done a great job. This is what it said in the acceptance criteria, but then our customers are saying, well, actually that's not working this isn't working and there's this kind of gap. Um, we had a great launch this year of agile requirements, it's one of the Broadcom tools. And that was the first time in, ever since I remember actually working within BT, I had customers saying to me, wow, you know, we want more of this. >>We want more projects to have extra requirements design on it because it allowed us to actually work with the business collaboratively. I mean, we talk about collaboration, but how do we actually, you know, do that and have something that both the business and technical people can understand. And we've actually been working with the business , using agile requirements designer to really look at what the requirements are, tease out requirements we hadn't even thought of and making sure that we've got high levels of test coverage. And what we actually deliver at the end of it, not only have we been able to generate tests more quickly, but we've got much higher test coverage and also can more smartly, using the kind of AI within the tool and then some of the other kinds of pipeline tools, actually deliver to choose the right tasks, and actually doing a risk based testing approach. So that's been a great launch this year, but just the start of many kinds of things that we're doing >>Well, what I hear in that, Glynn is a lot of positives that have come out of a very challenging situation. Talk to me about it. And I liked that perspective. This is a very challenging time for everybody in the world, but it sounds like from a collaboration perspective you're right, we talk about that a lot critical with devops. But those challenges there, you guys were able to overcome those pretty quickly. What other challenges did you face and figure out quickly enough to be able to pivot so fast? >>I mean, you talked about culture. You know, BT is like most companies  So it's very siloed. You know we're still trying to work to become closer as a company. So I think there's a lot of challenges around how would you integrate with other tools? How would you integrate with the various different technologies. And BT, we have 58 different IT stacks. That's not systems, that's stacks, all of those stacks can have hundreds of systems. And we're trying to, we've got a drive at the moment, a simplified program where we're trying to you know, reduce that number to 14 stacks. And even then there'll be complexity behind the scenes that we will be challenged more and more as we go forward. How do we actually highlight that to our users? And as an it organization, how do we make ourselves leaner, so that even when we've still got some of that legacy, and we'll never fully get rid of it and that's the kind of trade off that we have to make, how do we actually deal with that and hide that from our users and drive those programs, so we can, as I say, accelerate change,  reduce that kind of waste and that kind of legacy costs out of our business. You know, the other thing as well, I'm sure telecoms is probably no different to insurance or finance. When you take the number of products that we do, and then you combine them, the permutations are tens and hundreds of thousands of products. So we, as a business are trying to simplify, we are trying to do that in an agile way. >>And haven't tried to do agile in the proper way and really actually work at pace, really deliver value. So I think what we're looking more and more at the moment is actually  more value focused. Before we used to deliver changes sometimes into production. Someone had a great idea, or it was a great idea nine months ago or 12 months ago, but actually then we ended up deploying it and then we'd look at the users, the usage of that product or that application or whatever it is, and it's not being used for six months. So we haven't got, you know, the cost of the last 12 months. We certainly haven't gotten room for that kind of waste and, you know, for not really understanding the value of changes that we are doing. So I think that's the most important thing of the moment, it's really taking that waste out. You know, there's lots of focus on things like flow management, what bits of our process are actually taking too long. And we've started on that journey, but we've got a hell of a long way to go. But that involves looking at every aspect of the software delivery cycle. >> Going from, what 58 IT stacks down to 14 or whatever it's going to be, simplifying sounds magical to everybody. It's a big challenge. What are some of the core technology capabilities that you see really as kind of essential for enabling that with this new way that you're working? >>Yeah. I mean, I think we were started on a continuous testing journey, and I think that's just the start. I mean as I say, looking at every aspect of, you know, from a QA point of view is every aspect of what we do. And it's also looking at, you know, we've started to branch into more like AI, uh, AI ops and, you know, really the full life cycle. Um, and you know, that's just a stepping stone to, you know, I think autonomics is the way forward, right. You know, all of this kind of stuff that happens, um, you know, monitoring, uh, you know, watching the systems what's happening in production, how do we feed that back? How'd you get to a point where actually we think about change and then suddenly it's in production safely, or if it's not going to safety, it's automatically backing out. So, you know, it's a very, very long journey, but if we want to, you know, in a world where the pace is in ever-increasing and the demands for the team, and, you know, with the pressures on, at the moment where we're being asked to do things, uh, you know, more efficiently and as lean as possible, we need to be thinking about every part of the process and how we put the kind of stepping stones in place to lead us to a more automated kind of, um, you know, um, the future. >>Do you feel that that planned outcomes are starting to align with what's delivered, given this massive shift that you're experiencing? >>I think it's starting to, and I think, you know, as I say, as we look at more of a value based approach, um, and, um, you know, as I say, print, this was a kind of flow management. I think that that will become ever, uh, ever more important. So, um, I think it starting to people certainly realize that, you know, teams need to work together, you know, the kind of the cousin between business and it, especially as we go to more kind of SAS based solutions, low code solutions, you know, there's not such a gap anymore, actually, some of our business partners that expense to be much more tech savvy. Um, so I think, you know, this is what we have to kind of appreciate what is its role, how do we give the capabilities, um, become more of a centers of excellence rather than actually doing mounds amounts of work. And for me, and from a testing point of view, you know, mounds and mounds of testing, actually, how do we automate that? How do we actually generate that instead of, um, create it? I think that's the kind of challenge going forward. >>What are some, as we look forward, what are some of the things that you would like to see implemented or deployed in the next, say six to 12 months as we hopefully round a corner with this pandemic? >>Yeah, I think, um, you know, certainly for, for where we are as a company from a QA perspective, we are, um, you let's start in bits that we do well, you know, we've started creating, um, continuous delivery and DevOps pipelines. Um, there's still manual aspects of that. So, you know, certainly for me, I I've challenged my team with saying how do we do an automated journey? So if I put a requirement in JIRA or rally or wherever it is and why then click a button and, you know, with either zero touch for one such, then put that into production and have confidence that, that has been done safely and that it works and what happens if it doesn't work. So, you know, that's, that's the next, um, the next few months, that's what our concentration, um, is, is about. But it's also about decision-making, you know, how do you actually understand those value judgments? >>And I think there's lots of the things dev ops, AI ops, kind of that always ask aspects of business operations. I think it's about having the information in one place to make those kinds of decisions. How does it all try and tie it together? As I say, even still with kind of dev ops, we've still got elements within my company where we've got lots of different organizations doing some, doing similar kinds of things, but they're all kind of working in silos. So I think having AI ops as it comes more and more to the fore as we go to cloud, and that's what we need to, you know, we're still very early on in our cloud journey, you know, so we need to make sure the technologies work with cloud as well as you can have, um, legacy systems, but it's about bringing that all together and having a full, visible pipeline, um, that everybody can see and make decisions. >>You said the word confidence, which jumped out at me right away, because absolutely you've got to have be able to have confidence in what your team is delivering and how it's impacting the business and those customers. Last question then for you is how would you advise your peers in a similar situation to leverage technology automation, for example, dev ops, to be able to gain the confidence that they're making the right decisions for their business? >>I think the, the, the, the, the approach that we've taken actually is not started with technology. Um, we've actually taken a human centered design, uh, as a core principle of what we do, um, within the it part of BT. So by using human centered design, that means we talk to our customers, we understand their pain points, we map out their current processes. Um, and then when we mapped out what this process does, it also understand their aspirations as well, you know? Um, and where do they want to be in six months? You know, do they want it to be, um, more agile and, you know, or do they want to, you know, is, is this a part of their business that they want to do one better? We actually then looked at why that's not running well, and then see what, what solutions are out there. >>We've been lucky that, you know, with our partnership, with Broadcom within the payer line, lots of the tools and the PLA have directly answered some of the business's problems. But I think by having those conversations and actually engaging with the business, um, you know, especially if the business hold the purse strings, which in, in, uh, you know, in some companies include not as they do there is that kind of, you know, almost by understanding their, their pain points and then starting, this is how we can solve your problem. Um, is we've, we've tended to be much more successful than trying to impose something and say, well, here's the technology that they don't quite understand. It doesn't really understand how it kind of resonates with their problems. So I think that's the heart of it. It's really about, you know, getting, looking at the data, looking at the processes, looking at where the kind of waste is. >>And then actually then looking at the right solutions. Then, as I say, continuous testing is massive for us. We've also got a good relationship with Apple towards looking at visual AI. And actually there's a common theme through that. And I mean, AI is becoming more and more prevalent. And I know, you know, sometimes what is AI and people have kind of this semantics of, is it true AI or not, but it's certainly, you know, AI machine learning is becoming more and more prevalent in the way that we work. And it's allowing us to be much more effective, be quicker in what we do and be more accurate. And, you know, whether it's finding defects running the right tests or, um, you know, being able to anticipate problems before they're happening in a production environment. >>Well, thank you so much for giving us this sort of insight outlook at dev ops sharing the successes that you're having, taking those challenges, converting them to opportunities and forgiving folks who might be in your shoes, or maybe slightly behind advice enter. They appreciate it. We appreciate your time. >>Well, it's been an absolute pleasure, really. Thank you for inviting me. I have a extremely enjoyed it. So thank you ever so much. >>Excellent. Me too. I've learned a lot for Glenn Martin. I'm Lisa Martin. You're watching the cube >>Driving revenue today means getting better, more valuable software features into the hands of your customers. If you don't do it quickly, your competitors as well, but going faster without quality creates risks that can damage your brand destroy customer loyalty and cost millions to fix dev ops from Broadcom is a complete solution for balancing speed and risk, allowing you to accelerate the flow of value while minimizing the risk and severity of critical issues with Broadcom quality becomes integrated across the entire DevOps pipeline from planning to production, actionable insights, including our unique readiness score, provide a three 60 degree view of software quality giving you visibility into potential issues before they become disasters. Dev ops leaders can manage these risks with tools like Canary deployments tested on a small subset of users, or immediately roll back to limit the impact of defects for subsequent cycles. Dev ops from Broadcom makes innovation improvement easier with integrated planning and continuous testing tools that accelerate the flow of value product requirements are used to automatically generate tests to ensure complete quality coverage and tests are easily updated. >>As requirements change developers can perform unit testing without ever leaving their preferred environment, improving efficiency and productivity for the ultimate in shift left testing the platform also integrates virtual services and test data on demand. Eliminating two common roadblocks to fast and complete continuous testing. When software is ready for the CIC CD pipeline, only DevOps from Broadcom uses AI to prioritize the most critical and relevant tests dramatically improving feedback speed with no decrease in quality. This release is ready to go wherever you are in your DevOps journey. Broadcom helps maximize innovation velocity while managing risk. So you can deploy ideas into production faster and release with more confidence from around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi guys. Welcome back. So we have discussed the current state and the near future state of dev ops and how it's going to evolve from three unique perspectives. In this last segment, we're going to open up the floor and see if we can come to a shared understanding of where dev ops needs to go in order to be successful next year. So our guests today are, you've seen them all before Jeffrey Hammond is here. The VP and principal analyst serving CIO is at Forester. We've also Serge Lucio, the GM of Broadcom's enterprise software division and Glenn Martin, the head of QA transformation at BT guys. Welcome back. Great to have you all three together >>To be here. >>All right. So we're very, we're all very socially distanced as we've talked about before. Great to have this conversation. So let's, let's start with one of the topics that we kicked off the forum with Jeff. We're going to start with you spiritual co-location that's a really interesting topic that we've we've uncovered, but how much of the challenge is truly cultural and what can we solve through technology? Jeff, we'll start with you then search then Glen Jeff, take it away. >>Yeah, I think fundamentally you can have all the technology in the world and if you don't make the right investments in the cultural practices in your development organization, you still won't be effective. Um, almost 10 years ago, I wrote a piece, um, where I did a bunch of research around what made high-performance teams, software delivery teams, high performance. And one of the things that came out as part of that was that these teams have a high level of autonomy. And that's one of the things that you see coming out of the agile manifesto. Let's take that to today where developers are on their own in their own offices. If you've got teams where the team itself had a high level of autonomy, um, and they know how to work, they can make decisions. They can move forward. They're not waiting for management to tell them what to do. >>And so what we have seen is that organizations that embraced autonomy, uh, and got their teams in the right place and their teams had the information that they needed to make the right decisions have actually been able to operate pretty well, even as they've been remote. And it's turned out to be things like, well, how do we actually push the software that we've created into production that would become the challenge is not, are we writing the right software? And that's why I think the term spiritual co-location is so important because even though we may be physically distant, we're on the same plane, we're connected from a, from, from a, a shared purpose. Um, you know, surgeon, I worked together a long, long time ago. So it's been what almost 15, 16 years since we were at the same place. And yet I would say there's probably still a certain level of spiritual co-location between us, uh, because of the shared purposes that we've had in the past and what we've seen in the industry. And that's a really powerful tool, uh, to build on. So what do tools play as part of that, to the extent that tools make information available, to build shared purpose on to the extent that they enable communication so that we can build that spiritual co-location to the extent that they reinforce the culture that we want to put in place, they can be incredibly valuable, especially when, when we don't have the luxury of physical locate physical co-location. Okay. That makes sense. >>It does. I shouldn't have introduced us. This last segment is we're all spiritually co-located or it's a surge, clearly you're still spiritually co located with jump. Talk to me about what your thoughts are about spiritual of co-location the cultural impact and how technology can move it forward. >>Yeah. So I think, well, I'm going to sound very similar to Jeff in that respect. I think, you know, it starts with kind of a shared purpose and the other understanding, Oh, individuals teams, uh, contributed to kind of a business outcome, what is our shared goal or shared vision? What's what is it we're trying to achieve collectively and keeping it kind of aligned to that? Um, and so, so it's really starts with that now, now the big challenge, always these over the last 20 years, especially in large organization, there's been specialization of roles and functions. And so we, we all that started to basically measure which we do, uh, on a daily basis using metrics, which oftentimes are completely disconnected from kind of a business outcome or purpose. We, we kind of reverted back to, okay, what is my database all the time? What is my cycle time? >>Right. And, and I think, you know, which we can do or where we really should be focused as an industry is to start to basically provide a lens or these different stakeholders to look at what they're doing in the context of kind of these business outcomes. So, um, you know, probably one of my, um, favorites experience was to actually weakness at one of a large financial institution. Um, you know, Tuesday Golder's unquote development and operations staring at the same data, right. Which was related to, you know, in calming changes, um, test execution results, you know, Coverity coverage, um, official liabilities and all the all ran. It could have a direction level links. And that's when you start to put these things in context and represent that to you in a way that these different stakeholders can, can look at from their different lens. And, uh, and it can start to basically communicate and, and understand have they joined our company to, uh, to, to that kind of common view or objective. >>And Glen, we talked a lot about transformation with you last time. What are your thoughts on spiritual colocation and the cultural part, the technology impact? >>Yeah, I mean, I agree with Jeffrey that, you know, um, the people and culture, the most important thing, actually, that's why it's really important when you're transforming to have partners who have the same vision as you, um, who, who you can work with, have the same end goal in mind. And w I've certainly found that with our, um, you know, continuing relationship with Broadcom, what it also does though, is although, you know, tools can accelerate what you're doing and can join consistency. You know, we've seen within simplify, which is BTS flagship transformation program, where we're trying to, as it can, it says simplify the number of systems stacks that we have, the number of products that we have actually at the moment, we've got different value streams within that program who have got organizational silos. We were trying to rewrite, rewrite the wheel, um, who are still doing things manually. >>So in order to try and bring that consistency, we need the right tools that actually are at an enterprise grade, which can be flexible to work with in BT, which is such a complex and very dev, uh, different environments, depending on what area of BT you're in, whether it's a consumer, whether it's a mobile area, whether it's large global or government organizations, you know, we found that we need tools that can, um, drive that consistency, but also flex to Greenfield brownfield kind of technologies as well. So it's really important that as I say, for a number of different aspects, that you have the right partner, um, to drive the right culture, I've got the same vision, but also who have the tool sets to help you accelerate. They can't do that on their own, but they can help accelerate what it is you're trying to do in it. >>And a really good example of that is we're trying to shift left, which is probably a, quite a bit of a buzz phrase in their kind of testing world at the moment. But, you know, I could talk about things like continuous delivery direct to when a ball comes tools and it has many different features to it, but very simply on its own, it allows us to give the visibility of what the teams are doing. And once we have that visibility, then we can talk to the teams, um, around, you know, could they be doing better component testing? Could they be using some virtualized services here or there? And that's not even the main purpose of continuous delivery director, but it's just a reason that tools themselves can just give greater visibility of have much more intuitive and insightful conversations with other teams and reduce those organizational silos. >>Thanks, Ben. So we'd kind of sum it up, autonomy collaboration tools that facilitate that. So let's talk now about metrics from your perspectives. What are the metrics that matter? Jeff, >>I'm going to go right back to what Glenn said about data that provides visibility that enables us to, to make decisions, um, with shared purpose. And so business value has to be one of the first things that we look at. Um, how do we assess whether we have built something that is valuable, you know, that could be sales revenue, it could be net promoter score. Uh, if you're not selling what you've built, it could even be what the level of reuse is within your organization or other teams picking up the services, uh, that you've created. Um, one of the things that I've begun to see organizations do is to align value streams with customer journeys and then to align teams with those value streams. So that's one of the ways that you get to a shared purpose, cause we're all trying to deliver around that customer journey, the value with it. >>And we're all measured on that. Um, there are flow metrics which are really important. How long does it take us to get a new feature out from the time that we conceive it to the time that we can run our first experiments with it? There are quality metrics, um, you know, some of the classics or maybe things like defect, density, or meantime to response. Um, one of my favorites came from a, um, a company called ultimate software where they looked at the ratio of defects found in production to defects found in pre production and their developers were in fact measured on that ratio. It told them that guess what quality is your job to not just the test, uh, departments, a group, the fourth level that I think is really important, uh, in, in the current, uh, situation that we're in is the level of engagement in your development organization. >>We used to joke that we measured this with the parking lot metric helpful was the parking lot at nine. And how full was it at five o'clock. I can't do that anymore since we're not physically co-located, but what you can do is you can look at how folks are delivering. You can look at your metrics in your SCM environment. You can look at, uh, the relative rates of churn. Uh, you can look at things like, well, are our developers delivering, uh, during longer periods earlier in the morning, later in the evening, are they delivering, uh, you know, on the weekends as well? Are those signs that we might be heading toward a burnout because folks are still running at sprint levels instead of marathon levels. Uh, so all of those in combination, uh, business value, uh, flow engagement in quality, I think form the backbone of any sort of, of metrics, uh, a program. >>The second thing that I think you need to look at is what are we going to do with the data and the philosophy behind the data is critical. Um, unfortunately I see organizations where they weaponize the data and that's completely the wrong way to look at it. What you need to do is you need to say, you need to say, how is this data helping us to identify the blockers? The things that aren't allowing us to provide the right context for people to do the right thing. And then what do we do to remove those blockers, uh, to make sure that we're giving these autonomous teams the context that they need to do their job, uh, in a way that creates the most value for the customers. >>Great advice stuff, Glenn, over to your metrics that matter to you that really make a big impact. And, and, and also how do you measure quality kind of following onto the advice that Jeff provided? >>That's some great advice. Actually, he talks about value. He talks about flow. Both of those things are very much on my mind at the moment. Um, but there was this, I listened to a speaker, uh, called me Kirsten a couple of months ago. It taught very much around how important flow management is and removing, you know, and using that to remove waste, to understand in terms of, you know, making software changes, um, what is it that's causing us to do it longer than we need to. So where are those areas where it takes long? So I think that's a very important thing for us. It's even more basic than that at the moment, we're on a journey from moving from kind of a waterfall to agile. Um, and the problem with moving from waterfall to agile is with waterfall, the, the business had a kind of comfort that, you know, everything was tested together and therefore it's safer. >>Um, and with agile, there's that kind of, you know, how do we make sure that, you know, if we're doing things quick and we're getting stuff out the door that we give that confidence, um, that that's ready to go, or if there's a risk that we're able to truly articulate what that risk is. So there's a bit about release confidence, um, and some of the metrics around that and how, how healthy those releases are, and actually saying, you know, we spend a lot of money, um, um, an investment setting up our teams, training our teams, are we actually seeing them deliver more quickly and are we actually seeing them deliver more value quickly? So yeah, those are the two main things for me at the moment, but I think it's also about, you know, generally bringing it all together, the dev ops, you know, we've got the kind of value ops AI ops, how do we actually bring that together to so we can make quick decisions and making sure that we are, um, delivering the biggest bang for our buck, absolutely biggest bang for the buck, surge, your thoughts. >>Yeah. So I think we all agree, right? It starts with business metrics, flow metrics. Um, these are kind of the most important metrics. And ultimately, I mean, one of the things that's very common across a highly functional teams is engagements, right? When, when you see a team that's highly functioning, that's agile, that practices DevOps every day, they are highly engaged. Um, that that's, that's definitely true. Now the, you know, back to, I think, uh, Jeff's point on weaponization of metrics. One of the key challenges we see is that, um, organizations traditionally have been kind of, uh, you know, setting up benchmarks, right? So what is a good cycle time? What is a good lead time? What is a good meantime to repair? The, the problem is that this is very contextual, right? It varies. It's going to vary quite a bit, depending on the nature of application and system. >>And so one of the things that we really need to evolve, um, as an industry is to understand that it's not so much about those flow metrics is about our, these four metrics ultimately contribute to the business metric to the business outcome. So that's one thing. The second aspect, I think that's oftentimes misunderstood is that, you know, when you have a bad cycle time or, or, or what you perceive as being a buy cycle time or better quality, the problem is oftentimes like all, do you go and explore why, right. What is the root cause of this? And I think one of the key challenges is that we tend to focus a lot of time on metrics and not on the eye type patterns, which are pretty common across the industry. Um, you know, if you look at, for instance, things like lead time, for instance, it's very common that, uh, organizational boundaries are going to be a key contributor to badly time. >>And so I think that there is, you know, the only the metrics there is, I think a lot of work that we need to do in terms of classifying, descend type patterns, um, you know, back to you, Jeff, I think you're one of the cool offers of waterscrumfall as a, as, as a key pattern, the industry or anti-spatter. Um, but waterscrumfall right is a key one, right? And you will detect that through kind of a defect arrival rates. That's where that looks like an S-curve. And so I think it's beyond kind of the, the metrics is what do you do with those metrics? >>Right? I'll tell you a search. One of the things that is really interesting to me in that space is I think those of us had been in industry for a long time. We know the anti-patterns cause we've seen them in our career maybe in multiple times. And one of the things that I think you could see tooling do is perhaps provide some notification of anti-patterns based on the telemetry that comes in. I think it would be a really interesting place to apply, uh, machine learning and reinforcement learning techniques. Um, so hopefully something that we'd see in the future with dev ops tools, because, you know, as a manager that, that, you know, may be only a 10 year veteran or 15 year veteran, you may be seeing these anti-patterns for the first time. And it would sure be nice to know what to do, uh, when they start to pop up, >>That would right. Insight, always helpful. All right, guys, I would like to get your final thoughts on this. The one thing that you believe our audience really needs to be on the lookout for and to put on our agendas for the next 12 months, Jeff will go back to you. Okay. >>I would say look for the opportunities that this disruption presents. And there are a couple that I see, first of all, uh, as we shift to remote central working, uh, we're unlocking new pools of talent, uh, we're, it's possible to implement, uh, more geographic diversity. So, so look to that as part of your strategy. Number two, look for new types of tools. We've seen a lot of interest in usage of low-code tools to very quickly develop applications. That's potentially part of a mainstream strategy as we go into 2021. Finally, make sure that you embrace this idea that you are supporting creative workers that agile and dev ops are the peanut butter and chocolate to support creative, uh, workers with algorithmic capabilities, >>Peanut butter and chocolate Glen, where do we go from there? What are, what's the one silver bullet that you think folks to be on the lookout for now? I, I certainly agree that, um, low, low code is, uh, next year. We'll see much more low code we'd already started going, moving towards a more of a SAS based world, but low code also. Um, I think as well for me, um, we've still got one foot in the kind of cow camp. Um, you know, we'll be fully trying to explore what that means going into the next year and exploiting the capabilities of cloud. But I think the last, um, the last thing for me is how do you really instill quality throughout the kind of, um, the, the life cycle, um, where, when I heard the word scrum fall, it kind of made me shut it because I know that's a problem. That's where we're at with some of our things at the moment we need to get beyond that. We need >>To be releasing, um, changes more frequently into production and actually being a bit more brave and having the confidence to actually do more testing in production and go straight to production itself. So expect to see much more of that next year. Um, yeah. Thank you. I haven't got any food analogies. Unfortunately we all need some peanut butter and chocolate. All right. It starts to take us home. That's what's that nugget you think everyone needs to have on their agendas? >>That's interesting. Right. So a couple of days ago we had kind of a latest state of the DevOps report, right? And if you read through the report, it's all about the lost city, but it's all about sweet. We still are receiving DevOps as being all about speed. And so to me, the key advice is in order to create kind of a spiritual collocation in order to foster engagement, we have to go back to what is it we're trying to do collectively. We have to go back to tie everything to the business outcome. And so for me, it's absolutely imperative for organizations to start to plot their value streams, to understand how they're delivering value into aligning everything they do from a metrics to deliver it, to flow to those metrics. And only with that, I think, are we going to be able to actually start to really start to align kind of all these roles across the organizations and drive, not just speed, but business outcomes, >>All about business outcomes. I think you guys, the three of you could write a book together. So I'll give you that as food for thought. Thank you all so much for joining me today and our guests. I think this was an incredibly valuable fruitful conversation, and we appreciate all of you taking the time to spiritually co-located with us today, guys. Thank you. Thank you, Lisa. Thank you. Thank you for Jeff Hammond serves Lucio and Glen Martin. I'm Lisa Martin. Thank you for watching the broad cops Broadcom dev ops virtual forum.

>>Hello. I'm Professor Toru Suzuki Cherif cardiovascular medicine on associate dean of the College of Life Sciences at the University of Leicester in the United Kingdom, where I'm also director of the Lester Life Sciences accelerator. I'm also honorary consultant cardiologist within our university hospitals. It's part of the national health system NHS Trust. Today, I'd like to talk to you about our Lester Clinical Data Science Initiative. Now brief background on Lester. It's university in hospitals. Lester is in the center of England. The national health system is divided depending on the countries. The United Kingdom, which is comprised of, uh, England, Scotland to the north, whales to the west and Northern Ireland is another part in a different island. But national health system of England is what will be predominantly be discussed. Today has a history of about 70 years now, owing to the fact that we're basically in the center of England. Although this is only about one hour north of London, we have a catchment of about 100 miles, which takes us from the eastern coast of England, bordering with Birmingham to the west north just south of Liverpool, Manchester and just south to the tip of London. We have one of the busiest national health system trust in the United Kingdom, with a catchment about 100 miles and one million patients a year. Our main hospital, the General Hospital, which is actually called the Royal Infirmary, which can has an accident and emergency, which means Emergency Department is that has one of the busiest emergency departments in the nation. I work at Glen Field Hospital, which is one of the main cardiovascular hospitals of the United Kingdom and Europe. Academically, the Medical School of the University of Leicester is ranked 20th in the world on Lee, behind Cambridge, Oxford Imperial College and University College London. For the UK, this is very research. Waited, uh, ranking is Therefore we are very research focused universities as well for the cardiovascular research groups, with it mainly within Glenn Field Hospital, we are ranked as the 29th Independent research institution in the world which places us. A Suffield waited within our group. As you can see those their top ranked this is regardless of cardiology, include institutes like the Broad Institute and Whitehead Institute. Mitt Welcome Trust Sanger, Howard Hughes Medical Institute, Kemble, Cold Spring Harbor and as a hospital we rank within ah in this field in a relatively competitive manner as well. Therefore, we're very research focused. Hospital is well now to give you the unique selling points of Leicester. We're we're the largest and busiest national health system trust in the United Kingdom, but we also have a very large and stable as well as ethnically diverse population. The population ranges often into three generations, which allows us to do a lot of cohort based studies which allows us for the primary and secondary care cohorts, lot of which are well characterized and focused on genomics. In the past. We also have a biomedical research center focusing on chronic diseases, which is funded by the National Institutes of Health Research, which funds clinical research the hospitals of United Kingdom on we also have a very rich regional life science cluster, including med techs and small and medium sized enterprises. Now for this, the bottom line is that I am the director of the letter site left Sciences accelerator, >>which is tasked with industrial engagement in the local national sectors but not excluding the international sectors as well. Broadly, we have academics and clinicians with interest in health care, which includes science and engineering as well as non clinical researchers. And prior to the cove it outbreak, the government announced the £450 million investment into our university hospitals, which I hope will be going forward now to give you a brief background on where the scientific strategy the United Kingdom lies. Three industrial strategy was brought out a za part of the process which involved exiting the European Union, and part of that was the life science sector deal. And among this, as you will see, there were four grand challenges that were put in place a I and data economy, future of mobility, clean growth and aging society and as a medical research institute. A lot of the focus that we have been transitioning with within my group are projects are focused on using data and analytics using artificial intelligence, but also understanding how chronic diseases evolved as part of the aging society, and therefore we will be able to address these grand challenges for the country. Additionally, the national health system also has its long term plans, which we align to. One of those is digitally enabled care and that this hope you're going mainstream over the next 10 years. And to do this, what is envision will be The clinicians will be able to access and interact with patient records and care plants wherever they are with ready access to decision support and artificial intelligence, and that this will enable predictive techniques, which include linking with clinical genomic as well as other data supports, such as image ing a new medical breakthroughs. There has been what's called the Topol Review that discusses the future of health care in the United Kingdom and preparing the health care workforce for the delivery of the digital future, which clearly discusses in the end that we would be using automated image interpretation. Is using artificial intelligence predictive analytics using artificial intelligence as mentioned in the long term plans. That is part of that. We will also be engaging natural language processing speech recognition. I'm reading the genome amusing. Genomic announced this as well. We are in what is called the Midland's. As I mentioned previously, the Midland's comprised the East Midlands, where we are as Lester, other places such as Nottingham. We're here. The West Midland involves Birmingham, and here is ah collective. We are the Midlands. Here we comprise what is called the Midlands engine on the Midland's engine focuses on transport, accelerating innovation, trading with the world as well as the ultra connected region. And therefore our work will also involve connectivity moving forward. And it's part of that. It's part of our health care plans. We hope to also enable total digital connectivity moving forward and that will allow us to embrace digital data as well as collectivity. These three key words will ah Linkous our health care systems for the future. Now, to give you a vision for the future of medicine vision that there will be a very complex data set that we will need to work on, which will involve genomics Phanom ICS image ing which will called, uh oh mix analysis. But this is just meaning that is, uh complex data sets that we need to work on. This will integrate with our clinical data Platforms are bioinformatics, and we'll also get real time information of physiology through interfaces and wearables. Important for this is that we have computing, uh, processes that will now allow this kind of complex data analysis in real time using artificial intelligence and machine learning based applications to allow visualization Analytics, which could be out, put it through various user interfaces to the clinician and others. One of the characteristics of the United Kingdom is that the NHS is that we embrace data and captured data from when most citizens have been born from the cradle toe when they die to the grave. And it's important that we were able to link this data up to understand the journey of that patient. Over time. When they come to hospital, which is secondary care data, we will get disease data when they go to their primary care general practitioner, we will be able to get early check up data is Paula's follow monitoring monitoring, but also social care data. If this could be linked, allow us to understand how aging and deterioration as well as frailty, uh, encompasses thes patients. And to do this, we have many, many numerous data sets available, including clinical letters, blood tests, more advanced tests, which is genetics and imaging, which we can possibly, um, integrate into a patient journey which will allow us to understand the digital journey of that patient. I have called this the digital twin patient cohort to do a digital simulation of patient health journeys using data integration and analytics. This is a technique that has often been used in industrial manufacturing to understand the maintenance and service points for hardware and instruments. But we would be using this to stratify predict diseases. This'll would also be monitored and refined, using wearables and other types of complex data analysis to allow for, in the end, preemptive intervention to allow paradigm shifting. How we undertake medicine at this time, which is more reactive rather than proactive as infrastructure we are presently working on putting together what's it called the Data Safe haven or trusted research environment? One which with in the clinical environment, the university hospitals and curated and data manner, which allows us to enable data mining off the databases or, I should say, the trusted research environment within the clinical environment. Hopefully, we will then be able to anonymous that to allow ah used by academics and possibly also, uh, partnering industry to do further data mining and tool development, which we could then further field test again using our real world data base of patients that will be continually, uh, updating in our system. In the cardiovascular group, we have what's called the bricks cohort, which means biomedical research. Informatics Center for Cardiovascular Science, which was done, started long time even before I joined, uh, in 2010 which has today almost captured about 10,000 patients arm or who come through to Glenn Field Hospital for various treatments or and even those who have not on. We asked for their consent to their blood for genetics, but also for blood tests, uh, genomics testing, but also image ing as well as other consent. Hable medical information s so far there about 10,000 patients and we've been trying to extract and curate their data accordingly. Again, a za reminder of what the strengths of Leicester are. We have one of the largest and busiest trust with the very large, uh, patient cohort Ah, focused dr at the university, which allows for chronic diseases such as heart disease. I just mentioned our efforts on heart disease, uh which are about 10,000 patients ongoing right now. But we would wish thio include further chronic diseases such as diabetes, respiratory diseases, renal disease and further to understand the multi modality between these diseases so that we can understand how they >>interact as well. Finally, I like to talk about the lesser life science accelerator as well. This is a new project that was funded by >>the U started this January for three years. I'm the director for this and all the groups within the College of Life Sciences that are involved with healthcare but also clinical work are involved. And through this we hope to support innovative industrial partnerships and collaborations in the region, a swells nationally and further on into internationally as well. I realized that today is a talked to um, or business and commercial oriented audience. And we would welcome interest from your companies and partners to come to Leicester toe work with us on, uh, clinical health care data and to drive our agenda forward for this so that we can enable innovative research but also product development in partnership with you moving forward. Thank you for your time.

(upbeat music) >> Hello, and welcome to our Palo Alto studios in California. I'm John Furrier, host of theCUBE. This is a CUBE conversation with Jason Zintak, CEO of 6sense. This is part of our next gen conversation series. We talk about the technologies and the news and the people making it happen for the next generation technologies, clouds, and solutions. Jason, welcome to theCUBE conversation. Thanks for coming on. >> Thanks, happy to be here. >> So you guys got some news. So you got a couple weeks ago you announced $40 million in funding, which we'll talk about. I want to get that out right away. But I think, more importantly, we're seeing a trend where this next gen blank is happening. You know, I'm watching just the Super Bowl next gen stats is for NFL. You got next gen cloud, you got next gen data. The world of the technology is kind of shifting to a new architecture. You're starting to see visibility into what this next gen looks like. Your company is squarely in the middle of this next gen sales and marketing platform, solutions in the new model. Cloud-scale, data first, this is a core, major shift and it's a huge market. Look at Salesforce, look at all these companies that've been around. And they're incumbents now, you're the new guard. >> Jason: Yeah, yeah. >> Tell us, what's going on with you guys? >> Sure, well you're right. We just raised $40 million. It's our Series C from Insight Partners. Went through a lengthy evaluation process and compete and happy to have announce that last month. And as far as next generation, you're correct. I grew up in a world of email platforms and then big data platforms, marketing automation. And this is a data first strategy, where we allow, we now have compute power that allows us to process huge amounts of data sets. So it's our belief that it should all be data first and driven from AI and ML on top of data that drives a next generation marketing tactic or sales tactic, an email, or a display ad. >> What's interesting is that you mentioned you worked in previous old school technology. You were CEO of Responsys, which was sold to Oracle. That was a great wave that brought in the marketing technology stack. We saw the sales and marketing solutions from Salesforce.com obviously. That was the first wave that you were part of. Now the new wave is going to that next level. This is really the fundamental shift. And it's not so much they're being replaced, but they're just being abstracted away with new capabilities, in some cases being replaced. What's the core problem that customers are having, or the core problem that you're solving because some of these old solutions can't scale. >> Jason: Sure. >> Some of them are because they're big, but what's the core problem in the industry? >> The core problem is that these systems were designed to be contact first, or lead first. And as you know today, no one likes an abundance of emails in their inbox. And so companies have said, hey I want to have a relationship with my customer or prospect. I want it to be a cycle of engagement, an infinity loop. Which means we don't blast emails. We monitor a relationship, what that's like, how we might engage. And the data allows us to do that. We can see what's going on with the activity, and based on that engagement, AI tells us what tactic might be the most appropriate. Which is actually send less but more effective and more targeted. So it's a data-driven approach. It's an account based focus in B2B world, as opposed to old generation which is lead and actually rule based. And so we used to write these, call them journey maps, these if then statements, which were manual. And the second we got done doing weeks of if then statements, they become stale. And so now data helps us and AI helps us understand real time behavior with intent and then the tactic. >> Love the name 6sense. Obviously you want to get a sense of what's going on around you, six degrees of separation. You got network effect. We're seeing a new reality and that is organic kind of user experience is different happening outside the funnel, sometimes inside the funnel, as they talk about in the sales and marketing. But users, at the end of the day, they're downloading Brave browser. They don't necessarily want the ads, and so they're making these decisions based on their experience that they want. So this is changing some of the tactics. >> Jason: Absolutely. >> So talk about that dynamic because the old way was based on see an ad, click on it, go to a landing page, get a lead, throw it in the funnel, matriculate down, and sell them something. And time's not on your side. It's not real time. It's slow, antiquated, you know how to quit. >> Exactly right, so if you don't look at Forrester or Gartner, they'll give you stats that 80% of the B2B sales cycle is done anonymously today. Meaning, they don't want to contact the vendor. There's an abundance of data on the web. And so we appreciate that. We want to actually enable an engagement through learning. We call it the actual dark funnel. This is all the research where it's happening without the vendor being contacted, without someone raising their hand and saying I want a vendor message. Because of this activity that we're able to see and be patient with, we're allowed to engage when the prospect or customer says they want to. But in a nurture format, so it's more respectful of their time. And all the while, this engagement idea is we're giving them content when they want it, when it's on demand, and when it's appropriate. >> And there's all kinds of new data laws coming, so you got to navigate that kind of regulatory environment. But we've been saying on theCUBE, this is our 10th year, and you know the old way and now we got a new way that you're on with company is that people are connected. Everything can be instrumented. This is the big data revelation that started about 10 years ago when the big data movement, and when people said hey data's going to be a big part of it. But with the internet, everyone's kind of connected, so you can technically measure everything. So as a company, how do you look at data? I mean data's fundamental to your vision and your execution. How is that ingrained into the culture and your product? >> Good question and first like to say we respect privacy in the data and personal and companies. So we are GDPR compliant, SOC 2, CCPA, the new California laws as you know. And that is part and parcel to our strategy, respect it. But at the same time, today's consumers generally want to be known in some way, shape or form because they understand the experience of engagement, whether it's an account or an individual customer. The experience is that much richer, if it's personalized and done with taste. Meaning, it's not spam. It's not a thousand emails. It's a meaningful, purposeful, time-based engagement,' content's relative to when they want to know something. >> Well I like what you guys are doing. I like this next gen architecture. It's definitely been valid. You've seen the rise of Amazon. Microsoft's shifted their business model to the cloud. And you're starting to see other ones, other people shifting. IBM shifting to the cloud. So they're all shifting to this new business model. So for you guys, 6sense, talk about and tell me about your target market. What market are you going after? Is it the marketing automation? Is it like the sales platform? What's the market that you're in now, and what market are you expanding into? >> Interesting you say that, so we're classically B2B. We obviously have a bunch of tech customers as our, in the account universe. But also manufacturers, service businesses. We are going after the entire B2B organization because the world as you know it, relative to marketing and sales, is changing. And so it's not just marketing automation that we're replacing, or a next generation of, it's customer success. It's the sellers. Our customers' sales organizations use it with their sales people to understand insights of their accounts and how to engage. So I'd say it's that whole universe, and it's that infinity loop across customer, sellers, marketers. >> You know, I want to just before I get into some of the business model questions and target audience, the buyer, you mentioned customer success. We're seeing a lot of energy around what that is. It used to be customer success was like customer satisfaction, support organization. You're seeing companies bring customer success much further forward into the sales and marketing process for pre-sales and or ongoing engagement as some of these SaaS environments evolve. >> Jason: Yep. >> Are you seeing that, and what's going on with this customer success? I'm seeing a lot more other than lip service. It's pretty integral with companies, organizations these days. What's your thoughts on that? >> I think all of us drive to be customer first, customer happiness, loyalty. Sure, why not? I mean, that's what we should do as organizations. Our software actually, interestingly enough, allows customers to monitor how their customers are engaging with the vendor. And for instance, they may be, if we see a spike in looking at a competitor, the customer will say, hey are you happy? Or product telemetry and usage. We help companies track that usage and see spikes and based on that intent, you might engage with your customer differently, high or low propensity to actually churn. We help with churn mitigation and churn management. >> Okay, let's get in to the product. We're kind of teasing around the product. What is the product? What's the core jewel? What's the IP? What's the main platform look like? What's the product? >> So as mentioned, we're a big data company first. Meaning, we believe it all starts with the data. Because of the compute power available, we're analyzing data, which is your first party data. So all your historical sales and marketing outbound, maybe your CRM system, your marketing automation system, some of the systems that will continue to evolve. And we'll match that data with behavioral data. So what's happening on the web, what's happening through maybe it's cookies, email hashes, display account ID, advertising ID. And we've patented an approach called a company ID graph. And this ID graph is essentially this marriage of people, personas, and accounts and what's going on. Based on the insight that comes from this monitoring, you can create audiences or segments to market to, to sell to. So the insights would be on the marketing side, relative to how do I parse my total addressable market. Or on the seller's side, Oh, I can understand what my count or my prospect might be doing today, therefore I want to execute XYZ tactic, and all led by AI. >> And so I got a, good point there about sales and marketing. In the old way you had a marketing tech, and a sales tech. The lines have blurred, almost seem to be fully integrated now, they're one in the same now, seems like that's the way you guys look at it. Is that true? >> Absolutely, I grew up in sales and marketing and the old world they didn't talk to each other. Today this is absolutely the glue, the connective tissue for sales and marketing so you can start with, whether it's marketing or sales ops, you start with a central plan around your account universe, and then parse from there and segment from there. And so, marketers and sellers will come up with the annual strategy, but allows the conversation. So it's no longer is my lead any good. We've got data around the lead, is the customer responding to an ad campaign. We've got data that it's true. It's not, you know, maybe. >> Yeah, it's always the sales guys always tripping about the leads, these are good leads. The leads are from Glen Gary, Glen Ross, always great quote, good quote that in there. All kidding aside, at the end of the day it's about customer satisfaction. No one wants to be marketed to, so it's a wave of personalization coming. And we're starting to see that now with Big Data, kind of set the tone on that. How are you seeing this new account based marketing and company selling platform. To deliver this kind of personalization it adds value. How do you orchestrate all that? So this is the big challenge, how do you bring that all together? What's your thoughts? >> So, actually our platform allows for that. So as you might imagine, you mentioned the sales funnel, and start with you know customer having initial curiosity, or maybe down at the bottom of the funnel there, actual buying stages through procurement. Based on where we detect someone is in the funnel, you would personalize the content. So if we detect through ID graph, that the company or person might be interested in general awareness, awareness content. If they're down in the buying cycle, far down into the funnel, then it's more related to transactional, meaningful clips that would be more relevant. And that is the personalization, so it's stage appropriate as someone would want to consume it. As there engaging with us. >> Jason give us some of the top use cases that you guys are seeing, as you start to see visibility, you got $40 million in funding, third round venture. You got customer growth, good growth. What's the visibility, what do you see in front of you, what are the use cases? >> Great, so for the capital, I assume you mean. We've had two great years, we've doubled the company two years in a row. We're expanding, so it's actually going to be sort of broad brush, we're expanding our field organization, we're expanding the engineering. We're looking for acquisitions that are strategic, and so our growth will be both organic and inorganic, but it's because of the success and the growth. We want to build the product better to make the customer happier. And that is the general use, of our international expansion. >> So I'm a customer, sell me on this, what's the pitch? >> So-- >> I'm a big tech company, I've got five tons of data. People, internal knife fights going on, I got this platform, we got to get the ROI out of it. How do you, what's the, what's in it for me, pitch me? >> Hey, John is your sales organization happy with the leads? Do they think it's quality? >> The leads are shit. (John laughs) >> The leads are shit, we can help you there, we actually have you know AI helping us understand your account prospects of whose high propensity to buy. We help your sellers. Does marketing talk to sales, John? >> They have meetings, no one want to attend them, I mean this is the kind of thing that goes on. I mean we're talking about, kind of role playing here, but in real time, Hey, no, we're good. It's the sales guys fault, they're not good enough. >> Yeah, exactly, so-- >> The leads are terrible. So there's obviously, again, this is the kind of thing, the tension that goes on. >> Yes, so from the marketers perspective they're looking for a more data driven approach to, and again data helps, data doesn't lie. You know it's sort of math. And so it's no longer speculative, it's we can see the engagement if we run a campaign, whether it be email, ads, social posts, chat bots. All this is collecting data, and showing data relative to efficacy, and that is actually what the marketer wants, and candidly the CEO wants to the see the result of those joint selling and marketing efforts. >> All right, so you got me hooked. Let's do something. How do your clients engage with you? What do they do? A POC? Do they just have a sandbox, is there kind of a freemium tier? can you explain some of the business model and engagement? >> Sure, yeah. We do POC's, we do sandbox. But interestingly enough, we can turn the data on in an hour, an actually a prospect can see what's happening in their universe, they're competitive universe or their own. website, for instance. And so that's a very easy way, tell-tale sign to see data at work. We have low entry points, where companies can come in at 30K at 20K, and start. Or we have million dollar plus contracts that you know span the breadth of sales, marketing and customer success. So it's an easy entry point, you can grow with data, you can grow with users, or you can grow with models. >> So Facebook, and LinkedIn are on, and Twitter, but mainly Facebook and LinkedIn are showing micro targeting as highly valuable. I mean the election train wreck that's happened this past few years, and even this year, I see Facebook has their own issues, but LinkedIn, a lot of people from a B2B standpoint, like LinkedIn. It's network effect kind of distribution, you got targeting, you got a lot of metadata in there. So it's kind of brought up the conversation around micro-targeting. Why can't you just go at the people? You guys do an account based marketing and sales orchestration platform, and you've got these little walled garden organizations out there like LinkedIn. I'm not sure they're selling the data, do they do that? Do you work with LinkedIn, so will there be more LinkedIn? Nope, we got our data, we're going to keep it? Data becomes the key, but if they're going to hoard the data, it's a problem. How do you address that? First of all, do they hoard the data or not? And if so, how do you guys get around that? >> Well you know LinkedIn's got a wonderful business, and they, to agree some of this wall, are a partner of ours, and actually we'll have some announcements pending. So I'll save that for later, but -- >> So they are engaging with platforms, LinkedIn from a data standpoint. >> Very much so, we're an active talks with LinkedIn. And I think we all want to share for the benefit of the ultimate customer experience. And we believe that because we have the Big Data, and we also allow for that micro-segmenting. LinkedIn's another channel, and we want to activate every channel through our platform and that is our strategy. So we allow you as mentioned before, email, display, social sites. >> Do you guys have a program or approach or posture to the marketplace in terms of, if I have a platform, do I engage with you. Can I be a partner or am I a customer? How do you look at the biz dev or partner side of it? >> You know part of the $40 million funding is going to allow us to build out the partner ecosystem that's already in play. We work with agencies, ad agencies. We work with professional service organizations. We work with complimentary software products. We want it to be an open system. We want to be able to bring your own data, and we'll carry it for you to make the AI that much smarter. >> Awesome, great stuff, quick plug of the company, we're you guys at in terms of head count? What are some of your goals this year? And what are you guys looking for, obviously hiring, you said, you mentioned earlier? Give a quick plug for the company. >> Yeah, thank you for that. As I mentioned we doubled the company two years in a row. We've tripled our head count. You know we're hiring everyday in every single segment, looking for people. We'd love to talk to you. We've also tripled our customer base in that same period. So, things are going well, we're happy and I think the big challenge is just keep doing it, and deliver delightful experience for customers. >> Interesting, companies can be very successful Jason if they have a certain you know view. You guys are data first, you got to a horizontal view of the data, but yet providing a specific unique solution to differentiate off that. We're video first, that's our angle. A lot of people having virtual first. Your starting to see this new kind of scale with companies. So I want to ask you about your vision for the next few years. As you look out as the wave is coming in, it's very clear. Cloud-scale, the roll of data, machine learning and AI. It's going to build this Application Layer that has to be horizontally scalable, but yet vertically specialized, for the use cases. Which requires a very dynamic data intensive environment. What's your vision of the next few years? How do you see the world evolving? Because there's a lot of big companies, and start-ups that have been around doing a lot of these point solutions that are features. How do you see this next wave go in the next five years? >> I had a thesis three years ago, I joined the company that these point solutions would go away because they weren't data driven. The hard work is in the large data, the applying the ML and AI on top of that and then doing something with that. We surfaced in applications for the last two years, we've been building the apps that allow marketers, sellers, and customer success organizations to prosecute that data, understand the data and let AI recommend a tactic. So I think it'll just be more of the same but specialized by use case. So where some of our applicability is generic use cases, we'll get specific to telecom on that use case, we'll get more specific in customer success enabling turn mitigation as opposed to just sellers and marketers. >> That's awesome. And if you look at the current events, I got to get your expert opinion. Donald Trump, the Democrats, they've been using social platforms, political ads are being kicked off, but there is a lot more innovation that they're actually doing. So with all that they had actors out there, there's actually an innovation story that's going on under the covers. What's your view of that, I mean the bad stuff's out there, but they're leveraging the new architecture. Facebook's on record saying that Donald Trump ran the best campaign ever. Mentions why he's winning. >> That's the story and back story is sort of history unfolds when we understand it. Is that these election cycles have leveraged data to run their campaigns and it's the new world. And so while there may be bad actors, I think hopefully the world is majority good. And much like our story, we tryna bring a data solution and help decisioning. Obviously, the political campaigns are leveraging it to. >> Yeah, it's disastrous to see the applications fail like they did in Iowa, but the data's there, I mean it's about time. I always say it's going to be on block chain, and Andrew Yang is, just recently came out and said, All the voting should be on block chain. Maybe that's going to happen someday, we'll see. Jason thanks for coming, I appreciate the conversation. >> I appreciate the opportunity, thanks John. >> Jason Zintak, here the CEO of 6sense, industry veteran. Big pedigree, big company with $40 million in fresh funding. We're talking about next generation platforms, I'm John Furrier, thanks for watching. (upbeat music)

>>LA from Las Vegas. It's the cube hovering AWS reinvent 2019 brought to you by Amazon web services and they don't along with its ecosystem partners. >>Hey, welcome back to the cubes coverage of AWS 19 re-invent Los Vegas. Where else would it be? Why? Because there are about 65,000 people here with Stu Miniman and myself, Lisa Martin and we're pleased to welcome the guests from SAP, Germany, joining us as Glen Gonzalez, the chief technology officer in Germany. Glenn, welcome to the cube. Thank you. Nice place to be. Isn't it nice with XO? So this is day three of all the action. There is not a lull in the attendance of the expo halls. So much going on. So much news, but give us a little bit of an insight. SAP is a customer of AWS. AWS is a customer of SAP. Talk to us about all that you guys do together. >>Yeah, that's, that's interesting about our partnership cause it has so many dimensions and the firsthand, actually SAP is a big customer of AWS. We have huge workloads running on AWS or even our software as a service. Applications like conquer. Maybe the coolest travel a software you can have. For me as a traveler, I use it every day. And on the other hand it's like AWS is a big customer of SAP using conqueror themselves. And you know, I don't know, maybe a lot of people around us, they will know what I'm talking about. And then adding to this we also competitors, which is great because competition drives innovation. Everybody knows this. So, um, and that brings me actually to maybe the last part is like innovation is that maybe the center of our partnership is that we together can do so much more than each of us could do. >>And if we bring these two parts together, then then it's real value for the customers. And that's maybe the most exciting part about this. Clint, it's interesting. SAP Sapphire was actually the second show we ever did as the cube way back in 2010 before AWS reinvent existed. But even back then we started talking about transformation. You know, I think of SAP and SAP is the global ERP company. You started talking about all the different pieces. How should we be thinking about SAP in today's modern transformed cloud environment? Well that's, that's, that's a topic I have every week cause a lot of people perceive SAP as the ERP company and it's, it's so much more due to the changes that are happening around the world now. We stand for our business processes end to end, safe and secure. And business can run only if you have a software that can do this. But these processes are changing, you know, they're expanding and they are technically changing. For example, IOT brings actually that not a human being starts the process. It's a, it's a, it's a sensor but it's still an end to end process or technologies like blockchain and machine learning are changing the process itself. So some people think machine learning is only fancy of its autonomous driving, but put it into a process and it gets autonomous and that's a real value for customers. You can even calculate a business. So it's so much more than ERP today. >>We're at a Sapphire last year, so summer of 2018. One of the things that was interesting was a lot of messaging around ERP. You can talk to, of course here we are at AWS, lots of people know. Alexa, talk to us about some of the innovations and the emerging technologies that SAP is bringing into your flagship products and your other technologies that are really helping to transform way beyond ERP. >>Yeah, that's, that's, that's interesting cause we're doing a lot of this innovation together with AWS. Um, many of our biggest strategic customers are already running their workloads on AWS and, and so many more. I evaluating to do this. It's really exciting times. And as I mentioned earlier, new technologies kick in. So our, um, business technology platform based on SAP cloud platform is that by the customer's need, um, this connection from these new technologies into the businesses and we are developing so many softwares together now, um, like data intelligence or data warehouse cloud, um, and even the SAP custodian as we are now brought out last, uh, two months ago on our ticket. Yeah. So Glenn, uh, you know, SAP has always had a great few point into the productivity of the worker. Um, and there's so many technologies you brought up the autonomous bees. So it brought to mind one of the hardest areas in tech beyond just cloud is RPA, robotic process automation. >>Uh, you know, help us bring inside, you know, SAP is positioning there. Uh, and you know, even broader about just how workforce can be more efficient, not just get cool new tools, guide some, maybe one of the main topics of the ITD ciders around the world is how can I make things more simple? How can I reduce the complexity of my it? And it always starts at the user. Uh, it should at least not always, but it should. And if you can make the work of a user a lot, a lot more simple. And that's what these technologies bring in. Uh, automize or, or posh, partially automized processes. Um, the, the, the user of the software can do different things or you can only confront them with the difficult stuff on. The rest can be done kind by the system. And that's why these things are really important and we may main topic and you can put this new technology in with not such a big effort as others. Maybe >>the end user more productive is critical, right? Cause we're all end users at the end of the day and sometimes it's very challenging to get worked on if you have so many processes, there's so many different applications that you have to work with but to get the end user that productive to really streamline the enterprise software space. A lot of things we're >>hearing about, and Andy Jassy talked about this in his keynote Tuesdays organizations to truly transform a business all the way down to that end user level, you have to start at that senior executive level as the CTO of SAP in Germany, are you starting to see conversations shift up from that more end user space to that C-suite? Is is enterprise transformation really at that level and in your experience? Definitely. Definitely. It's a big topic but you have to see this. There are two to two versions of this. The one is to talk about it. And the other one is the execution on it. So we see a lot of companies are talking about unstop the execution. And it's a real transformational part and it's really hard for many companies because the change is drastic. And what we really see, it only works top down. So if the C level is not in it basically will not happen. And that's, that's something we've really learned within the last years. >>Yeah. Glenn, I always want to get to talk to a CTO. The changing role of technology in business today. Uh, you know, you used to be able to say there are certain industries, well, they might use of technology, but they're not technology companies. Now, you know, the, the, the meme is everyone is a software company and everyone's becoming technology. So you're gonna bring us inside your viewpoint as to, as a CTO about how, you know, how important, you know, this moment in time is, uh, in the technology industry. Huh. >>I think it's, it's, it's master key for, for many companies and even the role of the CTO changes from, for example, my role, I'm really customer centric. I spent many spend time with many customers a week. So I'm not in the machine room fixing things. I'm listening to customers because if you don't understand what they actually have for expectations, you will never fit. They're actually the expectation. So even even putting one on top, some of our customers need help to understand the expectations of their customers and that's the part of this digital transformation and these new businesses coming off. And so it's a lot more exciting than maybe a few years ago where we only talked about tech. I spent most of the time about how to use it and then afterwards how the tech has to be implemented to make this possible. What are your conversations with respect to people on that cultural change? >>As soon as point is, so many companies today have to be absolutely fueled by technology to be competitive because there are startups, right? You know, behind a lot of legacy businesses ready to cannibalize the business. But that cultural piece, it's really difficult. Talk to me about some of the conversations that you have with customers to help them maybe reset expectations but also get them understanding that that cultural transformation is critical to the digital transformation. That's maybe the unfair part. We come in and want to talk about technology and use it and then they, they start about change and not talking about change, their mindset change. It's the critical thing you can, you can have the best ideas. You can have the best technology behind this if you'll own organization but not go with this. You will not stop or you will not start. Sorry. So that's why I just earlier said, if it doesn't work top down, it will never work. >>If if the designer or let's say at the boss, if he's not in, if he doesn't understand the necessity to change this, it will not happen. And the changes is quite heavily. It's not agile. It's a lot more, it's about really thinking a different way or even understanding what the internet is doing to everybody. Some don't even understand that and it's, it's sometimes it's really surreal. You know, you're setting down, someone's telling you, Oh you know, my daughter uses a smartphone all day. I under don't understand her. And I said no. Then start understanding it cause that's how the world is turning out the moment. Right. And there are five different generations that are in workforce today. So businesses, SAP, your partners, your competitors, all have to cater to your point to a really broad level of technology understanding. Yeah, that's can be a big barrier. >>It is a barrier. But don't make the mistake to only get in the millennials and throw the old ones out because that's the biggest mistake you can do. Cause it's, it's about the mixture. It's about diversity in the team. I mean it's even if you can even ask scientists. Yeah. A lot of stuff you can read about this, but if you want to really make it happen, you have to live it. And this is where SAP, we had talked about it upfront that we have actually five generations within one company. And it's so important because the business process had the beginning. There's a reason why we did it this way. And if the new people don't understand this, they may make big mistakes. So that's the magic, bringing them together and making new kinds of teams. >>Yeah. Well Glen, I, I loved a couple minutes to go. You're talking about the requirements that top down leadership to be able to help and that really echoes what Andy Jassy talked about on the main stage. I'm wondering if you could give us a little bit of the global viewpoint, especially being from Europe here because you know very much we talk about that move from the bottoms up to the top down and it coming from both ways here. Is it very similar across the globe? Is there maturity or changes in some of the workforce that might be a little bit different in some geos versus others that you're working with? >>Without big differences, especially in Germany, which is a very mature market. Um, there's a lot of, actually there is a lot more talking about data security and privacy in Europe than we see in other regions of the world. For example, that doesn't matter. That doesn't say that. It doesn't matter that, but it's different talk. So, or even cloud. Uh, for some people it's cloud is like. I don't know. I can't really grab it. So an interesting is a different understanding of cloud of people. So it's, it's, it's regionally totally different how to go in and it's also a difference if you're talking to a big company, which is globally on the, on the road or others that are starting to get global for them, it's also change. Yeah. In other markets it's not a problem to do it that way, but in your own markets, a lot of, you know, say as like, Oh, let's wait. >>We have to discuss this first and that's maybe the wrong the wrong version. Yeah. As we look at how cloud in and of itself as an operating model, but also the technologies that define it, how they've evolved and changed. One of the things too that Andy Jassy talked about with our own John furrier is that, you know, a lot of the businesses that are going to be successful tomorrow are either going to be born in the cloud companies or they don't even exist yet. What are some of the things that you're seeing in the existing enterprise, not just in Germany, but globally? Are you seeing any industries in particular that you think are really right to become reborn in the cloud? For example, web far, especially in Germany where we have a lot of companies building machines, hardware, it's, it's more difficult to get this vision of being a digital company. >>You mentioned it earlier. Companies are becoming a software company, although they're building machines and the machine is only there to enable the service. This is a big change for them. It's of course a lot more complicated to to understand how these new technologies can help them on for them. They are actually in the beginning of understanding, but for others that come from a service side, for them, it's a lot easier for them to understand what these technologies can bring them and agility and flexibility and scalability. It opens totally new doors, but there's still a lot of education you have to do for them to understand that it's really a really the right door to go through and that's part of my job extending these things. Yeah. Glenn would love you. You've been, I'm sure talking to a lot of customers this week. Give us your final takeaways from AWS. >>Re invent 2019 wow. I'm a little bit overwhelmed by the input you can get here. I really tried to go to all sessions. I failed. Maybe next time there's only 2,500 of them figured out. Cloning highly shy manipulations is one of my CTO. Let's maybe I work on it for next time. We'll talk after this maybe next year so I need to be invited again and we can talk about that now. It's, it's a huge input we have here and it's a different stage. If I talk to customers it's a different talk cause we have more input from many sides and they are also open to talk about things that they may not be open to when they're at home cause she are things that there's so much positive input and so diverse input that it really helps to start different conversations. Well 2,500 sessions. That cloning thing will really help out. Not only with that, but also can you imagine how much better ERP would get if we had clones? So you'll have to come back cause we have to figure this out. I bet on that. All right, Glen, thank you for joining steward again for your time. All right, first two minute, man. I'm Lisa Martin. You're watching the cube and stick around because later today, Andy Jassy stops by. Thanks for watching.

>> from New York City. It's Theo Cube covering new relic Future Stack 2019. Brought to you by new relic. >> Welcome back on stupid a minute. This is the cubes coverage here of future stack 2019 new relics. 70 year they're doing the show is the U. S. Show. They actually bring these few locations around the globe, right next door to Grand Central Station and about 600 in attendance. And been really excited to kick off with the number of the users here at the show and happened. Welcome program. First time guests. Another cut. Who is the technical fellow in chief? Architect with Genesis. You been at the event a number of times. You're speaking at the event today, but let's start with Genesis. Customer experience is something that I think a lot of people been hearing about on. That is the product. The Genesis has tell us a little bit about the company itself. Sure. >> Yeah. So, Genesis, uh, brain that Maybe not. Everybody knows, but they certainly transitive Lee know us. We're a customer experience platform. We like to say that we're a technology company, but we power. The experience is about 25 billion customer experiences every year for 11,000 plus customers. About 1000 different countries around the world s So we are all about having a connection between brands and their customers, and we enable that >> s o not only some of the cloud shows. I was an enterprise connect earlier this year and definitely was, you know, something I heard a lot about see Exit really important Not only how customers interact with the brand, but internally how you know we treat the employees and that interaction is something that that is raised up. People are kind of important inside, but we're going to talk too much about the people here. We're gonna talk about the technology as the chief architect of this gives a little bit about what you have your arms around in a responsible for >> sure s o for for me, of the project. Your cloud was the name for for a long time, Genesis Cloud as of yesterday. So we are a public cloud offering as a CX platform and I say platform because we made the transition from just being a product to a platform. In my opinion last year, more than half of our FBI work is actually code we didn't right? So I think people using you as a programmable thing is when you become a platform. So I'm responsible for things like cloud architecture for understanding. Let's say industry trends. What technologies? We're gonna use a lot of eight of us service designed technical vetting, general cat hurting that sort of thing, >> Right? So you said your public cloud, but you said it sits on top of AWS. But it's a platform that your customers can then build on top. >> That's right. That's right. So we like to think of ourselves as C X. As a service. We've had some that use us still like a product all shrink wrapped, ready to go, others that want to extend us either writing their own. You guys writing their own back ends their own integration points. We make all of that possible. >> All right, so I'm expecting you have a bit of an opinion when it comes to that platform, As Lou said with a capital P A, and it's gotta be programmable, it's gonna be open. Tell us what your thoughts about new relic kind of entering, you know, new relic one being they said today the first, and only if their claim of observe ability platform s o give us your thoughts around. >> Absolutely. Yeah. S O. I like to think that we have been using the relic as a platform for awhile, whether they knew it or wanted it or not way have a fairly rigorous continuous delivery pipeline. And we are very big believers in infrastructure is code and develops principles. So for us, the engineering teams don't just own the code that they write, but they own the infrastructure definitions. They even own alert definitions, dashboard configurations. And we push that information directly into the relic as our deployments happen. Live hundreds of times a week around the globe. >> All right, so how do these modern architecture's enable you to run a team? >> I can't imagine trying to manage 350 plus Micro service is in production, which is roughly what we have today over 1000 Lambda Functions way can't improve what we don't measure. Everyone likes to say that, but it's true. I have a little bit of an a p m background from from places past. So I was a firm believer that you need to invest early and observe ability and metrics. So we've been a day one kind of new relic subscriber in the cloud space. Everything from understanding how the infrastructure parts work now to serve earless. It's all been about moving up the value stack like commodity metrics of servers is great and still needed. But transactional information and now trace information is absolutely essential. >> Okay, in the Kino this morning, they walk through their metrics events, logs and traces. Where are you with, you know, these various sources of data and harnessing the value of that. >> So I would say, with fairly early towards the tracing part before new relic headed as a managed thing they had cross at tracing. I'm sure you're familiar with that sort of the prior incarnation of distributed tracing on. We leverage that pretty pretty heavily, but it obviously doesn't have quite the same utility a cz what the new open tracing standards provide s so we do things like having correlation i d. S. That let us tag and follow things around. Now we just get to off load that from our team's being as responsible for it. And now the platform gives it to us. >> Yeah. Glen is open source important to your organization? >> Absolutely. We try Thio, give back some ourselves. In fact, one of the one of the nerd lets the nerd packs that Lou mentioned on stage was one that our team wrote s Oh, yeah, way believe not only that, we need a p i's and programmatic access to do our jobs, but we like toe enable and help other people with the same >> Eric Spence got a shout out on the Maquis note was that the thing that you were talking about it is >> I expect to see us probably released two or three more nerd packs before the end of the year Way, way are eager to do that rather than just investing in all of our own. You I that we had glass over the top of the relic. Now we actually just get to put those components deeper inside of new relic proper. >> Okay, eyes there. Anything else from the announcements this morning that you're looking forward to leveraging? >> So I think there's there's definite changes in the A p M space. You'll hear a little bit more, probably in the deep dives one of the talks I'm having later with not even she will be talking about. Some of those things were definitely interested in that. Open telemetry has some value. Greater Genesis definitely has investments around things like Prometheus and other sorts of monitoring. So if I'm not talking about just the public cloud side of it and other aspects there definitely things we can leverage. >> All right, Glenn gives us share a little bit, if you can. About what? What you're talking about here at the show. So one of >> the big mitts is entity centric. Observe, ability. The idea again that we're not just looking at servers and static infrastructure. We're looking at things that are very ephemeral. We have a lot of dynamics scale on our platform on. We need ways to actually frame what we're looking at at the level of Micro Service's but often level like business applications. So even when we're creating some of these extension points like the one you just mentioned way framed that within the context of a service that does a particular vertical slice on dhe, that's that's kind of where we like to invest. So we like to live. >> Okay, um, you know what's what's on your road map of? You know where you're going with your journey and is there anything that you're looking for? Beyond what was announced today from new relic ER from the ecosystem at large, >> I think there's lots of refinements of what was announced today that will help us theeighty I ops side, I think not just for noise reduction, but also for like, early early signal detection. It's a pretty fascinating space. Will likely invest some of our own dollars in times trying to help that along. Definitely Ah, lot of distributed tracing and Maur investment. There is a big piece for us. I think the A PM space. There are areas that I'd like to see a peon vendors invest in that goes beyond what now, I guess, is becoming more, more traditional, like transaction information. We have a lot of a i machine learning ourselves, and I think monitoring those types of workloads is going to be very different. As big of a paradigm shift as it was to go from classic monitoring Transactional. I think we're about to see that happen again in the >> industry. Yeah. What can you share some of the kind of the A I journey that you're going through a genesis where you are, You know what the maturity level is of solutions that you're using and >> sure way have a fairly robust aye aye team on products range from in the W m space back to the people that you mentioned at the first part of the talk way have workforce optimization, workforce management, and we brought a I algorithms to that a lot of time. Siri's forecasting that used certain machine learning techniques. We've invested a fair amount in until you and Opie any are so everything from sentiment detection to live transcription that we built in house to our own body engines that d'oh the new dialogue management. So we have a fairly robust bit there and some on the management side on the operational back in that we used to try to improve our quality of service on reduced any sort of incidents on the platform. >> All right, it's your third year. Third time coming to this show was what brings you back? What you excited about? I kind of dig in and take away from the event this year. >> I think the relics always been a partner in my stance, not just a vendor we believe so deeply in the observe ability message that one I want to be part of shaping that narrative. Eso coming to future sack actually talking to a lot of other executives, seeing where they're going and kind of sharing that use case, but also trying to be a little bit of a lighthouse. Thio, the new relic team as well, is what brings me back every year. >> Observe ability is something that it hurt. A number of startups talking about in the last couple of years were, in your opinion, does new Rolex it compared to the marketplace overall, obviously, they just kind of announced the observe ability, you know, full suite with new relic one. But you know what your viewpoint is? Toe have their wealth, their position? >> Where did I think their position? I think they are best of breed for what we're currently seeing. Owners of ability. There are other things, I think, where we could cobble together bits from multiple vendors but frankly, having application performance monitoring along with infrastructure, along with data being cold from the cloud platforms that we're all in, like, eight of us. They've got a unique place. I think the power of their agent technology has proven itself over time as well. My guidance to most other other companies that I speak with about this subject is don't just trust that it's all magic invest on. And I think they make themselves easy to invest in on. I think this platform play is a good one for them. >> All right. Well, another cut. Thank you so much for joining us. Sharing your journey, What we're doing in the best of luck on your presentation today. Thank you, sir. All right. Be back with lots more coverage here from a new relic. Future stack 2019. I'm still Minutemen. And thank you for watching the Cube.

>> live from San Francisco, California. It's the key. You covering the IBM chief Data officer? Someone brought to you by IBM. >> We're back at the IBM CDO conference. Fisherman's Worf Worf in San Francisco. You're watching the Cube, the leader in life tech coverage. My name is David Dante. Glenn Finches. Here's the global leader of Big Data Analytics and IBM, and we're pleased to have Mark Clare. He's the head of data enablement at AstraZeneca. Gentlemen, welcome to the Cube. Thanks for coming on my mark. I'm gonna start with this head of data Data Enablement. That's a title that I've never heard before. And I've heard many thousands of titles in the Cube. What is that all about? >> Well, I think it's the credit goes to some of the executives at AstraZeneca when they recruited me. I've been a cheap date officer. Several the major financial institutions, both in the U. S. And in Europe. Um, AstraZeneca wanted to focus on how we actually enable our business is our science areas in our business is so it's not unlike a traditional CDO role, but we focus a lot more on what the enabling functions or processes would be >> So it sounds like driving business value is really the me and then throw. Sorry. >> I've always looked at this role in three functions value, risk and cost. So I think that in any CDO role, you have to look at all three. I think the you'd slide it if you didn't. This one with the title. Obviously, we're looking at quite a bit at the value we will drive across the the firm on how to leverage our date in a different way. >> I love that because you can quantify all three. All right, Glenn. So you're the host of this event. So awesome. I love that little presentation that you gave. So for those you didn't see it, you gave us pay stubs and then you gave us a website and said, Take a picture of the paste up, uploaded, and then you showed how you're working with your clients. Toe. Actually digitize that and compress all kinds of things. Time to mortgage origination. Time to decision. So explain that a little bit. And what's that? What's the tech behind that? And how are people using it? You know, >> for three decades, we've had this OCR technology where you take a piece of paper, you tell the machine what's on the paper. What longitudinal Enter the coordinates are and you feed it into the hope and pray to God that it isn't in there wrong. The form didn't change anything like that. That's what that's way. We've lived for three decades with cognitive and a I, but I read things like the human eye reads things. And so you put the page in and the machine comes back and says, Hey, is this invoice number? Hey, is this so security number? That's how you train it as compared to saying, Here's what it So we use this cognitive digitization capability to grab data that's locked in documents, and then you bring it back to the process so that you can digitally re imagine the process. Now there's been a lot of use of robotics and things like that. I'm kind of taken existing processes, and I'm making them incrementally. Better write This says look, you now have the data of the process. You can re imagine it. However, in fact, the CEO of our client ADP said, Look, I want you to make me a Netflix, not a blood Urbach Blockbuster, right? So So it's a mind shift right to say we'll use this data will read it with a I will digitally re imagine the process. And it usually cuts like 70 or 80% of the cycle time, 50 to 75% of the cost. I mean, it's it's pretty groundbreaking when you see it. >> So markets ahead of data neighborhood. You hear something like that and you're not. You're not myopically focused on one little use case. You're taking a big picture of you doing strategies and trying to develop a broader business cases for the organization. But when you see an example like that and many examples out there, I'm sure the light bulbs go off. So >> I wrote probably 10 years cases down while >> Glenn was talking about you. You do get tactical, Okay, but but But where do you start when you're trying to solve these problems? >> Well, I look att, Glenn's example, And about five and 1/2 years ago, Glenn was one I went to had gone to a global financial service, firms on obviously having scale across dozens of countries, and I had one simple request. Thio Glenn's team as well as a number of other technology companies. I want cognitive intelligence for on data in Just because the process is we've had done for 20 years just wouldn't scale not not its speed across many different languages and cultures. And I now look five and 1/2 years later, and we have beginning of, I would say technology opportunities. When I asked Glenn that question, he was probably the only one that didn't think I had horns coming out of my head, that I was crazy. I mean, some of the leading technology firms thought I was crazy asking for cognitive data management capabilities, and we are five and 1/2 years later and we're seeing a I applied not just on the front end of analytics, but back in the back end of the data management processes themselves started automate. So So I look, you know, there's a concept now coming out day tops on date offices. You think of what Dev Ops is. It's bringing within our data management processes. It's bringing cognitive capabilities to every process step, And what level of automation can we do? Because the, you know, for typical data science experiment 80 to 90% of that work Estate engineering. If I can automate that, then through a date office process, then I could get to incite much faster, but not in scale it and scale a lot more opportunities and have to manually do it. So I I look at presentations and I think, you know, in every aspect of our business, where we clear could we apply >> what you talk about date engineering? You talk about data scientist spending his or her time just cleaning the wrangling data, All the all the not fun stuff exactly plugging in cables back in the infrastructure date. >> You're seeing horror stories right now. I heard from a major academic institution. A client came to them and their data scientists. They had spent several years building. We're spending 99% of their time trying to cleanse and prep data. They were spend 90% cleansing and prepping, and of the remaining 10% 90% of that fixing it where they fix it wrong and the first time so they had 1% of their job doing their job. So this is a huge opportunity. You can start automating more of that and actually refocusing data science on data >> science. So you've been a chief data officer number of financial institutions. You've got this kind of cool title now, which touches on some of the things a CDO might do and your technical. We got a technical background. So when you look a lot of the what Ginny Rometty calls incumbents, call them incumbent Disruptors two years ago at Ivy and think they've got data that has been hardened, you know, in all these projects and use cases and it's locked and people talk about the silos, part of your role is to figure out Okay, how do we get that data out? Leverage. It put it at the core. Is that is that fair? >> Well, and I'm gonna stay away from the word core cause to make core Kenan for kind of legacy processes of building a single repositories single warehouse, which is very time consuming. So I think I can I leave it where it is, but find a wayto to unify it. >> Not physically, exactly what I say. Corny, but actually the court, that's what we need >> to think about is how to do this logically and cream or of Ah unification approach that has speed and agility with it versus the old physical approaches, which took time. And resource is >> so That's a that's a computer science problem that people have been trying to solve for years. Decentralized, distributed, dark detectors, right? And why is it that we're now able Thio Tap your I think it's >> a perfect storm of a I of Cloud, the cloud native of Io ti, because when you think of I o. T, it's a I ot to be successful fabric that can connect millions of devices or millions of sensors. So you'd be paired those three with the investment big data brought in the last seven or eight years and big data to me. Initially, when I started talking to companies in the Valley 10 years ago, the early days of, um, apparatus, what I saw or companies and I could get almost any of the digital companies in the valley they were not. They were using technology to be more agile. They were finding agile data science. Before we call the data signs the map produce and Hadoop, we're just and after almost not an afterthought. But it was just a mechanism to facilitate agility and speed. And so if you look at how we built out all the way up today and all the convergence of all these new technologies, it's a perfect storm to actually innovate differently. >> Well, what was profound about my producing in the dupe? It was like leave the data where it is and shipped five megabytes a code two upended by the data and that you bring up a good point. We've now, we spent 10 years leveraging that at a much lower cost. And you've got the cloud now for scale. And now machine intelligence comes in that you can apply in the data causes. Bob Pityana once told me, Data's plentiful insights aren't Amen to that. So Okay, so this is really interesting discussion. You guys have known each other for a couple of couple of decades. How do you work together toe to solve problems Where what is that conversation like, Do >> you want to start that? >> So, um, first of all, we've never worked together on solving small problems, not commodity problems. We would usually tackle something that someone would say would not be possible. So normally Mark is a change agent wherever he goes. And so he usually goes to a place that wants to fix something or change something in an abnormally short amount of time for an abnormally small amount of money. Right? So what's strange is that we always find that space together. Mark is very judicious about using us as a service is firm toe help accelerate those things. But then also, we build in a plan to transition us away in transition, in him into full ownership. Right. But we usually work together to jump start one of these wicked, hard, wicked, cool things that nobody else >> was. People hate you. At first. They love you. I would end the one >> institution and on I said, OK, we're going to a four step plan. I'm gonna bring the consultants in day one while we find Thailand internally and recruit talent External. That's kind of phases one and two in parallel. And then we're gonna train our talent as we find them, and and Glenn's team will knowledge transfer, and by face for where, Rayna. And you know, that's a model I've done successfully in several organizations. People can. I hated it first because they're not doing it themselves, but they may not have the experience and the skills, and I think as soon as you show your staff you're willing to invest in them and give them the time and exposure. The conversation changes, but it's always a little awkward. At first, I've run heavy attrition, and some organizations at first build the organizations. But the one instance that Glen was referring to, we came in there and they had a 4 1 1 2 1 12 to 15 year plan and the C I O. Looked at me, he says. I'll give you two years. I'm a bad negotiator. I got three years out of it and I got a business case approved by the CEO a week later. It was a significant size business case in five minutes. I didn't have to go back a second or third time, but we said We're gonna do it in three years. Here's how we're gonna scale an organization. We scaled more than 1000 person organization in three years of talent, but we did it in a planned way and in that particular organization, probably a year and 1/2 in, I had a global map of every data and analytics role I need and I could tell you were in the US they set and with what competitors earning what industry and where in India they set and in what industry And when we needed them. We went out and recruited, but it's time to build that. But you know, in any really period, I've worked because I've done this 20 plus years. The talent changes. The location changes someone, but it's always been a challenge to find him. >> I guess it's good to have a deadline. I guess you did not take the chief data officer role in your current position. Explain that. What's what. What's your point of view on on that role and how it's evolved and how it's maybe being used in ways that don't I >> mean, I think that a CDO, um on during the early days, there wasn't a definition of a matter of fact. Every time I get a recruiter, call me all. We have a great CDO row for first time I first thing I asked him, How would you define what you mean by CDO? Because I've never seen it defined the same way into cos it's just that way But I think that the CDO, regardless of institutions, responsibility end in to make sure there's an Indian framework from strategy execution, including all of the governance and compliance components, and that you have ownership of each piece in the organization. CDO most companies doesn't own all of that, but I think they have a responsibility and too many organizations that hasn't occurred. So you always find gaps and each organization somewhere between risk costs and value, in terms of how how they're, how the how the organization's driving data and in my current role. Like I said, I wanted to focus. We want the focus to really be on how we're enabling, and I may be enabling from a risk and compliance standpoint, Justus greatly as I'm enabling a gross perspective on the business or or cost management and cost reductions. We have been successful in several programs for self funding data programs for multi gears. By finding and costs, I've gone in tow several organizations that it had a decade of merger after merger and Data's afterthought in almost any merger. I mean, there's a Data Silas section session tomorrow. It'd be interesting to sit through that because I've found that data data is the afterthought in a lot of mergers. But yet I knew of one large health care company. They've made data core to all of their acquisitions, and they was one the first places they consolidated. And they grew faster by acquisition than any of their competitors. So I think there's a There's a way to do it correctly. But in most companies you go in, you'll find all kinds of legacy silos on duplication, and those are opportunities to, uh, to find really reduce costs and self fund. All the improvements, all the strategic programs you wanted, >> a number inferring from the Indian in the data roll overlaps or maybe better than gaps and data is that thread between cost risk. And it is >> it is. And I've been lucky in my career. I've report toe CEOs. I reported to see Yellows, and I've reported to CEO, so I've I've kind of reported in three different ways, and each of those executives really looked at it a little bit differently. Value obviously is in a CEO's office, you know, compliance. Maurizio owes office and costs was more in the c i o domain, but you know, we had to build a program looking >> at all three. >> You know, I think this topic, though, that we were just talking about how these rules are evolving. I think it's it's natural, because were about 5 2.0. to 7 years into the evolution of the CDO, it might be time for a CDO Um, and you see Maur CEOs moving away from pure policy and compliance Tomb or value enablement. It's a really hard change, and that's why you're starting to Seymour turnover of some of the studios because people who are really good CEOs at policy and risk and things like that might not be the best enablers, right? So I think it's pretty natural evolution. >> Great discussion, guys. We've got to leave it there, They say. Data is the new oil date is more valuable than oil because you could use data to reduce costs to reduce risk. The same data right toe to drive revenue, and you can't put a gallon of oil in your car and a quart of oil in the car quarter in your house of data. We think it's even more valuable. Gentlemen, thank you so much for coming on the cues. Thanks so much. Lot of fun. Thanks. Keep right, everybody. We'll be back with our next guest. You're watching the Cube from IBM CDO 2019 right back.

>> Announcer: Live, from Orlando, Florida it's the Cube, covering Grace Hopper's Celebration of Women in Computing, brought to you by SiliconANGLE Media. >> Welcome back to the Cube's coverage of the Grace Hopper Conference, here at the Orange County Convention Center. I'm your host, Rebecca Knight. We're joined by Maureen Fan. She is the CEO and co-founder of Baobab Studio, which is the industry's leading VR animation studio, so, welcome Maureen. >> Thank you so much for having me. >> It's excited to talk to you, because you just won an Emmy. Congratulations. >> Thank you. >> You just won an Emmy for "Invasion", so, tell us a little bit about invasion. >> It was our first piece ever and it was just an experiment to see if we could even create VR and it's a story about these adorable little bunnies and you are actually a bunny too, you look down, you have a furry, little bunny body and these aliens that come to try to take over the Earth, with their advanced technology and you and your bunny friend end up saving the entire Earth and it's starring Ethan Hawk and it just came out last year. And we're really excited, because it became the number one top downloaded VR experience across all the headsets and it's getting turned into a Hollywood Feature Film. >> Very cool, very cool >> Thank you. >> And you have another film coming out too and this is "Rainbow Crow" >> Yes. >> Tell our viewers a little bit about "Rainbow". >> So, "Rainbow Crow" is based off of a Native American legend about how the crow used to have beautiful rainbow feathers and a beautiful singing voice and it's John Legend, in our piece and how he decides to sacrifice himself, by flying into the sun to bring warmth and fire back to the Earth and in the process, loses all his beautiful feathers, becomes black and burnt and his voice becomes like the crow's voice, but it's about how beauty is within and there's also, huge themes about diversity and how if you learn to accept yourself and your differences, that's when you can accept others and that's why we specifically cast minorities and women, so, we have John Legend, Constance Wu, from "Fresh off the Boat" as a skunk character, Diego Luna, from "Rogue One", for the moth character, as well as Randy Edmunds, as a Native American elder, narrator, and we have a whole bunch of other stars to announce, soon-- >> Well we cannot wait to hear. That's already an amazing line-up. >> Thank you. >> So, when you're thinking about "Rainbow Crow" and particularly, because it's VR, which is relatively new, still experimental, I mean, the messages of diversity, does it lend itself to VR, better than, say, a standard animation film? >> Absolutely, because if you think about stories that you just watch passively, the reason why we need stories and humanity, in general is to experience characters and stories beyond those we can experience in our real lives and we think, "Oh, how would I feel if I was in the "position of that character or what would I do?" but in VR, because you are actually playing a character in a role, you actually have to decide at that point, "what would I do?" so, it's not just a experience that I just see, it's one where I'm actively experiencing it, so, I create a memory and remember afterwards and there's all these research studies at Stanford by Jeremy Bailenson, who is head of the Stanford VR lab, that shows if you are made a homeless person, inside a VR experience and you have to go through a day in the life of a homeless person or you would look in the mirror and see that you are a black woman, that you, when you get out of the headset, you act completely differently. You have so much more empathy for these people than you would normally and so, it gets you to care about these characters, in a way that you don't normally and in VR, because you're doing it in a real-time game engine, these characters can act and react to what you do, so you can turn that empathy into action and actually act upon your caring, which we call compassion, so, it really changes you in a way, that normal, traditional story-telling doesn't, so, I think that having voices and characters that are different, in front of the screen, and also, behind the screen are really important to create role models and different perspectives for all the people out in the world. >> And these are movies that are targeted at kids, children, but do you see a future in which, where there is more targeted at adults, for VR? >> Absolutely. The funny thing is, in the beginning, the VR distributors didn't think that people would want our VR animation, because they're like, "Oh, it's just going to be these hardcore boys "that just love to play games. "Are they going to want this animation?" and VR is targeted towards adults, that's why they were surprised and we were surprised when "Invasion" became the number one downloaded VR experience. It shows that the audience for our content is from little kids to grandmas and everyone in between and that's probably why it became the top downloaded experience, is because it's universally appealing and has themes that are appealing to just, every single generation, so, absolutely, but for VR to become mainstream, there needs to be more universally appealing content. Right now, the content tends to be for games, like parkour games, as well as documentaries, which are two amazing pieces of content for this medium, but for it to become mainstream, we need more universally appealing content and I'm excited about, right now, it's a new industry. This is when minorities and women in particular, can enter the space and help shape the voices and the direction of the industry. >> That is exactly where I wanted to go next. So, let's talk a little bit about Baobab Studio. It's not that old and VR is not that old and so, why are there more opportunities, would you say, for women, and minorities? >> Well, if you look at traditional animation in the traditional entertainment fields that's a very mature industry and to break into that industry, you have to either have lots and lots of money or unfair distribution advantage, but VR, there's technological disruption, which means nobody has an advantage at all, means it's a level playing field and everybody can come in and start something, so, this is a perfect opportunity, when there's low barriers to entry of coming in, for women and minorities, anyone who wants their voice heard, to start companies or to make experiences and we can set the groundwork, because there's no one telling us what we can and can't do, because no one actually knows what we can and can't do yet. >> Right, right, but yet you are still of a female, asian figurehead of a studio, that will hopefully, someday be a major studio. You're working on it, but do you find that people take you as seriously in Hollywood? I mean, what are you coming up against? >> Well, it's really interesting, because I heard for even fundraising is one of the hardest parts of starting a company and there was a Stanford Research Study that showed that if you took a deck, a pitch deck for a company and you had a male voice-over versus a female voice-over the male voice-over was, I don't remember what, it was like 50% more likely to get funded than the woman with the same exact pitch deck, so I knew from that and they also show that if you are married and wear a ring you're taken more seriously, or if you're less attractive, also, you're taken more seriously and my hypothesis and some of the hypotheses out there, is it takes away the whole entire female attraction thing, like what does it mean to be an attractive female, so, I had to go into the meetings, knowing this. I even considered wearing a ring. I considered wearing a paper bag over my head. >> A bag over you head. Exactly, exactly. >> But at the same time I felt that I need to be myself and the best thing to, there's a correlation between the perceived leadership and confidence, that I needed to just go in there and be confident in myself so, I knew that, that could work against me, but I just needed to be myself, but I had to make sure that I was really confident and really believed in what I said and honestly, besides being confident and aggressive, I also, felt comfortable, because a lot of the people I talked to, I knew from my network and I had many of my male friends and female friends who knew these VC's, do the initial introduction, so I felt more comfortable going in, for them already knowing that I had somebody else saying that I was awesome. >> Yeah, and you've had many mentors and sponsors along the way too. >> Absolutely, I would say it's one of the most important things, for my career from the very beginning. When I graduated from business school, I actually emailed my mentors and said, "Here are the things I care about for finding a job." I didn't have to go find any jobs. They actually found all these jobs. for me, set up informational interviews, for me and I just went in and did it, all the informational interviews, got the offers and just choose one of them that I wanted to be in but, even for starting my company, my co-founder, Eric Darnell was a write and director of all four "Madagascar" films and I got introduced to him, through my mentor, Glen Entis who is the co-founder of PDI Dreamworks Animation and he was my mentor through Zynga and then, Gen Entis introduced me to Alvy Ray Smith, who is the co-founder of Pixar, who also became our advisor, Alvy Ray Smith, then introduced us to Glen Keane, who is the animator for "Little Mermaid", "Alaadin". >> The power of networks. >> It was all through the network and through my mentors that I found, a lot of the opportunities that I have and they also helped my through my personal life and how to navigate being entrepreneur and I rely on them so much. >> So, beyond finding the right mentor and sponsor what else would you give, your parting words to the young Maureen fans out there? >> I think there's a tendency for society to pressure you to conform, to money, fame, beauty and you don't need to listen to that and you don't need to be bucketed. I designed my own major at Stanford and with an eBay, I took four different roles. I just kept on creating my own roles and refusing to be bucketed as a creative or a suit and you can be who you are and create a category onto yourself and so, don't feel pressured to listen to what society is telling you. The other thing, is if you are faced with pushed back for being promoted and you feel like it's maybe because you're a woman, we have a tendency as women to start blaming ourselves and thinking there's something wrong with us, versus research shows men are most likely to blame the system, don't let it affect you and bring you down, because you need to actually be confident and believe in yourself in order to rise above. >> Great. Great advice. Maureen, it's been a pleasure having you on the show. Thanks so much. >> Thank you. >> And best of luck to you. >> Thank you, so much. >> Hope you win another Emmy. >> Thank you. >> Come back and talk to us again. >> Thank you. I'm Rebecca Knight, we'll have more from Grace Hopper, just after this. (techno music)

>> Narrator: It's theCUBE, covering Sapphire Now 2017. Brought to you by SAP Cloud platform and HANA Enterprise Cloud. >> Welcome, this is theCUBE. I'm John Furrier. Special Sapphire Now 2017 coverage here in theCUBE coverage. I'm John Furrier and I'm with Frank Pulambo with Cisco, on my left Margaret Anderson with SAP and Gary Gauba with CenturyLink. Some significant news announcing, in conjunction with Sapphire is the relationship between you guys around a significant cloud and cloud computing deal. Margaret, congratulations. This is significant because customers want more flexibility. They want to have a cloud native, they want to have the flexibility of a data center. Frank, talk about the deal from a Cisco perspective and the relationship to the context of this deal. >> John, we're super excited to be part of it, and especially with CenturyLink and a vendor like SAP. Really, our role in it is to help our customers run on print, run in the cloud and really our infrastructure products need to provide the simplicity to performance and the security for them to do that. >> Margaret, talk about the deal specifics. What does the deal entail? >> Well, it's a unique deal, because CenturyLink was already on of our partners for the HANA Enterprise cloud business. Then when the Cisco team came along and said, "You know what? "If we put our gear into the CenturyLink data centers "and power the SAP HANA Enterprise Cloud, "that would be a tremendous win for everybody." And so, we're very excited about that. >> Gary, I find this deal interesting because you have the engine of the SAP, you have the network and all the benefits that Cisco brings to the table. You're dealing with the implementation and manage to serve the customers. What's the important takeaway from this announcement? >> To me, I think if you take a step back, as we're siting down with some of the top SAP execs, and Chuck from Cisco and our CEO, Glen Post. We're talking, "Okay, how do we leverage "our joint assets together? "How do we create more value for our customers?" If you look at today, every customer the number one priority is digital transformation. The cloud obviously is part of that equation, right. The other bigger priority is, "Okay, how do I monetize this data "in this new connected world?" You're looking at 1.8 trillion connected objects by the year 2020, which will be generating 43 zetabytes of data. But let me quiz you out, how many zeros in a zetabyte? >> I think I knew this answer going in, since I got it wrong on our previous take. So, 12, it's 21. >> Maybe. (panel laughs) >> Okay. >> It's a lot. >> All right, I'm going to say 21 too. >> All right. >> Well, just a lot. It's a lot of data, a lot of devices. >> Yeah, I got to go with 21. It sounds like a good number. >> These are my cheat sheets, right. Anyway, they had-- >> Too many zeros. >> Too many zeros. >> Margaret: Yeah, too many zeros. >> And a significant, it's getting worse, the data tsunami is coming, it's already happening. >> It's happening. You think about the amount of data which is being generated. What we want to do is we want to provide highly secure, automated and in this case SAP certified private cloud platform from CenturyLink. Where we can really have the customers laser focus on the business they do. We help them with the deep platform which can help them monetize their data, help them drive their top line, increase the bottom line. Mitigate the rest in a fast manner and a highly secure manner as well, I'll talk more about that. >> Gary, you bring up a good point, and I want to get Frank involved in this and next I want to double down on the data comment. Because you factor in even the IoT, Internet of Things growth. It's going to be even more significant. Then, the security question pops up, because as the data centers move to the cloud and the hybrid and or pure cloud, the perimeter is gone. Now, the perimeter is how we managed security in the past. But now with the perimeter-less environments security is critical. How does this customer feel secure? And if you guys comment on the security aspect of how this relationship will roll out and the non-disruptive nature that you guys bring to the table? Because that's what customers want to know. >> John, from a Cisco perspective and Gary mentioned, when we talk about the digital transformation. And that's the buzzword in the industry, but most customers are looking for a business outcome. Really, to get that business outcome they have to make a business decision probably based off the technology like SAP HANA. This is mission critical information. At Cisco that's why we're in tune for what the network can do, because it's going to provide security at a lot of layers. Whether it's at the edge, whether it's at the aggregation point or all the way back to the data center, or into the cloud. The security at multi-layers and multiple places. With the bad actors out there that's the way you need to implement it. So you can start to bring this stuff together and give the customers the confidence that, "Hey, this transactionous data is going to be secure." >> Gary, the outcome that we're talking about is no hacks, no breaches, secure? >> Yeah, well it is highly secure, but you got to always be prepared. If you think about it, CenturyLink, we have 550,000 miles of fiber. Once we complete the level three acquisition, we'll add another 200,000 miles of fiber. You're talking about 750,000 miles of fiber touching 100 countries over six continents and 35 person of the global internet traffic going to our network. Once the level three acquisition is complete, we'll be the second largest network and we're critical to countries' infrastructure. There's a deep focus on cyber threats, things we do to avoid the bad actors getting in. >> I think that network intelligence too is going to give you more data to fight the actors as well. >> Absolutely, but to your earlier point, we truly, the situation gets compounded with this connected world revolution. When you are connecting these 1.8 trillion objects, you got to ensure each of these objects is secure. This is where the intersection of cyber security and IoT kicks in, and that's right in our wheel house. >> Don't forget latency, a word that's reering its head. More importantly it has people move across with IoT and across the cloud. Margaret, talk about the role that the HANA Enterprise Cloud plays in all of this. Is it the glue, is it the connecting tissue? What's the role of the HANA Cloud platform? HANA Enterprise Cloud, sorry. >> Yeah, I was just going to say. From a HANA Cloud perspective, it's a private managed cloud. We do an awful lot of work from an SAP perspective to make sure that the customer's environment is totally secure, but is also accessible all of the time for the customer. Because today, most businesses run 24/7 and they run all around the globe. When some team is working and this is their prime business day, later in the US business day, it's prime business time out in Asia, it's prime business time in Europe. You need a cloud that's always available, always up and it's secure. You can guarantee that no other data is leaking into it or out of it. That's really important for our customers, because if you think about it, we have customers today that are SAP customers running in the HANA Enterprise Cloud that are competitors to each other. You have to make sure that you can guarantee that there's not going to be any, data is going to sneak out the back door here or something is going to get in the back door over there. We have a fairly large security team at SAP. We have very stringent standards at SAP. Every time we form partnership with somebody, we say, "Here are our standards. "If you can't comply to these, "we can't do business with you as a partner." From our perspective, it's very important to know that anybody that we work with takes security as serious as we do. >> Well, certainly these guys are big partners, they're known companies, Cisco and CenturyLink. This is big. Talk about the unified, how you guys are going to help with the unified competing architecture that Cisco has. Because you guys have mastered this in the data center and you guys have a lot of experience with now the cloud. What's your vision on how this is going to play out from a Cisco standpoint? >> As Margaret described, their multi-tenancy in the SAP environment is a pillar for them, they have to make that work for the customer and that's something we've been driving at Cisco for a long time. When it comes to the computer platform, which we call UCS, we've been really the leader in converge infrastructure platforms for a long time with UCS and most recently now we're the number one platform for SAT with UCS in the converge environment. It's those attributes that have enabled us to really get inserted there to provide the customer value. >> You get the packet level of the network transformations, >> Yeah, John. >> So picked very nicely. >> You're more of a geek than I know. You're pretty good. >> Well, we follow Looking at the 5G network transformation you're seeing a lot of end-to-end architectures really winning out. I think why I'm excited about this new is essentially it's a partnership that's providing an need-to-end solution. Gary you are in the front line, you're the manager of service. Talk about the impact that you see this deal, because now you have to turn up services and have that customer touch point. >> If you look at how we're pulling together the solution. Obviously if look on the compute side we're using the UCS and on the switches there's the Cisco networking switches and then you're looking at the-- >> Nexus, Gary, Nexus. >> Nexus, all right. >> I didn't get to that. >> I wanted to make sure Frank is paying attention. (panel laughs) Then you're looking at the net app storage and the VMWare for virtualization. Then another key element we're using is Winamix, which integrates with the UCS and is pretty much integrating all the APIs for all these components. Obviously the powerful in memory HANA engine. All these things are pulled together to create a platform which we are providing to the customers, so that they can really focus on what they are good at, their business and they're delivering a powerful rally to their customers. What it does is, with the UCS and the Winamix and some of the other elements, we can onboard customers within this. What used to take us 32, 33 days now we can do that within 9 to 11 days which is a big big plus when a customer approaches us. >> It's a huge shape in the business model first of all, the client delivery on your end and service is significant. There's a lot of automation involved. In fact I was talking to a venture capitalist recently here in Silicon Valley and we were talking about network transformation, digital transformation and they invest in infrastructure. He goes, "Oh no, not a lot of new companies are coming out "with an infrastructure making hardware just as the cloud." He says plumbers are turning into machinists kind of indicating that a lot of automation is happening at the lower end of the stack and the action is happening higher up in the stack to your point. Things are happening faster in terms of level of services. >> Absolutely and I think another thing which we also bring to the table is that the deep essence expartice if you look at some of the mission critical applications whether it is SVOHANA, suite on HANA the ECC or the entire BI suite enterprise portal and the list goes on right. And that becomes critical so we can get them up and running very quickly and give us all in one on to the country. >> A significant endorsement with you, make a comment? >> Am just saying when you're talking about the visibility of the SAP platform this is something the customers thought about right. Its going to get to all levels of the organization and really they're making critical business decisions off of the SAP HANA platform its got to be there. I mean and that's obviously our challenge to meet that for customers. >> Margaret, HANA Enterprise cloud is the engine and you guys are providing significant on it. How do you see this collaboration playing out for customers? What should they expect from the collaboration of Cisco and CenturyLink? >> Well what we want the customer to have is a seamless experience. We tell them upfront that we have this partnership and that we're going to work together but we really want to keep them focused on the value that they get from running their applications. Because, think about it this way, when you get in the car and you turn on the engine, you want to drive away right? You know the engine is there, you know that the engine, depending on how much horse power you've invested in, depending on what kind of make and model, you get different things from it. But you know it's going to always be there, it's always going to work. Then if you want to enhance the engine, shall we say you want to know that the right people have the skillset for those enhancements. When customers come into the HANA Enterprise Cloud, sometimes they start off with the migration. One of the things you're doing is you're kind of sunsetting all the old stuff and you're helping them move on to our HANA platforms whether is S4, whether it's Suite on HANA, whether it's our VI suite, you're helping the customer get started. But that's the start, because after that there's all this other wonderful stuff that we develop at SAP being that we're a software house, like IoT and our Leonardo that's being announced in Sapphire and we want customers to be ready to take advantage of that. We tell the customers, "You need to move away from all the "old stuff because it's not reliable any more, "we want to move you into a different level "of reliability so that you're ready "to scale up your organization. "And so that you can do more business." And in the end the customer says, "Okay, I need to know the cloud is there for me, "I don't want to have to be thinking about it everyday. "What I want to be thinking about is "how can I do new business, how can I open up offices "in other parts of the world, "how can I transact business globally?" They're not really thinking about, okay, I need more stuff. If they need to expand we call up our partners and we say, Okay the customer wants five more terabits or something, 10 more terabits of whatever, and they just want it to happen. They don't want to have really long projects, they just want it to be there for them, they want to know that they can scale out and scale up. >> One thing just to add, I think Richard mentioned, it just becomes critical that the redundancy is there. This is where the MetroDR capabilities which we bring to the table. That becomes very powerful and if you look at what we have done is, we have created 10 parts, in the US you have those in Santa Clara, Dallas, Chicago and the Dc Sterling area. Then Singapore and the London market. The MetroDr capabilities are available in all the three markets. Which is a very critical element that the customers know. >> This is, the recovery certainly is really a big deal because you can think about after effect, you've got to think about the upfront in the service. You cannot just say, "Oh I forgot that thing about DR." >> The thing is, you get into a lot more preventive maintenance because this is where there are dedicated teams which are doing nothing. Once the bill processes down you get them up and running and how do you ensure we are able to predict before anything? >> I think this news is significant, one the size of the players involved but two, it speaks to the transformation that's happening in the market place. Customers are moving to a new way to operate their business and at the end of the day you guys are making it easier to run some of the core software that can scale. But it comes down to, how do you migrate and scale without having any kind rip and replace or lift and shift. Lift and shift, I would have said is codeword for rip and replace, but customers are on an evolution journey, there is no silver bullet in the cloud in my opinion, some we've covered that with you. You're thoughts on this, do you guys see it the same way? Customers aren't just going to move tomorrow, they're going to want an evolutionary approach, but they're going to want to have a scale. >> It's a journey, oftentimes customers say to us, what's the safest thing to do first? What pieces of my environment can I take into the cloud and then gradually over time, whether it's a few months, whether it's six months increments. It depends on how the customer is running their business. Can I keep moving it in there so I can keep getting value and at some point his is all done, goes away and now I'm fully transformed into the cloud. We see some customers who can do really quick projects, they're very agile and they're very nimble and so we can migrate them in and get them going and running very quickly. We've got other customers who are more risk adverse. They want to proceed a little slowly. We can do all of the flavors, depending on what they want to do and so when we first talk to customers about the HANA enterprise cloud, we talk about the options. We talk about what they can do, we make recommendations from an SAP perspective as to what we think their speed could be and then we help them get into the cloud. Because they might get left behind if they don't make that move. >> The modernization trend is key, people want to be modern, the apps are being written differently, the more the relevance is more towards agile and speed. I think I like this announcement you've got the reliability that Cisco would move in packets around the data center rock solid, the software, core engine on Enterprise cloud and scale the manage service level that can give you the flexibility to run whatever you want. I think that's interesting, did I get that right? >> I think the other thing is, it's not like we've been working on this thing for a week, we've bene working together on this solution for years now that you've got to make it right. There's different stages along the way. >> What are some of those key stages that a customer should know about? >> We were just talking about HANA Enterprise Cloud, could you run HANA in the cloud now? No, that's faded completely, it's already done you certainly can't. Customers can make decisions now to say, "Hey am I going to start on prime, am I going to start "in the cloud?" And go though that whole kind of process of what's the best thing. We're providing that flexibility but this is something we've been working on for a while and I think, when you look at the other solutions out there of customers making the right evaluation, you want to go with a CenturyLink SAP and Cisco to say, "Hey, this is the right muscle, "the right process behind us." >> This is amazing, you guys are a great deal, again Frank, senior vice president Cisco and Gary who is the chief relationship officer and president of the Advanced Solution Group and Margaret. All the senior executives, the principals here congratulation on this news, we'll wait to see how it plays out. >> Margaret: Thank you for having us today. >> Special keep coverage of Sapphire 2017, I'm John Furrier. Back with more live coverage after this short break. (electronic music)