>> Narrator: From theCUBE Studios in Palo Alto and Boston bringing you data-driven insights from theCUBE and ETR. This is breaking analysis with Dave Vellante >> Enterprise tech practitioners, like most of us they want to make their lives easier so they can focus on delivering more value to their businesses. And to do so, they want to tap best of breed services in the public cloud, but at the same time connect their on-prem intellectual property to emerging applications which drive top line revenue and bottom line profits. But creating a consistent experience across clouds and on-prem estates has been an elusive capability for most organizations, forcing trade-offs and injecting friction into the system. The need to create seamless experiences is clear and the technology industry is starting to respond with platforms, architectures, and visions of what we've called the Supercloud. Hello and welcome to this week's Wikibon Cube Insights powered by ETR. In this breaking analysis we give you a preview of Supercloud 2, the second event of its kind that we've had on the topic. Yes, folks that's right Supercloud 2 is here. As of this recording, it's just about four days away 33 guests, 21 sessions, combining live discussions and fireside chats from theCUBE's Palo Alto Studio with prerecorded conversations on the future of cloud and data. You can register for free at supercloud.world. And we are super excited about the Supercloud 2 lineup of guests whereas Supercloud 22 in August, was all about refining the definition of Supercloud testing its technical feasibility and understanding various deployment models. Supercloud 2 features practitioners, technologists and analysts discussing what customers need with real-world examples of Supercloud and will expose thinking around a new breed of cross-cloud apps, data apps, if you will that change the way machines and humans interact with each other. Now the example we'd use if you think about applications today, say a CRM system, sales reps, what are they doing? They're entering data into opportunities they're choosing products they're importing contacts, et cetera. And sure the machine can then take all that data and spit out a forecast by rep, by region, by product, et cetera. But today's applications are largely about filling in forms and or codifying processes. In the future, the Supercloud community sees a new breed of applications emerging where data resides on different clouds, in different data storages, databases, Lakehouse, et cetera. And the machine uses AI to inspect the e-commerce system the inventory data, supply chain information and other systems, and puts together a plan without any human intervention whatsoever. Think about a system that orchestrates people, places and things like an Uber for business. So at Supercloud 2, you'll hear about this vision along with some of today's challenges facing practitioners. Zhamak Dehghani, the founder of Data Mesh is a headliner. Kit Colbert also is headlining. He laid out at the first Supercloud an initial architecture for what that's going to look like. That was last August. And he's going to present his most current thinking on the topic. Veronika Durgin of Sachs will be featured and talk about data sharing across clouds and you know what she needs in the future. One of the main highlights of Supercloud 2 is a dive into Walmart's Supercloud. Other featured practitioners include Western Union Ionis Pharmaceuticals, Warner Media. We've got deep, deep technology dives with folks like Bob Muglia, David Flynn Tristan Handy of DBT Labs, Nir Zuk, the founder of Palo Alto Networks focused on security. Thomas Hazel, who's going to talk about a new type of database for Supercloud. It's several analysts including Keith Townsend Maribel Lopez, George Gilbert, Sanjeev Mohan and so many more guests, we don't have time to list them all. They're all up on supercloud.world with a full agenda, so you can check that out. Now let's take a look at some of the things that we're exploring in more detail starting with the Walmart Cloud native platform, they call it WCNP. We definitely see this as a Supercloud and we dig into it with Jack Greenfield. He's the head of architecture at Walmart. Here's a quote from Jack. "WCNP is an implementation of Kubernetes for the Walmart ecosystem. We've taken Kubernetes off the shelf as open source." By the way, they do the same thing with OpenStack. "And we have integrated it with a number of foundational services that provide other aspects of our computational environment. Kubernetes off the shelf doesn't do everything." And so what Walmart chose to do, they took a do-it-yourself approach to build a Supercloud for a variety of reasons that Jack will explain, along with Walmart's so-called triplet architecture connecting on-prem, Azure and GCP. No surprise, there's no Amazon at Walmart for obvious reasons. And what they do is they create a common experience for devs across clouds. Jack is going to talk about how Walmart is evolving its Supercloud in the future. You don't want to miss that. Now, next, let's take a look at how Veronica Durgin of SAKS thinks about data sharing across clouds. Data sharing we think is a potential killer use case for Supercloud. In fact, let's hear it in Veronica's own words. Please play the clip. >> How do we talk to each other? And more importantly, how do we data share? You know, I work with data, you know this is what I do. So if you know I want to get data from a company that's using, say Google, how do we share it in a smooth way where it doesn't have to be this crazy I don't know, SFTP file moving? So that's where I think Supercloud comes to me in my mind, is like practical applications. How do we create that mesh, that network that we can easily share data with each other? >> Now data mesh is a possible architectural approach that will enable more facile data sharing and the monetization of data products. You'll hear Zhamak Dehghani live in studio talking about what standards are missing to make this vision a reality across the Supercloud. Now one of the other things that we're really excited about is digging deeper into the right approach for Supercloud adoption. And we're going to share a preview of a debate that's going on right now in the community. Bob Muglia, former CEO of Snowflake and Microsoft Exec was kind enough to spend some time looking at the community's supercloud definition and he felt that it needed to be simplified. So in near real time he came up with the following definition that we're showing here. I'll read it. "A Supercloud is a platform that provides programmatically consistent services hosted on heterogeneous cloud providers." So not only did Bob simplify the initial definition he's stressed that the Supercloud is a platform versus an architecture implying that the platform provider eg Snowflake, VMware, Databricks, Cohesity, et cetera is responsible for determining the architecture. Now interestingly in the shared Google doc that the working group uses to collaborate on the supercloud de definition, Dr. Nelu Mihai who is actually building a Supercloud responded as follows to Bob's assertion "We need to avoid creating many Supercloud platforms with their own architectures. If we do that, then we create other proprietary clouds on top of existing ones. We need to define an architecture of how Supercloud interfaces with all other clouds. What is the information model? What is the execution model and how users will interact with Supercloud?" What does this seemingly nuanced point tell us and why does it matter? Well, history suggests that de facto standards will emerge more quickly to resolve real world practitioner problems and catch on more quickly than consensus-based architectures and standards-based architectures. But in the long run, the ladder may serve customers better. So we'll be exploring this topic in more detail in Supercloud 2, and of course we'd love to hear what you think platform, architecture, both? Now one of the real technical gurus that we'll have in studio at Supercloud two is David Flynn. He's one of the people behind the the movement that enabled enterprise flash adoption, that craze. And he did that with Fusion IO and he is now working on a system to enable read write data access to any user in any application in any data center or on any cloud anywhere. So think of this company as a Supercloud enabler. Allow me to share an excerpt from a conversation David Flore and I had with David Flynn last year. He as well gave a lot of thought to the Supercloud definition and was really helpful with an opinionated point of view. He said something to us that was, we thought relevant. "What is the operating system for a decentralized cloud? The main two functions of an operating system or an operating environment are one the process scheduler and two, the file system. The strongest argument for supercloud is made when you go down to the platform layer and talk about it as an operating environment on which you can run all forms of applications." So a couple of implications here that will be exploring with David Flynn in studio. First we're inferring from his comment that he's in the platform camp where the platform owner is responsible for the architecture and there are obviously trade-offs there and benefits but we'll have to clarify that with him. And second, he's basically saying, you kill the concept the further you move up the stack. So the weak, the further you move the stack the weaker the supercloud argument becomes because it's just becoming SaaS. Now this is something we're going to explore to better understand is thinking on this, but also whether the existing notion of SaaS is changing and whether or not a new breed of Supercloud apps will emerge. Which brings us to this really interesting fellow that George Gilbert and I RIFed with ahead of Supercloud two. Tristan Handy, he's the founder and CEO of DBT Labs and he has a highly opinionated and technical mind. Here's what he said, "One of the things that we still don't know how to API-ify is concepts that live inside of your data warehouse inside of your data lake. These are core concepts that the business should be able to create applications around very easily. In fact, that's not the case because it involves a lot of data engineering pipeline and other work to make these available. So if you really want to make it easy to create these data experiences for users you need to have an ability to describe these metrics and then to turn them into APIs to make them accessible to application developers who have literally no idea how they're calculated behind the scenes and they don't need to." A lot of implications to this statement that will explore at Supercloud two versus Jamma Dani's data mesh comes into play here with her critique of hyper specialized data pipeline experts with little or no domain knowledge. Also the need for simplified self-service infrastructure which Kit Colbert is likely going to touch upon. Veronica Durgin of SAKS and her ideal state for data shearing along with Harveer Singh of Western Union. They got to deal with 200 locations around the world in data privacy issues, data sovereignty how do you share data safely? Same with Nick Taylor of Ionis Pharmaceutical. And not to blow your mind but Thomas Hazel and Bob Muglia deposit that to make data apps a reality across the Supercloud you have to rethink everything. You can't just let in memory databases and caching architectures take care of everything in a brute force manner. Rather you have to get down to really detailed levels even things like how data is laid out on disk, ie flash and think about rewriting applications for the Supercloud and the MLAI era. All of this and more at Supercloud two which wouldn't be complete without some data. So we pinged our friends from ETR Eric Bradley and Darren Bramberm to see if they had any data on Supercloud that we could tap. And so we're going to be analyzing a number of the players as well at Supercloud two. Now, many of you are familiar with this graphic here we show some of the players involved in delivering or enabling Supercloud-like capabilities. On the Y axis is spending momentum and on the horizontal accesses market presence or pervasiveness in the data. So netscore versus what they call overlap or end in the data. And the table insert shows how the dots are plotted now not to steal ETR's thunder but the first point is you really can't have supercloud without the hyperscale cloud platforms which is shown on this graphic. But the exciting aspect of Supercloud is the opportunity to build value on top of that hyperscale infrastructure. Snowflake here continues to show strong spending velocity as those Databricks, Hashi, Rubrik. VMware Tanzu, which we all put under the magnifying glass after the Broadcom announcements, is also showing momentum. Unfortunately due to a scheduling conflict we weren't able to get Red Hat on the program but they're clearly a player here. And we've put Cohesity and Veeam on the chart as well because backup is a likely use case across clouds and on-premises. And now one other call out that we drill down on at Supercloud two is CloudFlare, which actually uses the term supercloud maybe in a different way. They look at Supercloud really as you know, serverless on steroids. And so the data brains at ETR will have more to say on this topic at Supercloud two along with many others. Okay, so why should you attend Supercloud two? What's in it for me kind of thing? So first of all, if you're a practitioner and you want to understand what the possibilities are for doing cross-cloud services for monetizing data how your peers are doing data sharing, how some of your peers are actually building out a Supercloud you're going to get real world input from practitioners. If you're a technologist, you're trying to figure out various ways to solve problems around data, data sharing, cross-cloud service deployment there's going to be a number of deep technology experts that are going to share how they're doing it. We're also going to drill down with Walmart into a practical example of Supercloud with some other examples of how practitioners are dealing with cross-cloud complexity. Some of them, by the way, are kind of thrown up their hands and saying, Hey, we're going mono cloud. And we'll talk about the potential implications and dangers and risks of doing that. And also some of the benefits. You know, there's a question, right? Is Supercloud the same wine new bottle or is it truly something different that can drive substantive business value? So look, go to Supercloud.world it's January 17th at 9:00 AM Pacific. You can register for free and participate directly in the program. Okay, that's a wrap. I want to give a shout out to the Supercloud supporters. VMware has been a great partner as our anchor sponsor Chaos Search Proximo, and Alura as well. For contributing to the effort I want to thank Alex Myerson who's on production and manages the podcast. Ken Schiffman is his supporting cast as well. Kristen Martin and Cheryl Knight to help get the word out on social media and at our newsletters. And Rob Ho is our editor-in-chief over at Silicon Angle. Thank you all. Remember, these episodes are all available as podcast. Wherever you listen we really appreciate the support that you've given. We just saw some stats from from Buzz Sprout, we hit the top 25% we're almost at 400,000 downloads last year. So really appreciate your participation. All you got to do is search Breaking Analysis podcast and you'll find those I publish each week on wikibon.com and siliconangle.com. Or if you want to get ahold of me you can email me directly at David.Vellante@siliconangle.com or dm me DVellante or comment on our LinkedIn post. I want you to check out etr.ai. They've got the best survey data in the enterprise tech business. This is Dave Vellante for theCUBE Insights, powered by ETR. Thanks for watching. We'll see you next week at Supercloud two or next time on breaking analysis. (light music)

>> From theCUBE studios in Palo Alto in Boston bringing you data driven insights from theCUBE and ETR. This is breaking analysis with Dave Vellante. >> The term Supercloud is somewhat new, but the concepts behind it have been bubbling for years, early last decade when NIST put forth a definition of cloud computing it said services had to be accessible over a public network essentially cutting the on-prem crowd out of the cloud conversation. Now a guy named Chuck Hollis, who was a field CTO at EMC at the time and a prolific blogger objected to that criterion and laid out his vision for what he termed a private cloud. Now, in that post, he showed a workload running both on premises and in a public cloud sharing the underlying resources in an automated and seamless manner. What later became known more broadly as hybrid cloud that vision as we now know, really never materialized, and we were left with multi-cloud sets of largely incompatible and disconnected cloud services running in separate silos. The point is what Hollis laid out, IE the ability to abstract underlying infrastructure complexity and run workloads across multiple heterogeneous estates with an identical experience is what super cloud is all about. Hello and welcome to this week's Wikibon cube insights powered by ETR and this breaking analysis. We share what we hope to learn from super cloud 22 next week, next Tuesday at 9:00 AM Pacific. The community is gathering for Supercloud 22 an inclusive pilot symposium hosted by theCUBE and made possible by VMware and other founding partners. It's a one day single track event with more than 25 speakers digging into the architectural, the technical, structural and business aspects of Supercloud. This is a hybrid event with a live program in the morning running out of our Palo Alto studio and pre-recorded content in the afternoon featuring industry leaders, technologists, analysts and investors up and down the technology stack. Now, as I said up front the seeds of super cloud were sewn early last decade. After the very first reinvent we published our Amazon gorilla post, that scene in the upper right corner here. And we talked about how to differentiate from Amazon and form ecosystems around industries and data and how the cloud would change IT permanently. And then up in the upper left we put up a post on the old Wikibon Wiki. Yeah, it used to be a Wiki. Check out my hair by the way way no gray, that's how long ago this was. And we talked about in that post how to compete in the Amazon economy. And we showed a graph of how IT economics were changing. And cloud services had marginal economics that looked more like software than hardware at scale. And this would reset, we said opportunities for both technology sellers and buyers for the next 20 years. And this came into sharper focus in the ensuing years culminating in a milestone post by Greylock's Jerry Chen called Castles in the Cloud. It was an inspiration and catalyst for us using the term Supercloud in John Furrier's post prior to reinvent 2021. So we started to flesh out this idea of Supercloud where companies of all types build services on top of hyperscale infrastructure and across multiple clouds, going beyond multicloud 1.0, if you will, which was really a symptom, as we said, many times of multi-vendor at least that's what we argued. And despite its fuzzy definition, it resonated with people because they knew something was brewing, Keith Townsend the CTO advisor, even though he frankly, wasn't a big fan of the buzzy nature of the term Supercloud posted this awesome Blackboard on Twitter take a listen to how he framed it. Please play the clip. >> Is VMware the right company to make the super cloud work, term that Wikibon came up with to describe the taking of discreet services. So it says RDS from AWS, cloud compute engines from GCP and authentication from Azure to build SaaS applications or enterprise applications that connect back to your data center, is VMware's cross cloud vision 'cause it is just a vision today, the right approach. Or should you be looking towards companies like HashiCorp to provide this overall capability that we all agree, or maybe you don't that we need in an enterprise comment below your thoughts. >> So I really like that Keith has deep practitioner knowledge and lays out a couple of options. I especially like the examples he uses of cloud services. He recognizes the need for cross cloud services and he notes this capability is aspirational today. Remember this was eight or nine months ago and he brings HashiCorp into the conversation as they're one of the speakers at Supercloud 22 and he asks the community, what they think, the thing is we're trying to really test out this concept and people like Keith are instrumental as collaborators. Now I'm sure you're not surprised to hear that mot everyone is on board with the Supercloud meme, in particular Charles Fitzgerald has been a wonderful collaborator just by his hilarious criticisms of the concept. After a couple of super cloud posts, Charles put up his second rendition of "Supercloudifragilisticexpialidoucious". I mean, it's just beautiful, but to boot, he put up this picture of Baghdad Bob asking us to just stop, Bob's real name is Mohamed Said al-Sahaf. He was the minister of propaganda for Sadam Husein during the 2003 invasion of Iraq. And he made these outrageous claims of, you know US troops running in fear and putting down their arms and so forth. So anyway, Charles laid out several frankly very helpful critiques of Supercloud which has led us to really advance the definition and catalyze the community's thinking on the topic. Now, one of his issues and there are many is we said a prerequisite of super cloud was a super PaaS layer. Gartner's Lydia Leong chimed in saying there were many examples of successful PaaS vendors built on top of a hyperscaler some having the option to run in more than one cloud provider. But the key point we're trying to explore is the degree to which that PaaS layer is purpose built for a specific super cloud function. And not only runs in more than one cloud provider, Lydia but runs across multiple clouds simultaneously creating an identical developer experience irrespective of a state. Now, maybe that's what Lydia meant. It's hard to say from just a tweet and she's a sharp lady, so, and knows more about that market, that PaaS market, than I do. But to the former point at Supercloud 22, we have several examples. We're going to test. One is Oracle and Microsoft's recent announcement to run database services on OCI and Azure, making them appear as one rather than use an off the shelf platform. Oracle claims to have developed a capability for developers specifically built to ensure high performance low latency, and a common experience for developers across clouds. Another example we're going to test is Snowflake. I'll be interviewing Benoit Dageville co-founder of Snowflake to understand the degree to which Snowflake's recent announcement of an application development platform is perfect built, purpose built for the Snowflake data cloud. Is it just a plain old pass, big whoop as Lydia claims or is it something new and innovative, by the way we invited Charles Fitz to participate in Supercloud 22 and he decline saying in addition to a few other somewhat insulting things there's definitely interesting new stuff brewing that isn't traditional cloud or SaaS but branding at all super cloud doesn't help either. Well, indeed, we agree with part of that and we'll see if it helps advanced thinking and helps customers really plan for the future. And that's why Supercloud 22 has going to feature some of the best analysts in the business in The Great Supercloud Debate. In addition to Keith Townsend and Maribel Lopez of Lopez research and Sanjeev Mohan from former Gartner analyst and principal at SanjMo participated in this session. Now we don't want to mislead you. We don't want to imply that these analysts are hopping on the super cloud bandwagon but they're more than willing to go through the thought experiment and mental exercise. And, we had a great conversation that you don't want to miss. Maribel Lopez had what I thought was a really excellent way to think about this. She used TCP/IP as an historical example, listen to what she said. >> And Sanjeev Mohan has some excellent thoughts on the feasibility of an open versus de facto standard getting us to the vision of Supercloud, what's possible and what's likely now, again, I don't want to imply that these analysts are out banging the Supercloud drum. They're not necessarily doing that, but they do I think it's fair to say believe that something new is bubbling and whether it's called Supercloud or multicloud 2.0 or cross cloud services or whatever name you choose it's not multicloud of the 2010s and we chose Supercloud. So our goal here is to advance the discussion on what's next in cloud and Supercloud is meant to be a term to describe that future of cloud and specifically the cloud opportunities that can be built on top of hyperscale, compute, storage, networking machine learning, and other services at scale. And that is why we posted this piece on Answering the top 10 questions about Supercloud. Many of which were floated by Charles Fitzgerald and others in the community. Why does the industry need another term what's really new and different? And what is hype? What specific problems does Supercloud solve? What are the salient characteristics of Supercloud? What's different beyond multicloud? What is a super pass? Is it necessary to have a Supercloud? How will applications evolve on superclouds? What workloads will run? All these questions will be addressed in detail as a way to advance the discussion and help practitioners and business people understand what's real today. And what's possible with cloud in the near future. And one other question we'll address is who will build super clouds? And what new entrance we can expect. This is an ETR graphic that we showed in a previous episode of breaking analysis, and it lays out some of the companies we think are building super clouds or in a position to do so, by the way the Y axis shows net score or spending velocity and the X axis depicts presence in the ETR survey of more than 1200 respondents. But the key callouts to this slide in addition to some of the smaller firms that aren't yet showing up in the ETR data like Chaossearch and Starburst and Aviatrix and Clumio but the really interesting additions are industry players Walmart with Azure, Capital one and Goldman Sachs with AWS, Oracle, with Cerner. These we think are early examples, bubbling up of industry clouds that will eventually become super clouds. So we'll explore these and other trends to get the community's input on how this will all play out. These are the things we hope you'll take away from Supercloud 22. And we have an amazing lineup of experts to answer your question. Technologists like Kit Colbert, Adrian Cockcroft, Mariana Tessel, Chris Hoff, Will DeForest, Ali Ghodsi, Benoit Dageville, Muddu Sudhakar and many other tech athletes, investors like Jerry Chen and In Sik Rhee the analyst we featured earlier, Paula Hansen talking about go to market in a multi-cloud world Gee Rittenhouse talking about cloud security, David McJannet, Bhaskar Gorti of Platform9 and many, many more. And of course you, so please go to theCUBE.net and register for Supercloud 22, really lightweight reg. We're not doing this for lead gen. We're doing it for collaboration. If you sign in you can get the chat and ask questions in real time. So don't miss this inaugural event Supercloud 22 on August 9th at 9:00 AM Pacific. We'll see you there. Okay. That's it for today. Thanks for watching. Thank you to Alex Myerson who's on production and manages the podcast. Kristen Martin and Cheryl Knight. They help get the word out on social media and in our newsletters. And Rob Hof is our editor in chief over at SiliconANGLE. Does some really wonderful editing. Thank you to all. Remember these episodes are all available as podcasts wherever you listen, just search breaking analysis podcast. I publish each week on wikibon.com and Siliconangle.com. And you can email me at David.Vellantesiliconangle.com or DM me at Dvellante, comment on my LinkedIn post. Please do check out ETR.AI for the best survey data in the enterprise tech business. This is Dave Vellante for theCUBE insights powered by ETR. Thanks for watching. And we'll see you next week in Palo Alto at Supercloud 22 or next time on breaking analysis. (calm music)

(upbeat music) >> On August 9th at 9:00 am Pacific, we'll be broadcasting live from theCUBE Studios in Palo Alto, California. Supercloud22, an open industry event made possible by VMware. Supercloud22 will lay out the future of multi-cloud services in the 2020s. John Furrier and I will be hosting a star lineup, including Kit Colbert, VMware CTO, Benoit Dageville, co-founder of Snowflake, Marianna Tessel, CTO of Intuit, Ali Ghodsi, CEO of Databricks, Adrian Cockcroft, former CTO of Netflix, Jerry Chen of Greylock, Chris Hoff aka Beaker, Maribel Lopez, Keith Townsend, Sanjiv Mohan, and dozens of thought leaders. A full day track with 17 sessions. You won't want to miss Supercloud22. Go to thecube.net to mark your calendar and learn more about this free hybrid event. We'll see you there. (upbeat music)

(upbeat music) >> Every company needs great salespeople, it's one of the most lucrative professions out there. And there's plenty of wisdom and knowledge that's been gathered over the years about selling. We've heard it all, famous quotes from the greatest salespeople of our time, like Zig Ziglar and Jeffrey Gitomer, and Dale Carnegie and Jack Welch, and many others. Things like, "Each of us has only 24 hours in a day, "it's all about how we use our time." And, "You don't have to be great to start, "but you have to start to be great." And then I love this one, "People hate to be sold, but they love to buy." "There are no traffic jams on the extra mile, "make change before you have to." And the all time classic, "Put that coffee down. "Coffee is for closers." Thousands of pieces of sales advice are readily available in books, videos, on blogs and in podcasts, and many of these are free of charge. So why would entrepreneurs start a company to train salespeople? And how is it that sharp investors are pouring millions of dollars into this space? Hello everyone, and welcome to this Cube Video Exclusive, my name is Dave Vellante, and today we welcome Paul Fifield who's the co-founder and CEO of Sales Impact Academy who's going to answer these questions and share some exciting news on the startups. Paul, welcome to "The Cube" good to see you again. >> Yeah, good to see you again, Dave, great to be here. >> Hey, so before we get into the hard news, tell us a little bit about the Sales Impact Academy, why'd you start the company, maybe some of the fundamentals of this market, your total available market, who you're targeting, you know, what's the premise behind the company? >> Yeah sure. So I mean, I started the company, it was actually pretty organic in the way it began. I had a 10 year career as a CRO and it was, you know, had a couple of great hits with two companies, but it was a real struggle to basically, you know, operate as a CRO and learn your craft at the same time. And so when I left my last company, I kind of got out there, I wanted to kind of give back a little bit and I started doing some voluntary teaching in and around London, and I actually, one of the companies I started was in New York so I got schooled very much on a sort of US approach to how you build a modern you know, go to market and sales operation. Started going out there, doing some teaching, realized that so many people just didn't have a clue about how to build a scalable and predictable revenue function, and I kind of felt sorry for them. So I literally started doing some, you know, online classes myself, got my co-founder Alex to put curriculum together as well and we literally started just doing online classes, very live, very organic, just a Google Drive and some decks, and it really just blew up from there. >> That's amazing. I mean, so you've my, you know, tongue and cheek up front, but people might wonder, why do you need a platform 'cause there's so much free information out there? Is it to organize, is it a discipline thing? Explain that. >> Well, I think the way I sort of see this is that is that the lack of structured learning and education is actually one of the greatest educational travesties, I think, of the last 50 years, okay. Now sales and go to market is a huge global profession, right? Half the world's companies are B2B, so roughly that's a proxy for half the world's GDP, right? Which is $40 trillion of GDP. Now that 40 trillion rests on kind of the success of the growth and the sales functions of all those companies. Yet in its infinite wisdom, the global education system literally just ignored sales and go to market as a profession. Some universities are kind of catching up, but it's really too little too late. So what I sort of say to people, you imagine this Dave, right. You imagine if the way that law worked as a profession let's say, is that there's no law school, there's no law training, there's no even in work professional continuous professional development in law. The way that it works is you leave university, join a company, start practicing law and just use like YouTube just to maybe like, you know, where you're struggling, just use YouTube to like work out what's going on. The legal profession would be in absolute chaos. And that's what's happened in the sales and go to market profession, okay. What this profession desperately desperately needs is structured learning, good pedagogy, good well designed course and curriculum. And here's the other thing, right? Is the sort of paradox of infinite information is that just because all the information is out there, right, doesn't mean it's actually a good learning experience. Like, where do you find it? What's good? What's not good? And also the other thing I'd point out is that there is this kind of myth that all the information is out there on the internet. But actually what we do, and we'll come into it in a second is, the people teaching on our platform are the elite people from the industry. They haven't got time to do blog posts and just explain to people how they operate. They're going from company to company working at like, you know, working at these kind of elite companies. And they're the people that teach, and that information is not readily available and freely out there on the internet. >> Yeah, real opportunity, you made some great points there. I think business schools are finally starting to teach a little bit about public speaking and presenting, but nobody's teaching us how to sell. As Earl Nightingale says, "To some degree we're all salespeople, "selling our family on living the good life" or whatever. What movie we want to see tonight. But okay, let's get to the hard news. You got fresh funding of 22 million, tell us about that, congratulations. You know, the investors, what else can you share with us? >> Sure. Well, I mean, obviously, you know, immensely proud. We started from very sort of humble beginnings, as I said, we've now scaled very rapidly, we're a subscription business, we're a SaaS business. We'll come onto some of the growth metrics shortly, but just in a couple of years, you know, the last year which ended January, we grew 500% from year one, we're now well over 125 people, and I'm very, very, very honored, flattered, humbled that MIT, obviously one of those prestigious universities in the world, has taken a direct investment by their endowment fund, HubSpot Ventures. Another Boston great has also taken a direct investment as well. They actually began as a customer and loved what we were doing so much that they then decided to make an investment. Stage 2 Capital who invested in our seed round pretty much tripled down, played a huge role in helping us assemble MIT and HubSpot ventures as investors, and they continue to be an incredible VC giving us amazing, amazing support that their LP network of go to market leaders is second to none. And then Emerge Education, who is our pre-seed investor, they're actually based in London, also joined this round as well. >> Great, well actually, let's jump ahead. Let's talk about the metrics. I mean, if Stage Two is involved, they're hardcore. What can you share with us about, you know, everybody's chasing AR and NR and the like, what can you share with us? >> They are both pretty important. Well, I think from a headcount perspective, so as I mentioned our fiscal ends at the end of January, each year. We've gone from 25 to over 125 employees in that time. We've gone from 82 to 260 customers also in that time. And customers now include HubSpot, Gong, Klaviyo, GitHub, GT, Six Cents, so some really sort of major SaaS companies in the space. Our revenue's grown significantly with 5X. So 500% increase in revenue year over year, which is pretty fast, very proud of that. Our learning community has gone from over 3000 people to almost 15,000 professionals, and that makes us comfortably, the largest go to market learning community in the world. >> How did you decide when to scale? What were the sort of signals that said to you, "Okay, we're ready, "we have product market fit, "we can now scale the go to market." What were the signals there, Paul? >> Yeah. Well, I mean, I think for a very small team to achieve that level of growth in customers, to be kind of honest with you, like it's the pull that we're getting from the market. And I think the thing that has surprised me the most, perhaps in the last 12 months, is the pull we're getting from the enterprise. We're you know, I can't really announce, we've actually got a huge pilot with one of the largest companies actually in the world which is going fantastically well, our pipeline for enterprise customers is absolutely huge. But as you can imagine, if you've got distributed teams all over the world, we're living and working in this kind of hybrid world, how on earth do you kind of upscale all those people, right, that are, like I say, that are so distributed. It's impossible. Like in work, in the office delivery of training is pretty much dead, right? And so we sort of fill this really big pain, we solved this really, really big pain of how to effectively upskill people through this kind of live curriculum and this live teaching approach that we have. So I think for me, it's the pull that we're getting from the market really meant that you know, we have to double down. There is such a massive TAM, it is absolutely ridiculous. I mean, I think there are 20 million people just in sales and go to market in tech alone, right. And I mentioned to you earlier, half the world's companies effectively, you know, are B2B and therefore represent, you know, at its largest scope, our TAM. >> Excellent, thank you for that. Tell us more about the product and the platform. How's it work if I'm a customer, what type of investment do I have to make both financially? And what's my time commitment? How do you structure that? >> So the model is basically on a seat model. So roughly speaking, every seat's about a thousand dollars per year per rep. The lift is light. So we've got a very low onboarding, it's not a highly complex technical product, right? We've got a vast curriculum of learning that covers learning for, you know, SDRs, and the AEs, and CS reps, and leadership management training. We're developing curriculum for technical pre-sales, we're developing curriculum for revenue operations. And so it's very, very simple. We basically, it's a seat model, people literally just send us the seats and the details, we get people up and running in the platform, they start then enrolling and we have a customer success team that then plots out learning journeys and learning pathways for all of our customers. And actually what's starting to happen now, which is very, very exciting is that, you know, we're actually a key part of people's career development pathway. So to go from you know, SDR1 let's say to SDR2, you have to complete these three courses with Sales Impact Academy, and let's say, get 75% in your exam and it becomes a very powerful and simple way of developing career pathway. >> Yeah, so really detailed curriculum. So I was going to say, do I as a sales professional, do I pick and choose the things that are most relevant for me? Or are people actually going through a journey in career progression, or maybe both? >> Yeah, it's a mixture of both. We tend to see now, we're sort of starting to standardize, but really we're developing enough curriculum that over, let's say a 15 year period, you could start with us as an SDR and then end as a chief revenue officer, you know, running the entire function. This is the other thing about the crazy world of go to market. Very often, people are put into roles and it's sink or swim. There's no real learning that happens, there's no real development that happens before people take these big steps. And what this platform does so beautifully is is it equips people with the right skills and knowledge before they take that next step in their profession and in their career. And it just dramatically improves their chances of succeeding. >> Who are the trainers? Who's leading the classes, how do you find these guys, how do you structure? What are the content, you know, vectors, where's all that come from? >> Yeah. So the sort of secret source of what we do, beyond just the live instruction, beyond the significant amount of peer to peer learning that goes on, is that we go and source the absolute most elite people in go to market to teach, okay. Now I mentioned to you before, you've got these people that are going from like job to job at the very like the sort of peak of their careers, working for these incredible companies, it's that knowledge that we want to get access to, right. And so Stage 2 Capital is an incredible resource. The interesting thing about Stage 2 Capital as you know Dave, you know, run by Mark Roberge, who was on when we spoke last year and also Jay Po is all the LPs of Stage 2 Capital represent 3 to 400 of the most elite go to market professionals in the world. So, you know, about seven or eight of those are now on an advisory board. And so we have access to this incredible pool of talent. And so we know by consulting these amazing people who are the best people in certain aspects of go to market. We reach out to them and very often they're at a stage in their career where they're really kind of willing to give back, of course there are commercials around it as well, and there's lots of other benefits that we provide our teachers and our faculty, and what we call our coaches. But yeah, we source the very, very best people in the world to teach. >> Now, how does it work as a user of your service? Is it all on demand? Do you do live content or a combination? >> Yeah look, one of the big differentiators is this is a live delivery of learning, okay. Most learning online is typically done on demand, self-directed, and there's a ton of research. There's a great blog post on Andrew's recent site. A short time ago, which is talking about how the completion rates of on demand learning are somewhere between 3 and 6%. That is like, that's awful. >> Terrible. >> I was like why bother? However, we're seeing through that live instruction. So we teach two, one hour classes a week, that's it. We're upskilling very busy people, they're stressed, they've got targets. We have to be very, very cognizant of that. So we teach two, one hour classes a week. Typically, you know, Monday and a Wednesday, or a Tuesday and a Thursday. And that pace of learning is about right, it's kind of how humans learn as well. You know, short bursts of information, and then put that learning and those skills that you've acquired in class literally to work minutes after the class finishes. And so through that, and it sits in your calendar like a meeting, it doesn't feel overwhelming, you're learning together as a team as well. And all that combined, we see completion rates often in excess of 80% for our courses. >> Okay, so they block that time out- >> In the calendar, yeah. >> And they make an investment. Go ahead, please. >> Yeah yeah, exactly, sorry Dave. Yeah, yeah, exactly. So like, you know, we have course lengths. So one of our shorter courses are like four hours long over two weeks. And again, it's just literally in the calendar. We also teach what we call The Magic Learning Hour. And the magic learning hour is this one specific hour in the day that enables teams all over the western hemisphere to join the same class. And that magic learning hour is eight o'clock Pacific 11 o'clock Eastern, >> 4: 00 PM over in the UK, and 5:00 PM in the rest of Europe. And that one time in the day means that these enterprises have got teams all over the western hemisphere joining that class, learning together as a team, plus it's in the calendar and it's that approach is why we're seeing such high engagement and completion. >> That's very cool, the time zone thing. Now who's the target buyer? Are you selling only to sales teams? Can I as an individual purchase your service? >> Yeah, that's a good question. Currently it's a very much like a B2B motion. As I mentioned earlier on, we're getting an enormous pull from the enterprise, which is very exciting. You know, we have an enterprise segment, we have sort of more of a startup earlier stage segment, and then we have a mid-market segment that we call our sort of strategic, and that's typically and most of like venture backed, fast growth tech companies. So very much at the moment a B2B motion. We're launching our own technology platform in the early summer, and then later on this year we're going to be adding what's called PLG or a product led growth, so individuals can actually sign up to SIA. >> Yeah, I mean, I think you said $1,000 per year per rep, is that right? I mean, that's- >> Yeah. >> That's a small investment for an individual that wants to be part of, you know, this community and grow his or her career. So that's the growth plan? You go down market I would imagine, you talked about the western hemisphere, there's international opportunities maybe, local language. What's the growth plan? >> Yeah, I mean look, we've identified the magic learning hour for the middle east and APAC, which is eight o'clock in the morning in Istanbul, right. Is 5:00 PM in Auckland, it's quite fun trying to work out like what this optimum magic learning hour is. What's incredible is we teach in that time and that opens up the whole of the middle east and the whole of APAC, right, right down to Australia. And so once we're teaching the curriculum in those two slots, that means literally you can have teams in any country in the world, I think apart from Hawaii, you can actually access our live learning products in work time and that's incredibly powerful. So we have so many like axis of growth, we've got single users as I mentioned, but really Dave that's single users we'll be winning from the enterprise and that will represent pipeline that we could then potentially convert as well. And look, you make a very good point. You know, we've seen students are now leaving university with over $100,000 dollars in debt. We've got a massive, massive debt problem here in the US with student debt. You could absolutely sign up to our platform at let's say a hundred bucks a month, right. And probably within six months, gain enough knowledge and skill to walk into a $60,000 a year based salary job as an SDR, that's a huge entry level salary. And you could do that without even going to university. So there could be a time here where we become a really viable alternative to actually even going to university. >> I love it. The cost education going through the roof, it's out of reach for so many people. Paul, congratulations on the progress, the fresh funding. Great to have you back in "The Cube." We'd love to have you back and follow your ascendancy. I think great things ahead for you guys. >> Thank you very much, Dave. >> All right, and thank you for watching. This is Dave Vellante for "The Cube, we'll see you next time. (upbeat music)

>>Hey everyone. Lisa Martin here with John farrier. Welcome to the women in tech global event, featuring international women's day. John, this is an exciting day, March 8th, 2022. How did this all get started? >>Well, we started it out when we realized there was more stories to be told with virtual, with COVID. The virtualization of virtual events allowed us to do more stories. So we've been on this new format where we're creating seasons and episodic events, meaning you can still do an event and do 30 interviews like we're doing here for international women's day from around the world. We could have done a hundred there's enough stories out there. There's thousands of stories out there that need to be told, need to be scaled. And so we're just scratching the surface. So we are just starting to do is celebrate international women's day with as many videos we could do in a week, which is 30 and be part of widths and Stanford here in California, as part of their events with Stanford. And we're going to continue with international women's day. >>It's the big celebration, it's the big day, but then when it's over, we're going to continue with more episodes. So this is technically, I guess, season one episode, one of the international women's community site portal is going to be open and open to everyone. Who's going to be a community vibe and, uh, we'll get sponsors, but overall it's about bringing people together, creating tribes, letting people form their own communities and hopefully, uh, making the world a better place and supporting the mission, which is a great mission. Diversity inclusion and equity is a big mission. Uh, it's good for everyone. Everyone wins. >>Everyone does win. What are some of the interesting conversations that you've had with our international women's day guests that really were poignant to you? >>Well, the, one of the things was interesting by region. They had different kind of, um, feelings. The Asia Pacific was heavily skewed on a lot of international diversity around culture. Latin America was just all cloud computing. For instance, I felt that to be very technical, uh, more than agents in the interviews. Um, um, more diversity I study in Asia Pacific and Amy. It was really interesting because you have a lot going on there right now in Europe. So, um, and I'll see from a technical standpoint, data sovereignty and sustainability are two big themes. So from a tech trend standpoint, it was really amazing leaders. We interviewed, um, from technical, uh, folks to analysts, to senior executives in the C-suite. So it really good mix of people in the program. Uh, for today, >>We also had a young girl that I had the chance to speak with her and her father. And it was such a lovely conversation cause it reminded me of my dad's relationship with me. But she was told in high school age, no, you can't do physics. No, you can't do computer science. So the parents pulled her out of school. And so the, and she's brand new in her career path. And it was so nice to hear, to see that, that family, the role models within the family saying she wants to do physics and computer science. Let's find a place for her to be able to do that and have her start being able to, to build her own personal board of directors. At the age of like 22, 23, >>We hit an entrepreneur down in New Zealand. I interviewed she was from indigenous area and she had no milk or food on the table. They were so poor. They could barely get food. She worked her way through it and went to school. Education was number when it goes, she was so persistent, she got her education. And now she's the CEO of an AI company, amazing person. And she's like, Hey, there's no wall I can't run through. So that attitude was just so refreshing. And that was a consistent this year and it wasn't an in your face. It was just more of we're here, we're kicking butt. So let's keep it going. So on the entrepreneurial side, I found that really awesome on the senior leadership side, it was very much, um, community oriented, very open about sharing their experiences and also being a sponsor. So you're going to hear a lot about breaking the bias, but it's also about sponsoring opportunities and then helping people get involved so that they can get understand biases because everyone brings biases to the table. So I personally learned a lot this, this, this, uh, event. >>Yeah. I think the, the light that was shined on the bias was incredibly important. You know, the break, the bias, as you said, is the theme of this year's international women's day. And I, and I asked everybody that I spoke with, what does that mean to you? And where do you think we are on that journey? A world free of bias and stereotypes and discrimination. Obviously we're not there yet, but a lot of the women talked about the fact that that light is shining brightly, that the awareness is there, that for diversity equity and inclusion and having that awareness, there is a great launching launching pad, if you will, for being able to make more progress on actually breaking the bias. >>Yeah. That was a great point. I would also say to add to that by saying a lot of comments were on the same theme of check your bias when you fall, you speak in meetings. And it was just a lot of like protocol tactical, uh, ways to do things like, think about other people in the room versus just barreling ahead. Most guys do that actually. Um, and so that was another instructful thing. I think the other thing too was is that there was, again, more and more sharing. I mean, we had one person that you interviewed, her name was Anne green. Yeah. She's doing her own series. Uh, we're content. She's interviewing people, she's being a mentor and sharing it through content, Manny theory of AWS in Singapore, she's in space and Aero science talking about how the satellites are helping in the Ukraine, give information to everyone on the ground, not just governments and that's helping democracy. And that she's really excited that that contributes to some good there. Um, and she fled from a town where it was bombed. She was in a war zone and she escaped and got educated. So education's a theme. Um, don't let anyone tell you, you can't do it. Uh, and don't think there's only one pathway, right? This is tons of opportunities for participating in the tech economy for good, uh, in, in, in tech. So those are the keys. >>That's always been one of my favorite themes when we do women in tech events, John is that there is no direct pathway necessarily. I always love understanding those stories, but this year, one of the things that also was really clear was that women feeling what can't I do. And that sentiment was really echoed throughout. I think everybody that I spoke with that there was no, can I do this? Why can't I Not confidence? Which is palpable. Even when you're doing an interview by zoom, you can feel it. You can be inspired by, >>Well, at least a year, you do all the, a lot of the interviews. You're the face I had, you know, step aside for you because you're amazing. But one of the things you, you get appreciate this and love to get your reaction. One of the things I observed this year was because it was international focus, there was huge cube demand to be come to their region. We had one of the guests that won from Bahrain. She's like, I'll do the cube here. I'll be the host. So I think there was a real appetite for this kind of open dialogue conversations where they want the cube to come to their area. And so I know anyone watching wants to be a cube host in those areas, let us know, um, we're open. And to me that was more refreshing. Cause you know, me, I always wanna see the cube global go everywhere. But this year people are actually turning on their own cameras. They're doing their own interviews. They're sharing content and content creates community and bonding. And that was the big experience I saw this year was a lot more user generated activity engagement with each other in the group. >>I think that may have even been a product of the last two years of the pandemic of people really understanding the importance of community and collaboration and that it can be done via if you're only limited to video, you can do that. You can build a community and grow it and foster it in that way and create the content that really helps support it. >>That's a great point. That's actually one of the guests said COVID polled the future forward and digital. We see the value and other on the cyber side, um, Sally, as I mentioned there, um, earlier who we interviewed before, she's a cyber policy analyst and she's so smart. She's like, yeah, this is putting fold forward. And people understand cyber now, cyber misinformation, cyber war, the role of working at home, being isolated versus community. These are core societal issues that need to be solved and it's not just code that solves them. So it's going to be solved by the community. And that's really, that was the key. One of the key messages. It was very refreshing. >>It was very refreshing. I always love hearing the stories. I, the more personal the story, the more real it is and the more opportunities I think that it unlocks for the audience watching. Yeah, >>I mean, we had one person said she did a project on the side. It's going to be your big initiative within Amazon. You know, Amazon, one of our sponsors has a slogan think big, but deep dive deep. And she took a project on about educating, um, young girls and young women. And it turned out to be basically a build lab inside schools. And it took off. It is so successful side project, side hustle gone, gone big. So again, sparks of creativity, innovation can come from anywhere. It's just great stories. >>Another thing that came up in several of the conversations that I had was the data, the data that support that organizations that have at least 30% females at the executive level are better performing organizations. They are more profitable as well. So it was fun to kind of call out if we're talking about data science, what not the data that supports why international women's day is what it is, why it's becoming even bigger than that and the importance of showcasing those voices so that she can be what she can see. >>Yeah. Amazing stories. I got to say it again. I think the virtual studios where we have now with the pandemic is going to give us much more opportunities to get those stories out. And Lisa, you've done an amazing job. Your interviews were awesome. Thank you. And we can do a hundred. We'll give you a hundred interviews a week. >>We can, are you setting me up? No, it was fun. The international influence this year was fun. I mean, I think I started one of my interview days at 6:00 AM and it was just exciting to be able to connect to different parts of the world and to hear these stories and for the cube to be able to be the platform that is sharing all of that >>And the diversity of the interviews itself and the diversity of the environments that for instance, in Asia Pacific and your are diverse areas and they see it it's much further along. They live it every day. They know the benefits. So that again, that was another aha moment for us, I think this year. >>So how many, how many segments do we have for international women's day John >>30 segments, uh, 32 counting our little segments here. So 32 interviews. Um, we're going to probably add a section on the site for people to submit stories like a directory, uh, this, a zillion things going on, women of web three, Sandy, Carter's putting on an event. I know there's a security called. She S she scarcity events, she security, uh, going on women in security. Um, there's tons of activities it's vibrant tomorrow. Today. It'll be very much bumping up. So we'll try to curate as much links as possible. >>Awesome. John has been great doing this program with you. I look forward to seeing the interviews and being inspired by the many, many stories. You're going to be watching the cubes coverage of women in tech global event, featuring international women's day for John furrier. I'm Lisa Martin. We'll see you soon.

>>Speed is everything in a hyper competitive climate. The faster we get insights from data and get data products to market. The faster we grow and the more competitive we become, this is Dave Volante from the cutie inviting you to join us on Thursday, August 5th, for cloud areas, industry insights. We'll look at the biggest challenges facing businesses today, especially the need to access and leverage data at an accelerated velocity. You'll hear from industry leaders like Nick Collison, whose cloud era's president, Rob Bearden, the CEO of Cloudera, Michelle Goetz from Forrester. You'll hear from Nvidia and industry experts in insurance, manufacturing, retail, and public sector. Who can address your biggest concerns? Like how do I remove constraints and put data at the core of my business, streaming begins at 9:00 AM Pacific on the Q3 65. You're a leader in global enterprise tech coverage.

(gentle upbeat music) >> People hate to be sold, but they love to buy. We become what we think about, think, and grow rich. If you want to gather honey, don't kick over the beehive. The world is replete with time-tested advice and motivational ideas for aspiring salespeople, Dale Carnegie, Napoleon Hill, Norman Vincent Peale, Earl Nightingale, and many others have all published classics with guidance that when followed closely, almost always leads to success. More modern personalities have emerged in the internet era, like Tony Robbins, and Gary Vaynerchuk, and Angela Duckworth. But for the most part, they've continued to rely on book publishing, seminars, and high value consulting to peddle their insights and inspire action. Welcome to this video exclusive on theCUBE. This is Dave Vellante, and I'm pleased to welcome back Professor Mark Roberge, who is one of the Managing Directors at Stage 2 Capital, and Paul Fifield, who's the CEO and Co-Founder of Sales Impact Academy. Gentlemen, welcome. Great to see you. >> You too Dave and thanks. >> All right, let's get right into it. Paul, you guys are announcing today a $4 million financing round. It comprises $3 million in a seed round led by Stage 2 and a million dollar in debt financing. So, first of all, congratulations. Paul, why did you start Sales Impact Academy? >> Cool, well, I think my background is sort of two times CRO, so I've built two reasonably successful companies. Built a hundred plus person teams. And so I've got kind of this firsthand experience of having to learn literally everything on the job whilst delivering these very kind of rapid, like achieving these very rapid growth targets. And so when I came out of those two journeys, I literally just started doing some voluntary teaching in and around London where I now live. I spend a bunch of time over in New York, and literally started this because I wanted to sort of kind of give back, but just really wanted to start helping people who were just really, really struggling in high pressure environments. And that's both leadership from sense of revenue leadership people, right down to sort of frontline SDRs. And I think as I started just doing this voluntary teaching, I kind of realized that actually the sort of global education system has done is a massive, massive disservice, right? I actually call it the greatest educational travesty of the last 50 years, where higher education has entirely overlooked sales as a profession. And the knock-on consequences of that have been absolutely disastrous for our profession. Partly that the profession is seen as a bit sort of embarrassing to be a part of. You kind of like go get a sales job if you can't get a degree. But more than that, the core fundamental within revenue teams and within sales people is now completely lacking 'cause there's no structured formal kind of like learning out there. So that's really the problem we're trying to solve on the kind of like the skill side. >> Great. Okay. And mark, always good to have you on, and I got to ask you. So even though, I know this is the wheelhouse for you and your partners, and of course, you've got a deep bench of LPs, but lay out the investment thesis here. What's the core problem that you saw and how are you looking at the market? >> Yeah, sure, Dave. So this one was a special one for me. We've spoken in the past. I mean, just personally I've always had a similar passion to Paul that it's amazing how important sales execution is to all companies, nevermind just the startup ecosystem. And I've always personally been motivated by anything that can help the startup ecosystem increase their success. Part of why I teach at Harvard and try to change some of the stuff that Paul's talking about, which is like, it's amazing how little education is done around sales. But in this particular one, not only personally was I excited about, but from a fun perspective, we've got to look at the economic outcomes. And we've been thinking a lot about the sales tech stack. It's evolved a ton in the last couple of decades. We've gone from the late '90s where every sales VP was just, they had a thing called the CRM that none of their reps even used, right? And we've come so far in 20 years, we've got all these amazing tools that help us cold call, that help us send emails efficiently and automatically and track everything, but nothing's really happened on the education side. And that's really the enormous gap that we've seen is, these organizations being much more proactive around adopting technology that can prove sales execution, but nothing on the education side. And the other piece that we saw is, it's almost like all these companies are reinventing the wheel of looking in the upcoming year, having a dozen sales people to hire, and trying to put together a sales enablement program within their organization to teach salespeople sales 101. Like how to find a champion, how to develop a budget, how to develop sense of urgency. And what Paul and team can do in the first phase of essay, is can sort of centralize that, so that all of these organizations can benefit from the best content and the best instructors for their team. >> So Paul, exactly, thank you, mark. Exactly what do you guys do? What do you sell? I'm curious, is this sort of, I'm thinking in my head, is this E-learning, is it really part of the sales stack? Maybe you could help us understand that better. >> Well, I think this problem of having to upscale teams has been around like forever. And kind of going back to the kind of education problem, it's what's wild is that we would never accept this of our lawyers, our accountants, or HR professionals. Imagine like someone in your finance team arriving on day one and they're searching YouTube to try and work out how to like put a balance sheet together. So it's a chronic, chronic problem. And so the way that we're addressing this, and I think the problem is well understood, but there's always been a terrible market, sort of product market fit for how the problem gets solved. So as mark was saying, typically it's in-house revenue leaders who themselves have got massive gaps in their knowledge, hack together some internal learning that is just pretty poor, 'cause it's not really their skillset. The other alternative is bringing in really expensive consultants, but they're consultants with a very single worldview and the complexity of a modern revenue organization is very, very high these days. And so one consultant is not going to really kind of like cover every topic you need. And then there's the kind of like fairly old fashioned sales training companies that just come in, one big hit, super expensive and then sort of leave again. So the sort of product market fit to solve, has always been a bit pretty bad. So what we've done is we've created a subscription model. We've essentially productized skills development. The way that we've done that is we teach live instruction. So one of the big challenges Andreessen Horowitz put a post out around this so quite recently, one of the big problems of online learning is that this kind of huge repository of online learning, which puts all the onus on the learner to have the discipline to go through these courses and consume them in an on-demand way is actually they're pretty ineffective. We see sort of completion rates of like 7 to 8%. So we've always gone from a live instruction model. So the sort of ingredients are the absolute very best people in the world in their very specific skill teaching live classes just two hours per week. So we're not overwhelming the learners who are already in work, and they have targets, and they've got a lot of pressure. And we have courses that last maybe four to like 12 hours over two to sort of six to seven weeks. So highly practical live instruction. We have 70, 80, sometimes even 90% completion rates of the sort of live class experience, and then teams then rapidly put that best practice into practice and see amazing results in things like top of funnel, or conversion, or retention. >> So live is compulsory and I presume on-demand? If you want to refresh you have an on demand option? >> Yeah, everything's recorded, so you can kind of catch up on a class if you've missed it, But that live instruction is powerful because it's kind of in your calendar, right? So you show up. But the really powerful thing, actually, is that entire teams within companies can actually learn at exactly the same pace. So we teach it eight o'clock Pacific, 11 o'clock Eastern, >> 4: 00 PM in the UK, and 5:00 PM Europe. So your entire European and North American teams can literally learn in the same class with a world-class expert, like a Mark, or like a Kevin Dorsey, or like Greg Holmes from Zoom. And you're learning from these incredible people. Class finishes, teams can come back together, talk about this incredible best practice they've just learned, and then immediately put it into practice. And that's where we're seeing these incredible, kind of almost instant impact on performance at real scale. >> So, Mark, in thinking about your investment, you must've been thinking about, okay, how do we scale this thing? You've got an instructor component, you've got this live piece. How are you thinking about that at scale? >> Yeah, there's a lot of different business model options there. And I actually think multiple of them are achievable in the longer term. That's something we've been working with Paul quite a bit, is like, they're all quite compelling. So just trying to think about which two to start with. But I think you've seen a lot of this in education models today. Is a mixture of on-demand with prerecorded. And so I think that will be the starting point. And I think from a scalability standpoint, we were also, we don't always try to do this with our investments, but clearly our LP base or limited partner base was going to be a key ingredient to at least the first cycle of this business. You know, our VC firm's backed by over 250 CRO CMOs heads of customer success, all of which are prospective instructors, prospective content developers, and prospective customers. So that was a little nicety around the scale and investment thesis for this one. >> And what's in it for them? I mean, they get paid. Obviously, you have a stake in the game, but what's in it for the instructors. They get paid on a sort of a per course basis? How does that model work? >> Yeah, we have a development fee for each kind of hour of teaching that gets created So we've mapped out a pretty significant curriculum. And we have about 250 hours of life teaching now already written. We actually think it's going to be about 3000 hours of learning before you get even close to a complete curriculum for every aspect of a revenue organization from revenue operations, to customer success, to marketing, to sales, to leadership, and management. But we have a development fee per class, and we have a teaching fee as well. >> Yeah, so, I mean, I think you guys, it's really an underserved market, and then when you think about it, most organizations, they just don't invest in training. And so, I mean, I would think you'd want to take it, I don't know what the right number is, 5, 10% of your sales budget and actually put it on this and the return would be enormous. How do you guys think about the market size? Like I said before, is it E-learning, is it part of the CRM stack? How do you size this market? >> Well, I think for us it's service to people. A highly skilled sales rep with an email address, a phone and a spreadsheet would do really well, okay? You don't need this world-class tech stack to do well in sales. You need the skills to be able to do the job. But the reverse, that's not true, right? An unskilled person with a world-class tech stack won't do well. And so fundamentally, the skill level of your team is the number one most important thing to get right to be successful in revenue. But as I said before, the product market for it to solve that problem, has been pretty terrible. So we see ourselves 100%. And so if you're looking at like a com, you look at Gong, who we've just signed as a customer, which is fantastic. Gong has a technology that helps salespeople do better through call recording. You have Outreach, who is also a customer. They have technologies that help SDRs be more efficient in outreach. And now you have Sales Impact Academy, and we help with skills development of your team, of the entirety of your revenue function. So we absolutely see ourselves as a key part of that stack. In terms of the TAM, 60 million people in sales are on, according to LinkedIn. You're probably talking 150 million people in go to market to include all of the different roles. 50% of the world's companies are B2B. The TAM is huge. But what blows my mind, and this kind of goes back to this why the global education system has overlooked this because essentially if half the world's companies are B2B, that's probably a proxy for the half of the world's GDP, Half of the world's economic growth is relying on the revenue function of half the world's companies, and they don't really know what they're doing, (laughs) which is absolutely staggering. And if we can solve that in a meaningfully meaningful way at massive scale, then the impact should be absolutely enormous. >> So, Mark, no lack of TAM. I know that you guys at Stage 2, you're also very much focused on the metrics. You have a fundamental philosophy that your product market fit and retention should come before hyper growth. So what were the metrics that enticed you to make this investment? >> Yeah, it's a good question, Dave, 'cause that's where we always look first, which I think is a little different than most early stage investors. There's a big, I guess, meme, triple, triple, double, double that's popular in Silicon Valley these days, which refers to triple your revenue in year one, triple your revenue in year two, double in year three, and four, and five. And that type of a hyper growth is critical, but it's often jumped too quickly in our opinion. That there's a premature victory called on product market fit, which kills a larger percentage of businesses than is necessary. And so with all our investments, we look very heavily first at user engagement, any early indicators of user retention. And the numbers were just off the charts for SIA in terms of the customers, in terms of the NPS scores that they were getting on their sessions, in terms of the completion rate on their courses, in terms of the customers that started with a couple of seats and expanded to more seats once they got a taste of the program. So that's where we look first as a strong foundation to build a scalable business, and it was off the charts positive for SIA. >> So how about the competition? If I Google sales training software, I'll get like dozens of companies. Lessonly, and MindTickle, or Brainshark will come up, that's not really a fit. So how do you think about the competition? How are you different? >> Yeah, well, one thing we try and avoid is any reference to sales training, 'cause that really sort of speaks to this very old kind of fashioned way of doing this. And I actually think that from a pure pedagogy perspective, so from a pure learning design perspective, the old fashioned way of doing sales training was pull a whole team off site, usually in a really terrible hotel with no windows for a day or two. And that's it, that's your learning experience. And that's not how human beings learn, right? So just even if the content was fantastic, the learning experience was so terrible, it was just very kind of ineffective. So we sort of avoid kind of like sales training, The likes of MindTickle, we're actually talking to them at the moment about a partnership there. They're a platform play, and we're certainly building a platform, but we're very much about the live instruction and creating the biggest curriculum and the broadest curriculum on the internet, in the world, basically, for revenue teams. So the competition is kind of interesting 'cause there is not really a direct subscription-based live like learning offering out there. There's some similar ish companies. I honestly think at the moment it's kind of status quo. We're genuinely creating a new category of in-work learning for revenue teams. And so we're in this kind of semi and sort of evangelical sort of phase. So really, status quo is one of the biggest sort of competitors. But if you think about some of those old, old fashioned sort of Miller Heimans, and then perhaps even like Sandlers, there's an analogy perhaps here, which is kind of interesting, which is a little bit like Siebel and Salesforce in the sort of late '90s, where in Siebel you have this kind of old way of doing things. It was a little bit ineffective. It was really expensive. Not accessible to a huge space of the market. And Salesforce came along and said, "Hey, we're going to create this cool thing. It's going to be through the browser, it's going to be accessible to everyone, and it's going to be really, really effective." And so there's some really kind of interesting parallels almost between like Siebel and Salesforce and what we're doing to completely kind of upend the sort of the old fashioned way of delivering sort of sales training, if you like. >> And your target customer profile is, you're selling to teams, right? B2B teams, right? It's not for individuals. Is that correct, Paul? >> Currently. Yeah, yeah. So currently we've got a big foothold in series A to series B. So broadly speaking out, our target market currently is really fast growth technology companies. That's the sector that we're really focusing on. We've got a very good strong foothold in series A series B companies. We've now won some much larger later stage companies. We've actually even won a couple of corporates, I can't say names yet, but names that are very, very, very familiar and we're incredibly excited by them, which could end up being thousand plus seat deals 'cause we do this on a per seat basis. But yeah, very much at the moment it's fast growth tech companies, and we're sort of moving up the chain towards enterprise. >> And how do you deal with the sort of maturity curve, if you will, of your students? You've got some that are brand new, just fresh out of school. You've got others that are more seasoned. What do you do, pop them into different points of the curriculum? How do you handle it? >> Yeah we have, I'll say we have about 30 courses right now. We have about another 15 in development where post this fundraise, we want to be able to get to around about 20 courses that we're developing every quarter and getting out to market. So we're literally, we've sort of identified about 20 to 25 key roles across everything within revenue. That's, let's say revenue ops, customer success, account management, sales, engineering, all these different kinds of roles. And we are literally plotting the sort of skills development for these individuals over multiple, multiple years. And I think what we've never ceases to amaze me is actually the breadth of learning in revenue is absolutely enormous. And what kind of just makes you laugh is, this is all of this knowledge that we're now creating it's what companies just hope that their teams somehow acquire through osmosis, through blogs, through events. And it's just kind of crazy that there is... It's absolutely insane that we don't already exist, basically. >> And if I understand it correctly, just from looking at your website, you've got the entry level package. I think it's up to 15 seats, and then you scale up from there, correct? Is it sort of as a seat-based license model? >> Yeah, it's a seat-based model, as Mark mentioned. In some cases we sell, let's say 20 or $30,000 deal out the gate and that's most of the team. That will be maybe a series A, series B deal, but then we've got these land and expand models that are working tremendously well. We have seven, eight customers in Q1 that have doubled their spend Q2. That's the impact that they're seeing. And our net revenue retention number for Q2 is looking like it's going to be 177% to think exceeds companies like Snowflakes. Well, our underlying retention metrics, because people are seeing this incredible impact on teams and performance, is really, really strong. >> That's a nice metric compare with Snowflake (Paul laughs) It's all right. (Dave and Paul laugh) >> So, Mark, this is a larger investment for Stage 2 You guys have been growing and sort of upping your game. And maybe talk about that a little bit. >> Yeah, we're in the middle of Fund II right now. So, Fund I was in 2018. We were doing smaller checks. It was our first time out of the gate. The mission has really taken of, our LP base has really taken off. And so this deal looks a lot like more like our second fund. We'll actually make an announcement in a few weeks now that we've closed that out. But it's a much larger fund and our first investments should be in that 2 to $3 million range. >> Hey, Paul, what are you going to do with the money? What are the use of funds? >> Put it on black, (chuckles) we're going to like- (Dave laughs) >> Saratoga is open. (laughs) (Mark laughs) >> We're going to, look, the curriculum development for us is absolutely everything, but we're also going to be investing in building our own technology platform as well. And there are some other really important aspects to the kind of overall offering. We're looking at building an assessment tool so we can actually kind of like start to assess skills across teams. We certify every course has an exam, so we want to get more robust around the certification as well, because we're hoping that our certification becomes the global standard in understanding for the first time in the industry what individual competencies and skills people have, which will be huge. So we have a broad range of things that we want to start initiating now. But I just wanted to quickly say Stage 2 has been nothing short of incredible in every kind of which way. Of course, this investment, the fit is kind of insane, but the LPs have been extraordinary in helping. We've got a huge number of them are now customers very quickly. Mark and the team are helping enormously on our own kind of like go to market and metrics. I've been doing this for 20 years. I've raised over 100 million myself in venture capital. I've never known a venture capital firm with such value add like ever, or even heard of other people getting the kind of value add that we're getting. So I just wanted to a quick shout out for Stage 2. >> Quite a testimony of you guys. Definitely Stage 2 punches above its weight. Guys, we'll leave it there. Thanks so much for coming on. Good luck and we'll be watching. Appreciate your time. >> Thanks, Dave. >> Thank you very much. >> All right, thank you everybody for watching this Cube conversation. This is Dave Vellante, and we'll see you next time.

(smooth music) >> Hi, everybody. This is Dave Vellante, and with me is Sandeep Singh. He's the vice president of storage marketing at Hewlett Packard Enterprise, and we're going to riff on some of the trends in the industry, what we're seeing, and we got a little treat for you, Sandeep. Great to see you, man. >> Dave, it's a pleasure to be here. >> You and I have known each other for a long time. We've had some great discussions, some debates, (chuckles) some intriguing mind benders. What are you seeing out there in storage? So much has changed. What are the key trends you're seeing? And let's get into it. >> Yeah. Across the board, as you said, so much has changed. When you reflect back at the underlying transformation that's taking place with data, cloud, and AI across the board, first of all, for our customers, they're seeing this massive data explosion that literally now spans edge to core to cloud. They're also seeing a diversity of the application workloads across the board. The emphasis that it's placing is on the complexity that underlies overall infrastructure and data management. Across the board, we're hearing a lot from customers about just the underlying infrastructure and complexity, and the infrastructure sprawl. And then the second element of that is really extending into the complexity of data management. >> So it's interesting to talk about data management. You remember you and I were in... Well, you were in Andover. I don't know. It was probably like five years ago. And all we were talking about was media, flash this and flash that, and at the time that was kind of the hot storage topic. Well, flash came in, addressed some of the clicks that we historically talked about. Now the problem statement is really kind of, quote unquote, metaphorically moving up the stack, if you will. You mentioned management. But let's dig into that a little bit. I mean, what is management? I mean, a lot of people... That means different things to different people. You talk to a database person or a backup person. How do you look at management? What does that mean to you? >> Yeah, Dave. You mentioned that flash came in, and it actually accelerated the overall speed and latency that storage was delivering to the application workloads. But fundamentally, when you look back at storage over a couple of decades, the underlying way of how you're managing storage hasn't fundamentally changed. There's still an incredible amount of complexity for ITs. It's still a manual admin-driven experience for customers. And what that's translating to is, more often than not, IT is in the world of firefighting, and it leaves them unable to help with the more strategic projects to innovate for the business. And basically IT has that pressure point of moving beyond that, and helping bring greater levels of agility that line of business owners are asking for, and to be able to deliver on more of the strategic projects. So that's one element of it. The second element that we're hearing from customers about is as more and more data just continues to explode from edge to core to cloud, and as basically the infrastructure has grown from just being on-prem, to being at the edge, to being in the cloud, now that complexity is expanding from just being on-prem to across multiple different clouds. So when you look across the data life cycle, how do you store it? How do you secure it? How do you basically protect it, and archive it, and analyze that data? That end to end life cycle management of data, today resides on just a fragmented set of overall infrastructure, and tools, and processes, and administrative boundaries. That's creating a massive challenge for customers. And the impact of that, ultimately, is essentially comes at a cost to agility, to innovation, and ultimately business risk. >> Yeah, so we've seen obviously the cloud has addressed a lot of these problems, but the problem is the cloud is in the cloud. And much of my stuff, most of my stuff, isn't in the cloud. (chuckles) So I have all these other workloads that are either on-prem, and now you've got this emerging edge. And so I wonder if we could just talk a little vision here for a minute. I mean, what I've been envisioning is this abstraction layer that cuts across all, whether... It doesn't really matter where it is. If it's on-prem, if it's across cloud, if it's in the cloud, on the edge. We could talk about what that all means. But if customers that I talk to, they're sort of done with the complexity of that underlying infrastructure. They want technology to take care of that. They want automation. They want AI brought into that equation. And it seems like we're on the cusp of the decade where that might happen. What's your take? >> Well, yeah. Certainly, I mentioned that data cloud and AI are really the disruptive forces that are propelling the digital transformation for customers. Cloud has set the standard for agility, and AI-driven insights and intelligence are really helping to make the underlying infrastructure invisible. And yet a lot of their application workloads and data is on-prem and is increasingly growing at the edge. So they want that same experience to be able to truly bring that agility to wherever their data and apps load. And that's one of the things that we're continuing to hear from customers. >> And this problem's just going to get worse. I mean, we... For decades we marched to the cadence of Moore's law, and everybody's kind of forgets about Moore's law. And they'll say, "Ah, it's dying," or whatever. But actually, when you look at the processing power that's coming out now, it's not... It's more than doubling every two years, quadrupling every two years. So now you've got this capability in your hands, and application designers, storage companies, networking companies, they're going to have all this power to now bring in AI and do things that we've never even imagined before. So it's not about the box, and the speeds and feeds of the box. It's really more about this abstraction layer that I was talking about, the management, if you will, that you were discussing, and what we can do in terms of being able to power new workloads, machine intelligence. It's this kind of ubiquitous... Call it the cloud, but it's expanding pretty much everywhere in every part of our lives, (chuckles) even to the edge. You think about autonomous vehicles, you think about factories. It's actually quite mind boggling where we're headed. >> It is, and you touched upon AI, and certainly when you look at infrastructure, for example, there's been a ton of complexity in infrastructure management. One of the studies that was done, actually by IDC, indicated that over 90% of the challenges that arise, for example, ultimately down at the storage infrastructure layer that's powering the apps, ultimately, arises from way above the stack all the way from the server layer on down, or even the virtual machine layer. And there, for example, AI ops for infrastructure has become a game changer for customers to be able to bring the power of AI, and machine learning, and multi-variate analysis to be able to predict and prevent issues. Dave, you also touched upon edge, and across the board, what we're seeing is the enterprise edge is becoming that frontier for customer experiences, and the opportunity to reimagine customer experiences, as well as just the frontier for commerce that's happening when you look at retail, and manufacturing, and/or financial services. So across the board, with the data growth that's happening, and this edge becoming the strategic frontier for delivering the customer experiences, how you power your application workloads there, how you deliver that data, and protect that data, and be able to seamlessly manage that overall infrastructure, as you mentioned, abstracted away at a higher level, becomes incredibly important for our customers. >> It's so interesting to hear how the conversation's changing, I'd like to say. I go back to whatever it was, five years ago, we're talking about flash, storage class memory, and NVMe, and those things are still there, but your emphasis now, you're talking about machine learning, AI, math around deep learning. It's really software is really what you're focusing on these days. >> Very much so. Certainly, this notion of software and services that are delivering and unlocking a whole new experience for customers, that's really the game changer going forward for customers, and that's what we're focused on. >> Well, I said we had a little surprise for you. So you guys are having an event on May 4th. It's called Unleash the Power of Data. What's that event all about, Sandeep? >> Yeah. We are very much excited about our May 4th event. As you mentioned, it's called Unleash the Power of Data. And as most organizations today are data driven, and data is at the heart of what they're doing, we're excited to invite everyone to join this event. And through this event, we're unveiling a new vision for data that accelerates the data-driven transformation from edge to cloud. This event promises to be a pivotal event, and one that IT admins, cloud architects, virtual machine admins, vice-presidents, directors of IT, and CIOs really won't want to miss. Across the board, this event is just bringing a new way of articulating the overall problem statement, and a market-in focused the articulation of the trends that we were just discussing. It's an event that's going to be hosted by business and technology journalist, Shibani Joshi. It will feature a market-in panel with a focus on the crucial role that data is playing in customers' digital transformation. It will also include and feature Antonio Neri, CEO of HPE, and Tom Black, senior vice president and general manager of HPE storage business, and industry experts, including Julia Palmer, research vice president at Gartner. We will unveil game-changing HPE innovations that will make it possible for organizations across edge to cloud to unleash the power of data. >> Sounds like a great event. I presume I can go to hpe.com. And what? Get information. Is it a registered event? How does that all work? >> Yeah, we invite everyone to visit hpe.com, and by visiting there, you can click and save the date of May 4th at 8:00 AM Pacific. We invite everyone to join us. We couldn't be more excited to get to this event, and be able to share the vision and game-changing HPE innovations. >> Awesome. So it's... So I don't have to register, right? I don't have to give up my three children's name, and my social security number to attend your event, is that right? (chuckles) >> No registration required. Come by, click on hpe.com. Save the date on your calendar. And we very much look forward to having everyone join us for this event. >> I love it. It's pure content event. I'm not going to get a phone call afterwards saying, "Hey, buy some stuff from me." That could come other channels, so that's good. (chuckles) Thank you for that. Thanks for providing that service to the industry. I'm excited to see what you guys are going to be announcing that day. And look, Sandeep, I mean, like I said, we've known each other a while. We've seen a lot of trends, but the next 10 years, it ain't going to look like the last 10, is it? >> It's going to be very different, and we couldn't be more excited. >> Well, Sandeep, thanks so much for coming to theCUBE, and riffing with me on the industry, and giving us a preview for your event. Good luck with that, and always great to see you. >> Thanks a lot, Dave. Always great to see you as well. >> All right, and thank you, everybody. This is Dave Vellante for theCUBE, and we'll see you next time. (smooth music)

>>from >>around >>The globe. It's the cube with digital coverage of IBM think 2021 brought to you >>by IBM. Welcome back to IBM Think 2021. This is the cubes ongoing coverage where we go out to the events in this case virtually to extract the signal from the noise. Now we're gonna talk about one of the deepest customer relationships in the tech business with Pete Bill, who is the IBM managing director for American Express. Pete, great to see you. Thanks for coming on. >>Thanks for having me Dave. >>So as I said, this is one of the deepest vendor client relationships. I mean, it's more than that because you're not a vendor, your partner, very deep relationship, many, many, you know, decades plus uh, executives know each other. There's been a some senior executives from American Express as I recall, came over to IBM of course, famously Lou Gerstner um, but but talk about the, just give us the overview of the evolution of that partnership. >>Yeah, well, as you rightfully mentioned, uh the relationship is long and deep, its over 100 years. I mean the original deal was probably around the club buying clocks and uh scales and all that kind of stuff and it evolved over time. But what it does indeed create is a long deep lasting relationship as a fundament for doing business and uh yeah that business has gone through a lot of cycles over the last decades, as you say, uh from from buying stuff. But I would say over time evolving really into a partnership around services, mutual business back and forth, exchanging executives on board level american Express executives on the board of IBM and vice versa. So yeah, it's a very very deep long relationship of two iconic companies in in Manhattan. >>Yeah, so it's got to be more than just buying stuff. Obviously there's a lot of business being transacted but you've got to intimate your title has american Express in it. So you've got to intimately understand your client's business. I mean that's I guess that's always the case, but we're taking it to another level here, >>aren't we? Yeah. Yeah, absolutely. I mean, so what you really are after and what we do is IBM is really get into the shoes basically of american Express trying to support their business to their clients. So american Express is very focused on small and medium business. So we tip into how can we help the small and medium businesses part of the american Express custom set and how can we help evolve their business models, that technology their services to serve their clients better because in the old days indeed. To your point it was like, oh we wanted to buy the right stuff and then we use that to do our thing. But that the technology today, the area in which we operate is completely different. If you don't understand the the client of american Express, we cannot serve american express as a company. So it is indeed very important and it is therefore deeper and it requires way more focused on the clients of american express than in the old days I would say. >>Well the pandemic must have been a challenging uh environment. Of course. I mean you know people aren't out shopping as much although you know people are waiting, they can't wait to get back out. They say it's gonna be like Woodstock with the american express cards. But so so maybe talk a little bit about how how you work together during the pandemic. >>Yeah. So well first of all like anybody, we're all work from home. But american express really uh I would say almost reengaged on what is core and the threat that used to support to small and medium business. So american Express started this stand for small initiative led by steve Square himself about how can we enable the small enterprises uh in doing business in the covid period? What do they need? I mean, yeah, they need money, but they also need help like how to deal with your financials with your people. Can we use the spare time to do more education? And so IBM was one of the partners that jumped on board immediately to say, okay, let us help in that platform support you were necessary with the platform but definitely help you in that platform to reach out to the small and medium enterprises uh specifically in the new york area. And like many other partners, we all got on board and I think it got another focus. Again, I mean small and medium business has always been a focus, but it's different when so many companies are struggling right now. And so we get a got on board. And I think that that is really a very clear partnership expression. I would say, >>how do you measure success with with american Express? What are some of the key things that you guys look at? How, how have you evolved that over time? >>Well, ultimately, I would say it's client satisfaction in the end, it sounds like an open door, but it really is. I mean the real the real measurement, I mean there's always money measurements back and forth. You can argue that that is of course you need to do solid business. There's no discussion there, but I would say it's where do we align on the strategic intent from both companies and let me elaborate a second on that one. If american Express is really transforming its business to become way more, I would say uh cloud enabled hybrid technologies enabled. Uh we provide a lot of that material. Uh so we are really working together on trying to leverage each other in building that hybrid platform that will enable that future. And why do you need that? Well, because american Express needs to be dynamic and getting fit, excellent board, getting exchanges with with with with new companies going way faster. It's not a traditional old style anymore where you could go for transformations for years now it needs to be on the spot. Um so we show our strategies are really well aligned and I would say the real measurement of success is how can we now make that to the benefit of american Express? And on the back of that we will do good business. So uh client satisfaction should be the primary one. Strategic alignment important. And then of course doing the sound business on the back of that for both sides, >>financial services firms have always been pretty savvy when it comes to applying technology to business some of the most demanding customers and more advanced. Uh So you know the american express is likely already on a digital transformation prior to the covid hitting at the same time. It talks about it being accelerated. But I think what people miss is that it wasn't well they don't miss it but you know to think about it in this way, it wasn't planned, it was like forced. And so you just you have no choice, you couldn't think about it, you just have to do an act. And so on the one hand, okay, that's good. It was a forcing function. It also served as a Petri just but on the other hand, I'm sure a lot of mistakes were made now as we exit the pandemic step back and say okay wow, we learned a lot now. We can make a more planned full approach and really go deeper and lean in over the next several years. What are your thoughts on that and how does it relate to what you guys are doing with american Express? >>I think that's a very good point. I agree. It's what you see is that uh this indeed has forced us in a lot of things. I mean I think the good news is american Express was already enabled for a lot of that new technology. They have invested, they have a lot of very skilled good people uh a very clear strategy and what they were after this and they put more pressure on it. I think what you will see happening in the foreseeable future after we get out of all of this is that the, let's say the urgency to complete the transformation on cloud and data will become even more crucial. And so the priority will become higher and it will not be just higher because of the Turkish wanting it to do it, but because the business needs it. So uh needed from a risk perspective, they needed, from an agility perspective, go to market of new products. Uh they need to really move fast. It's a fast moving market, you get a lot of the media competition is there? Uh so to enable that the move to get new technologies and faster is becoming pivotal and crucial. And I think for now it's more of almost like a survival statement. We need to get through this bubble of Covid as soon as that's done, we need to think way more on the structural elements of debt and how we enable a hybrid strategy going forward. >>So in the spirit of you need to understand your customer in this case american Express and understand their business. An american express is uh make you laugh anytime I call american Express, you know, if I have to work out a problem or whatever. Uh and I gotta talk to some customer service. They always thank me for my loyalty because I have been a customer for a long time. You know, back when probably when Ronald Reagan was president, it was my first Amex card. And so they're like, oh thank you, Mr Volonte. We we really appreciate your loyalty. So loyalty is a big thing for american Express with its customers. So what about IBM and American Express? How are you breeding, you know, what's that loyalty factor look like for you guys? >>Yeah, I think it's a very important element. I mean to your point, I have the same experience, It's it's a it's a crucial element. Uh the whole, I mean american Express is famous for its loyalty schemes for loyalty as a company. I think loyalty like the business has evolved, I think the loyalty evolves in the same style in I would say in the in the in the old days, I would say the argument was you need to have the best product, you know, you you need to be and then we'll buy the product in the current environment. I would argue that it's way more about skills, Do we have the right people? Do we have the right technology strategy kind of stuff? I would say for the future, it's way more about do we have the right trust, commitment and loyalty of the people that work with us going forward to serve the client needs? And I think that evolution, it's almost like you have an industrial revolution, there was an information resolution. I think there's more of a loyalty revolution coming up where the real differentiating factors is because we can study this and argue this for ages. But a lot of parties will deliver a lot of good technology to the market, they will deliver a lot of good people, they will have good price points. So what's the real differentiating factor? It's like, do we really trust these people? And then I think relationship loyalty will really come and play and it will not become and play just between an IBM and an american express, but I would argue it will come and play in the whole business cycle of american express to their clients. I mean if the credit card swipe of your american express card in a shop fails, It needs to be my problem. If I deliver the service to American express it cannot be that Oh American express has a problem and you know what, it's 8:00 in the evening uh yeah we have reduced services. No we never had that, we will never have that but we need to get even deeper in understanding what the effects are of these business issues. >>Yeah I mean you're right the nature of loyalty, the preservative products have changed. I mean I remember you know I used to travel overseas with american Express traveler's checks that was a staple of every overseas trip that I ever took you no matter where I was going, whether it's asia pacific or or europe, I had to have that packet and I and I had you know, there were times when, when you know one time particularly had a problem film, they were right there to solve the problem. Of course, many young people in the audience don't even know what american express traveler's check is. They probably don't know what cash is carrying around crypto in their wallet. But but but that's an example and that's about trust, trust that product, I trust that company behind the product. Again, that has to extend to your relationship doesn't. >>Absolutely. So the technology that an american Express users, whether they do it themselves or whether it's provided by partners like IBM it needs to be seamless because let's face it, you would not be interested to know who provides you the security on your credit card. If you have an american Express card, you expect expect american Express to deliver your the security that you need and whether american Express delivers that or IBM you couldn't care less and you shouldn't care less. But what it does require is that in the old school I would say it was more like okay we'll give some services and some products to american Express and guys could look now, we need to think ahead and I think that's where the power of IBM comes in where that we really attuned by industry to the industry issues like compliance, security, stability services, um to the inclined to you. So you need to feel if I cannot explain what I do to american express in your terms as an infusion of an express credit card, you can argue what's the real value add. And definitely if there's like 345 parties playing exactly the same game, it needs to be differentiating and I think a company like IBM we have differentiating value but we need to make it very clear and that's I think where you see companies like american Express really work together and that's what loyalty and trust really comes in play. >>Last question when we got to go is you have american expression, your title are other companies jealous >>we >>want that >>to, they should, they should >>be. Uh >>but I I must say, I mean we deal with a ton of financial institutions as you know around the globe including the other credit cards. But yeah, I think where these deep relationship ships commonplace indeed too. I mean they're so old, so deep, so and entrenched and it really start there's different dimensions to it and it's not always that hard coded anymore, it's the subtlety of really relying on each other. I mean when something happens in the middle of the night with american express, all of IBM is on board as of the second and it's not driven by contracts or by anything. It's by people that have an American Express logo on the forehead and work for an IBM. >>Yeah. Right. It's awesome. Pete Pete bills. Great story. Thanks so much for coming to the cube. It's great to have you. >>Thanks for having me. >>All right and keep it right. There is day volonte ongoing coverage of think 2021. You're watching the Cube? Yeah.

>>From around the globe. It's the cube with digital coverage of AWS reinvent 2020 sponsored by Intel, AWS and our community partners. >>Hi everyone. Welcome back to the cubes. Live coverage cube live program for re:Invent 2020. This is our Q virtual. We're not in person like we normally are. Today is the AWS public sector. Worldwide celebration day. A lot of content coming from Teresa Carlson and her team and highlighting everything. Of course, the cube channel on the re:Invent events site. Well, the content we streaming there, if you go to the description, you can click on the link and check out all the on-demand interviews. We've done hundreds of videos live before the event pre recorded as well as here live today for public sector day, I'm showing Lisa Martin co-hosts of the cube. Who's been involved in a lot of those interviews. Uh, Lisa, great to see you before we good to see you. Thanks for coming on. >>Likewise. Good to see you too, John. Glad that you're staying safe. >>Well, a lot of good action. And before we get started, I do want to put a plug out for, um, some Salesforce, big party virtual event. Uh, Salesforce is having a big party at re:Invent 2020 a virtual house party with chance the rapper performing an exclusive set with surprise celebrities and DJ in residence December 10th that's tomorrow at 5:00 PM Pacific, go to salesforce.com/big party to check out chance the rapper. Uh, I'm a big fan. Of course my kids are more fans than, uh, check out the sales report. Okay. Back to cube virtual Lisa. Great to see you. >>Likewise John. So public sector day, a lot of transformation mean re:Invent being reinvented, being virtual 500,000 registered. And so, so much has changed, but a lot also that Teresa Carlson spoke about in her keynote and this morning about the transformation across the public sector, that's really been driven by necessity with COVID. It was really impressive to hear and see all of the good things that AWS is facilitating across healthcare, government, education, state, and local. You name it. >>Yeah. The thing I love about Theresa is she's always been ever since I've known her now she's been on the cube every year, since 2013, since we've been covering re:Invent, she's always had a big, bold vision, and she's always kind of stayed on that track. And this year that was really clear out of the box on her, her leadership session. You got to think big and you got to look at the value of the data. That was the key message from her, her and her group public sector, by the way, has been highly active with the COVID pandemic. A lot of public services have been leveraging Amazon cloud to serve, uh, their, their, their people, whether it's getting them the checks for entitlements or getting them, you know, pharmacy drugs and whatnot, and helping them with the pandemic. But clearly Amazon has stepped up and helped education with, with, uh, remotes. So Theresa's team has been pretty busy. So I think that they had more time to prepare for the virtual keynote. I should've gotten chock full of more announcements. >>Yeah. And also some great examples. As you mentioned, we heard from UK biobank, some of the interviews also that have already happened on the kid that you've done showed some amazing work that AWS has helped to facilitate for school districts in Los Angeles, for example, the government of Rhode Island. And those are some of the great things cabbage, what they were able to enable Kevin's to do, to deliver small business loans of so quickly. A lot of that, I thought, I wish we're hearing more about how technology is facilitating so much. Goodness, in COVID on the news. Of course, we're hearing a lot of the challenges with online learning, but there's a lot of amazing things that AWS has been able to facilitate incredibly quickly. >>You know, one of the concerns I have with Theresa and her team years and years ago was this idea of national parks, right? You know, we have spaces where we can go visit and why isn't there a cyber version of that. And so you S you saw that progression and she'd been doing a lot of deals where they're using the cloud and donating their technology for the betterment of society. And one of the things that was, um, news today was an advancement of their open data registry, which has been kind of this open commons of, you know, health data and whatnot. And now they have all the sequencing data that's searchable, readable, uh, from the national Institute of health for DNA sequencing. So this is going to be, again, more commons, like approach is starting to see that I think this is going to be a real big trend lease. >>I think you're going to start to see the big companies have to really contribute to society in a way that we've never seen before, because they have the large scale. You can donate large compute to say research projects. So you starting to see, uh, from Teresa's team, the bubbling up of these new shared experiences around technology for the betterment of society. I think that sequencing was one, the renewable energy project. Another one, again, they're investing in women owned businesses and underrepresented minorities, and at small, medium size businesses to fund them, we saw a guy launching stuff in space that can create, you know, synthetic satellites. So you can look through clouds. This is new. I mean, this is interesting. >>It is interesting. And it actually, to your point is impactful at every level across the globe, going from when they talked about we farm creating this network of small scale of farmers, connectivity was their biggest problem. And now there's over a million. I'm sure that number it's probably even bigger. I've connected farmers due to AWS. You talked about also it's the cord 19 search, which is the expansion of their open research dataset. COVID open research data set that is only possible because of cloud computing and AWS hundreds of thousands of assets in there. Um, 200 plus open data sets for genomic research. She talked about how that's been at the of some of the things that we've seen go on so quickly with operation work speed, uh, with respect to the vaccine. So a lot of acceleration when we know public sector kind of traditionally not necessarily fast movers, but of course, as we've all said, a number of times recently necessity is the mother of invention and the speed element and the connectivity element were things that really spoke loudly to me with what Teresa said today, about the importance of extracting value from data. >>You know, when I talked to Andy Jassy and he talked about this in his keynote, the digital transformation is on full display. And the necessity being the mother of invention is a great phrase, the system and sticking because you can't hide. I mean, you have to deliver these services in the public sector, or, you know, people's lives are going to be impacted in certainly this there's death involved, right? So you have that and then you've got education. I mean, people want to see that changed quicker. There's always been conscious, Oh, education has got to be re-imagined well, guess what? There's no school open. So we got to re-imagine it now. So you get a lot of pressure, unprecedented demand. She said, Theresa said, three's a crosswind actually set onstage for education change. Um, so that's huge. Right? And then the other thing that she mentioned, I think that's going to be a big focus. >>It's not as, um, you know, headline news oriented is this whole jobs training piece. Um, that's a huge deal because the, the tsunami that hits so fast on this digital transformation, because the COVID, we're going to have a post COVID era of rapid acceleration of new skills. So people gotta get trained. So this ain't going to be the boring training programs, the guy who get kind of get better. So I think you're going to see some innovation Lisa, around how people think about delivering and constructing training programs to be much more real world thinking outside the box, you're going to start to see new things. Otherwise it's just going to be too slowly, the training right now. It's just, you know, sign up for the courseware and get a certification. Yeah, you got to do those things, but how can you get sort of cases done faster? How do you get people with the skills in their hands and virtual hands, if you will, to stand up more cloud, more AI, the pressure's there. So we can, that's going to be a huge thing to watch. >>Okay. The pressure is there. You're right. And a need is there. She talked about a lot of the demand that their customers are driving for some of the services and the education services as well that they're offering. But I'd like to point about upskilling focusing on the people, not just the people, but also the diversity inclusion. And we all know how impactful thought diversity is. So their, their dedication, their in their focus there, and also her recommendation to be bold. And I think in the education, respect was really critical. There is no time like now to move digital transformation. If education systems aren't there, then you know, it's a huge challenge and it impacts every person, every element of every family. So what they're able to do there, by focusing on the people and enabling folks to get trained faster, more resources online can only be a good, you know, Theresa >>Has always, um, has her own flare to style to her. She's incredible business woman and have such respect for her. She's been so successful. Um, but she always sends her presentations with the, kind of the, the kind of her to dues. Um, and you kind of pointed that out. So just review them with you. And I want to get your reaction. Number one, she said, you got to re-imagine and enable a digital, a digitally enabled business. Number two, identify data has an realized value and then increase your diversity. And she pointed to avis.training. Um, and that's kind of her kind of get out there and do those things so digitally enabled business, get that unrealized day to get it into work and increase your diversity. And then she had had a big party every year just said, instead of a party go out and do a random act of kindness act. So, yeah, typical, three's a flare, you know, she kind of ended it with a random act of kindness, but, but her bold vision, those are practical, uh, mandates. What's your reaction to, to that? >>I bold vision. I absolutely 100% I think right now is the time that no business can afford to be hiding under the covers. We have to be, they have to be very thoughtful and very prescriptive, but be bold. There's so much opportunity right now. We're seeing a ton of invention and innovation, John, that we've seen over the last nine months. There's a lot of COVID catalysts that we've been talking about on the cube that are really fantastic. So I think that recommendation to set a bold vision is absolutely imperative, not easy to achieve, but I think right now more than ever, it could really be what sets apart, the winners and losers of tomorrow. >>Yeah. I love it. I just say that on this final note, um, cloud and AI is really in play cloud-scale machine learning, which essentially feeds AI is all about data compute going down to the chip level, AI and software and data is critical for cloud. So really awesome keynote again, leadership session by Teresa Carlson, and there's a whole site of content available. Checkout the cube page, click down on the main page. You'll see that description. You'll see a link to the re:Invent page and check on public sector. A lot of great content. Lisa final question for us to kind of close out this keynote leadership session analysis here on all sector day. I want to get your take on, um, the interviews you've done with the Amazon folks and partners and customers. What are the themes that have been boiling out of those? What have you have been hearing? What's your take and observation of the common pattern? >>You know, given the fact that we haven't all been able to be together at my last cube event in person was reinvent 2019. And we're so used to having, you know, three, four days of wall-to-wall coverage, two sides, being able to have those close personal conversations with our guests this year really did a phenomenal job of recreating that same experience, digitally there's tremendous amount of innovation happening. I think that was the one thing that really jumped out at me, the speed with which it's happening, how so many different types businesses have pivoted, not once, but again and again, and again, as times are changing and how even I yesterday I interviewed Boone, supersonic CEO, some of the things that they're facilitating to get commercial supersonic flight back that fully cloud and AI machine learning can do that. There was no stoppage of innovation this year. In fact, that actually got faster. And I think that was a resounding theme and a lot of positivity from the guests. >>You know, the cue, his business was to go to events and extract the signal from the noise. Guess what? There's no physical events. We have the cube virtual. We have pivoted. We are now in our eighth, ninth month of cube virtual. It's been a new model. We've gotten more interviews, more people can just click into the cube virtual. We have more virtual sets, the Cuban virtualized Lisa. Although I miss them in real life as a whole new ballgame for us, >>It is a whole new ball game. And it also provides a lot of opportunities for businesses to get their messaging out and connect and engage with their audience, which is important. >>Well, I miss real life. I miss everybody out there. I wish we could be there in person. Uh, the world will stay hybrid. I think with virtual, I think this has been a great format. There's been some great benefits, but we want to be in person. I want you on the desk with us. So, and all the folks out there I wish we could see. And then we'll see you next year. Thanks everyone for watching the key. This is our keynote analysis and leadership analysis of the worldwide public sector. Teresa Carlson, Kenya. I'm John from Lisa Martin. Thanks for watching.

>> Narrator: From around the globe, it's theCUBE! With digital coverage of DevOps Virtual Forum, brought to you by Broadcom. >> Hi, Lisa Martin here covering the Broadcom DevOps Virtual Forum. I'm very pleased to be joined today by a CUBE alumni, Jeffrey Hammond, the Vice President and Principal Analyst serving CIOs at Forrester. Jeffrey, nice to talk with you today. >> Good morning, it's good to be here. >> So, a virtual forum, a great opportunity to engage with our audiences. So much has changed in the last, it's an understatement, right? Or it's an overstated thing, but it's obvious. So much has changed. When we think of DevOps, one of the things that we think of is speed, enabling organizations to be able to better serve customers or adapt to changing markets like we're in now. Speaking of the need to adapt, talk to us about what you're seeing with respect to DevOps and Agile in the age of COVID. What are things looking like? >> Yeah, I think that for most organizations, we're in a period of adjustment. When we initially started, it was essentially a sprint. You run as hard as you can for as fast as you can for as long as you can and you just kind of power through it. And that's actually what the folks at GitHub saw in May, when they run an analysis of how developers commit times and level of work that they were committing and how they were working. In the first couple months of COVID, was progressing, they found that developers, at least in the Pacific Time Zone, were actually increasing their work volume, maybe 'cause they didn't have two hour commutes, or maybe because they work stuck away in their homes, but for whatever reason, they were doing more work. And it's almost like, if you've ever run a marathon, the first mile or two in the marathon, you feel great, you just want to run and you want to power through it, you want to go hard. And if you do that, by the time you get to mile 18 or 19, you're going to be gassed, sucking for wind. And that's I think where we're starting to hit. So as we start to gear our development shops up for the reality that most of us won't be returning into an office until 2021 at the earliest. And many organizations will be fundamentally changing their remote work policies, we have to make sure that the agile processes that we use, and the DevOps processes and tools that we use to support these teams are essentially aligned to help developers run that marathon, instead of just kind of power through. So, let me give you a couple specifics. For many organizations, they have been in an environment where they will tolerate remote work and what I would call remote work around the edges, like developers can be remote, but product managers and essentially scrum masters and all the administrators that are running the SCM repositories and the DevOps pipelines are all in the office. And it's essentially centralized work. That's not where we are anymore. We're moving from remote workers at the edge to remote workers at the center of what we do. And so, one of the implications of that is that we have to think about all the activities that you need to do from a DevOps perspective, or from an agile perspective. They have to be remotable. One of the things I found with some of the organizations I talked to early on was, there were things that administrators had to do that required them to go into the office, to reboot the SCM server as an example, or to make sure that the final approvals for production were made. And so, the code could be moved into the production environment. And so, it actually was a little bit difficult because they had to get specific approval from the HR organizations to actually be allowed to go into the office in some states. And so, one of the the results of that is that, while we've traditionally said tools are important, but they're not as important as culture, as structure, as organization, as process, I think we have to rethink that a little bit. Because to the extent that tools enable us to be more digitally organized and to achieve higher levels of digitization in our processes, and be able to support the idea of remote workers in the center. They're now on an equal footing with so many of the other levers that organizations have at their disposal. I'll give you another example. For years, we've said that the key to success with Agile at the team level is cross functional, co-located teams that are working together. Physically co-located. It's the easiest way to show agile success. We can't do that anymore. We can't be physically located at least for the foreseeable future. So, how do you take the low hanging fruits of an agile transformation and apply it in the time of COVID? Well, I think what you have to do is you have to look at what physical co-location has enabled in the past and understand that it's not so much the fact that we're together looking at each other across the table, it's the fact that we're able to get into a shared mind space. From a measurement perspective, we can have shared purpose, we can engage in high bandwidth communications. It's the spiritual aspect of that physical co-location that is actually important. So, one of the biggest things that organizations need to start to ask themselves is, how do we achieve spiritual co-location with our Agile teams, because we don't have the ease of physical co-location available to us anymore. >> Well, spiritual co-location is such an interesting kind of provocative phrase there, but something that probably was a challenge. Here we are seven, eight months in, for many organizations as you say, going from physical workspaces, co-location, being able to collaborate face to face to a light switch flip overnight, and this undefined indeterminate period of time where all we were living with was uncertainty. How does spiritual... When you talk about spiritual co-location in terms of collaboration and processes and technology. Help us unpack that and how are you seeing organizations adopt it? >> Yeah, it's a great question. And I think it goes to the very root of how organizations are trying to transform themselves to be more agile and to embrace DevOps. If you go all the way back to the original Agile Manifesto. There were four principles that were espoused. Individuals and interactions over processes and tools. That's still important, individuals and interactions are at the core of software development. Processes and tools that support those individuals in those interactions are more important than ever. Working software over comprehensive documentation. Working software is still more important. But when you are trying to onboard employees, and they can't come into the office, and they can't do the two day training session, and kind of understand how things work, and they can't just holler over theCUBE, to ask a question, you may need to invest a little bit more in documentation to help that onboarding process be successful in a remote context. Customer collaboration over contract negotiation. Absolutely still important. But employee collaboration is equally as important if you want to be spiritually co-located and if you want to have a shared purpose. And then, responding to change over following a plan. I think one of the things that's happened in a lot of organizations is we have focused so much of our DevOps effort around velocity. Getting faster, we need to run as fast as we can. Like that sprinter, okay? Trying to just power through it as quickly as possible. But as we shift to the marathon way of thinking, velocity is still important but agility becomes even more important. So when you have to create an application in three weeks to do track and trace for your employees, agility is more important than just flat out velocity. And so, changing some of the ways that we think about DevOps practices is important to make sure that that agility is there. For one thing, you have to defer decisions as far down the chain to the team level as possible. So those teams have to be empowered to make decisions. Because you can't have a program level meeting of six or seven teams in one large hall and say, here's the lay of the land, here's what we're going to do, here are our processes, and here are our guardrails. Those teams have to make decisions much more quickly. The developers are actually developing code in smaller chunks of flow. They have to be able to take two hours here, or 50 minutes there and do something useful. And so, the tools that support us have to become tolerant of the reality of how we're working. So, if they work in a way that it allows the team together to take as much autonomy as they can handle, to allow them to communicate in a way that delivers shared purpose, and allows them to adapt and master new technologies, then they're in the zone, they'll get spiritually connected. I hope that makes sense (chuckles). >> It does, I think we all could use some of that. But you talked about in the beginning and I've talked to numerous companies during the pandemic on theCUBE about the productivity or rather the number of hours worked has gone way up for many roles, and times that they normally at late at night on the weekends. So, but it's a cultural, it's a mind shift. To your point about DevOps focused on velocity, sprint, sprint, sprint, and now we have to. So that cultural shift is not an easy one for developers and even the biz folks to flip so quickly. What have you seen in terms of the velocity at which businesses are able to get more of that balance between the velocity, the sprint and the agility? >> I think at the core, this really comes down to management sensitivity. When everybody was in the office, you could kind of see the mental health of development teams by watching how they work, you can call it management by walking around, right? We can't do that, managers have to be more aware of what their teams are doing, because they're not going to see that developer doing a check in at 9:00 p.m. on a Friday, because that's what they had to do to meet the objectives. And they're going to have to find new ways to measure engagement and also potential burnout. A friend of mine once had a great metric that he called the Parking Lot Metric. It was helpful as the parking lot at nine and helpful was it at five. And that gives you an indication of how engaged your developers are. What's the digital equivalent of the Parking Lot Metric in the time of COVID, it's commit stats, it's commit rates, it's the turn rate that we have in our code. So we have this information, we may not be collecting it, but then the next question becomes how do we use that information? Do we use that information to say, well, this team isn't delivering at the same level of productivity as another team? Do we weaponize that data? Or do we use that data to identify impedances in the process? Why isn't a team working effectively? Is it because they have higher levels of family obligations, and they've got kids that are at home? Is it because they're working with hardware technology, and guess what, it's not easy to get the hardware technology into their home office, because it's in the lab, at the corporate office. Or they're trying to communicate halfway around the world. And they're communicating with an office lab that is also shut down. And the bandwidth just doesn't enable the level of high bandwidth communications. So, from a DevOps perspective, managers have to get much more sensitive to the exhaust that the DevOps tools are throwing off, but also how they're going to use that in a constructive way to prevent burnout. And then they also need to, if they're not already managing, or monitoring or measuring the level of developer engagement they have, they really need to start. Whether that's surveys around developer satisfaction, whether it's more regular social events where developers can kind of just get together and drink a beer and talk about what's going on in the project and monitoring who checks in and who doesn't. They have to work harder, I think than they ever have before. >> Well, and you mentioned burnout. And that's something that I think we've all faced in this time at varying levels, and it changes and it's a real, there's a tension in the air regardless of where you are. There's a challenge, as you mentioned, people having their kids as co-workers and fighting for bandwidth, because everyone is forced in this situation. I'd love to get your perspective on some businesses that have done this, well, this adaptation. What can you share in terms of some real world examples that might inspire the audience? >> Yeah, I'll start with Stack Overflow. They recently published a piece in the Journal of the ACM around some of the things that they had discovered. First of all, just a cultural philosophy. If one person is remote, everybody is remote. And you just think that way from the executive level. Social spaces, one of the things that they talk about doing is leaving the video conference room open at the team level all day long. And the team members will go on mute, so that they don't have to, that they don't necessarily have to be there with somebody else listening to them. But if they have a question, they can just pop off mute really quickly and ask the question and if anybody else knows the answer, it's kind of like being in that virtual pod, if you will. Even here at Forrester, one of the things that we've done is we've invested in social ceremonies. We've actually moved our team meetings on my analyst team from once every two weeks to weekly. And we have built more time in for socialization, just so we can see how we're doing. I think Microsoft has really made some good information available in how they've managed things like the onboarding process. I think Amanda Silver over there mentioned that a couple of weeks ago, a presentation they did that Microsoft's onboarded over 150,000 people since the start of COVID. If you don't have good remote onboarding processes, that's going to be a disaster. Now, they're not all developers, but if you think about it, everything from how you do the interviewing process, to how you get people their badges, to how they get their equipment. Security is another issue that they called out. Typically, IT security, security of developers machines, ends at the corporate desktop. But now since we're increasingly using our own machines, our own hardware, security organization's going to have to extend their security policies to cover employee devices. And that's caused them to scramble a little bit. So, the examples are out there. It's not a lot of like, we have to do everything completely differently. But it's a lot of subtle changes that have to be made. I'll give you another example. One of the things that we are seeing is that more and more organizations to deal with the challenges around agility with respect to delivering software and embracing low code tools. In fact, we see about 50% of firms are using low code tools right now, we predict it's going to be 75% by the end of next year. So, figuring out how your DevOps processes support an organization that might be using Mendix or OutSystems, or the Power Platform, building the front end of an application, like a track and trace application really, really quickly. But then hooking it up to your back end infrastructure. Does that happen completely outside the DevOps investments that you're making? And the agile processes that you're making? Or do you adapt your organization. Are hybrid teams now, teams that not just have professional developers, but also have business users that are doing some development with a low code tool. Those are the kinds of things that we have to be willing to entertain in order to shift the focus a little bit more toward the agility side, I think. >> A lot of obstacles but also a lot of opportunities for businesses to really learn, pay attention here, pivot and grow and hopefully some good opportunities for the developers and the business folks to just get better at what they're doing and learning to embrace spiritual co-location. Jeffrey, thank you so much for joining us on the program today, very insightful conversation. >> It's my pleasure, it's an important thing. Just remember, if you're going to run that marathon, break it into 26, 10 minute runs, take a walk break in between each, and you'll find that you'll get there. >> Digestible components, wise advice. Jeffrey Hammond, thank you so much for joining. For Jeffrey, I'm Lisa Martin. You're watching Broadcom's DevOps Virtual Forum. (bright upbeat music)

>>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi, Lisa Martin here covering the Broadcom dev ops virtual forum. I'm very pleased to be joined today by a cube alumni, Jeffrey Hammond, the vice president and principal analyst serving CIO is at Forester. Jeffrey. Nice to talk with you today. >>Good morning. It's good to be here. Yeah. >>So a virtual forum, great opportunity to engage with our audiences so much has changed in the last it's an understatement, right? Or it's an overstated thing, but it's an obvious, so much has changed when we think of dev ops. One of the things that we think of is speed, you know, enabling organizations to be able to better serve customers or adapt to changing markets like we're in now, speaking of the need to adapt, talk to us about what you're seeing with respect to dev ops and agile in the age of COVID, what are things looking like? >>Yeah, I think that, um, for most organizations, we're in a, uh, a period of adjustment, uh, when we initially started, it was essentially a sprint, you know, you run as hard as you can for as fast as you can for as long as you can and you just kind of power through it. And, and that's actually what, um, the folks that get hub saw in may when they ran an analysis of how developers, uh, commit times and a level of work that they were committing and how they were working, uh, in the first couple of months of COVID was, was progressing. They found that developers, at least in the Pacific time zone were actually increasing their work volume, maybe because they didn't have two hour commutes or maybe because they were stuck away in their homes, but for whatever reason, they were doing more work. >>And it's almost like, you know, if you've ever run a marathon the first mile or two in the marathon, you feel great and you just want to run and you want to power through it and you want to go hard. And if you do that by the time you get to mile 18 or 19, you're going to be gassed. It's sucking for wind. Uh, and, and that's, I think where we're starting to hit. So as we start to, um, gear our development chops out for the reality that most of us won't be returning into an office until 2021 at the earliest and many organizations will, will be fundamentally changing, uh, their remote workforce, uh, policies. We have to make sure that the agile processes that we use and the dev ops processes and tools that we use to support these teams are essentially aligned to help developers run that marathon instead of just kind of power through. >>So, um, let me give you a couple of specifics for many organizations, they have been in an environment where they will, um, tolerate Rover remote work and what I would call remote work around the edges like developers can be remote, but product managers and, um, you know, essentially scrum masters and all the administrators that are running the, uh, uh, the SCM repositories and, and the dev ops pipelines are all in the office. And it's essentially centralized work. That's not, we are anymore. We're moving from remote workers at the edge to remote workers at the center of what we do. And so one of the implications of that is that, um, we have to think about all the activities that you need to do from a dev ops perspective or from an agile perspective, they have to be remote people. One of the things I found with some of the organizations I talked to early on was there were things that administrators had to do that required them to go into the office to reboot the SCM server as an example, or to make sure that the final approvals for production, uh, were made. >>And so the code could be moved into the production environment. And so it actually was a little bit difficult because they had to get specific approval from the HR organizations to actually be allowed to go into the office in some States. And so one of the, the results of that is that while we've traditionally said, you know, tools are important, but they're not as important as culture as structure as organization as process. I think we have to rethink that a little bit because to the extent that tools enable us to be more digitally organized and to hiring, you know, achieve higher levels of digitization in our processes and be able to support the idea of remote workers in the center. They're now on an equal footing with so many of the other levers, uh, that, that, um, uh, that organizations have at their disposal. Um, I'll give you another example for years. >>We've said that the key to success with agile at the team level is cross-functional co located teams that are working together physically co located. It's the easiest way to show agile success. We can't do that anymore. We can't be physically located at least for the foreseeable future. So, you know, how do you take the low hanging fruits of an agile transformation and apply it in, in, in, in the time of COVID? Well, I think what you have to do is that you have to look at what physical co-location has enabled in the past and understand that it's not so much the fact that we're together looking at each other across the table. It's the fact that we're able to get into a shared mindspace, uh, from, um, uh, from a measurement perspective, we can have shared purpose. We can engage in high bandwidth communications. It's the spiritual aspect of that physical co-location that is actually important. So one of the biggest things that organizations need to start to ask themselves is how do we achieve spiritual colocation with our agile teams? Because we don't have the, the ease of physical co-location available to us anymore? >>Well, the spiritual co-location is such an interesting kind of provocative phrase there, but something that probably was a challenge here, we are seven, eight months in for many organizations, as you say, going from, you know, physical workspaces, co-location being able to collaborate face to face to a, a light switch flip overnight. And this undefined period of time where all we were living with with was uncertainty, how does spiritual, what do you, when you talk about spiritual co-location in terms of collaboration and processes and technology help us unpack that, and how are you seeing organizations adopted? >>Yeah, it's, it's, um, it's a great question. And, and I think it goes to the very root of how organizations are trying to transform themselves to be more agile and to embrace dev ops. Um, if you go all the way back to the, to the original, uh, agile manifesto, you know, there were four principles that were espoused individuals and interactions over processes and tools. That's still important. Individuals and interactions are at the core of software development, processes and tools that support those individual and interact. Uh, those individuals in those interactions are more important than ever working software over comprehensive documentation. Working software is still more important, but when you are trying to onboard employees and they can't come into the office and they can't do the two day training session and kind of understand how things work and they can't just holler over the cube, uh, to ask a question, you may need to invest a little bit more in documentation to help that onboarding process be successful in a remote context, uh, customer collaboration over contract negotiation. >>Absolutely still important, but employee collaboration is equally as important if you want to be spiritually, spiritually co-located. And if you want to have a shared purpose and then, um, responding to change over following a plan. I think one of the things that's happened in a lot of organizations is we have focused so much of our dev ops effort around velocity getting faster. We need to run as fast as we can like that sprinter. Okay. You know, trying to just power through it as quickly as possible. But as we shift to, to the, to the marathon way of thinking, um, velocity is still important, but agility becomes even more important. So when you have to create an application in three weeks to do track and trace for your employees, agility is more important. Um, and then just flat out velocity. Um, and so changing some of the ways that we think about dev ops practices, um, is, is important to make sure that that agility is there for one thing, you have to defer decisions as far down the chain to the team level as possible. >>So those teams have to be empowered to make decisions because you can't have a program level meeting of six or seven teams and one large hall and say, here's the lay of the land. Here's what we're going to do here are our processes. And here are our guardrails. Those teams have to make decisions much more quickly that developers are actually developing code in smaller chunks of flow. They have to be able to take two hours here or 50 minutes there and do something useful. And so the tools that support us have to become tolerant of the reality of, of, of, of how we're working. So if they work in a way that it allows the team together to take as much autonomy as they can handle, um, to, uh, allow them to communicate in a way that, that, that delivers shared purpose and allows them to adapt and master new technologies, then they're in the zone in their spiritual, they'll get spiritually connected. I hope that makes sense. >>It does. I think we all could use some of that, but, you know, you talked about in the beginning and I've, I've talked to numerous companies during the pandemic on the cube about the productivity, or rather the number of hours of work has gone way up for many roles, you know, and, and, and times that they normally late at night on the weekends. So, but it's a cultural, it's a mind shift to your point about dev ops focused on velocity, sprints, sprints, sprints, and now we have to, so that cultural shift is not an easy one for developers. And even at this folks to flip so quickly, what have you seen in terms of the velocity at which businesses are able to get more of that balance between the velocity, the sprint and the agility? >>I think, I think at the core, this really comes down to management sensitivity. Um, when everybody was in the office, you could kind of see the mental health of development teams by, by watching how they work. You know, you call it management by walking around, right. We can't do that. Managers have to, um, to, to be more aware of what their teams are doing, because they're not going to see that, that developer doing a check-in at 9:00 PM on a Friday, uh, because that's what they had to do, uh, to meet the objectives. And, um, and, and they're going to have to, to, um, to find new ways to measure engagement and also potential burnout. Um, friend of mine once had, uh, had a great metric that he called the parking lot metric. It was helpful as the parking lot at nine. And how full was it at five? >>And that gives you an indication of how engaged your developers are. Um, what's the digital equivalent equivalent to the parking lot metric in the time of COVID it's commit stats, it's commit rates. It's, um, you know, the, uh, the turn rate, uh, that we have in our code. So we have this information, we may not be collecting it, but then the next question becomes, how do we use that information? Do we use that information to say, well, this team isn't delivering as at the same level of productivity as another team, do we weaponize that data or do we use that data to identify impedances in the process? Um, why isn't a team working effectively? Is it because they have higher levels of family obligations and they've got kids that, that are at home? Um, is it because they're working with, um, you know, hardware technology, and guess what, they, it's not easy to get the hardware technology into their home office because it's in the lab at the, uh, at the corporate office, uh, or they're trying to communicate, uh, you know, halfway around the world. >>And, uh, they're communicating with a, with an office lab that is also shut down and, and, and the bandwidth just doesn't enable the, the level of high bandwidth communications. So from a dev ops perspective, managers have to get much more sensitive to the, the exhaust that the dev ops tools are throwing off, but also how they're going to use that in a constructive way to, to prevent burnout. And then they also need to, if they're not already managing or monitoring or measuring the level of developer engagement, they have, they really need to start whether that's surveys around developer satisfaction, um, whether it's, you know, more regular social events, uh, where developers can kind of just get together and drink a beer and talk about what's going on in the project, uh, and monitoring who checks in and who doesn't, uh, they have to, to, um, work harder, I think, than they ever have before. >>Well, and you mentioned burnout, and that's something that I think we've all faced in this time at varying levels and it changes. And it's a real, there's a tension in the air, regardless of where you are. There's a challenge, as you mentioned, people having, you know, coworker, their kids as coworkers and fighting for bandwidth, because everyone is forced in this situation. I'd love to get your perspective on some businesses that are, that have done this well, this adaptation, what can you share in terms of some real-world examples that might inspire the audience? >>Yeah. Uh, I'll start with, uh, stack overflow. Uh, they recently published a piece in the journal of the ACM around some of the things that they had discovered. Um, you know, first of all, just a cultural philosophy. If one person is remote, everybody is remote. And you just think that way from an executive level, um, social spaces. One of the things that they talk about doing is leaving a video conference room open at a team level all day long, and the team members, you know, we'll go on mute, you know, so that they don't have to, that they don't necessarily have to be there with somebody else listening to them. But if they have a question, they can just pop off mute really quickly and ask the question. And if anybody else knows the answer, it's kind of like being in that virtual pod. Uh, if you, uh, if you will, um, even here at Forrester, one of the things that we've done is we've invested in social ceremonies. >>We've actually moved our to our team meetings on, on my analyst team from, from once every two weeks to weekly. And we have built more time in for social Ajay socialization, just so we can see, uh, how, how, how we're doing. Um, I think Microsoft has really made some good, uh, information available in how they've managed things like the onboarding process. I think I'm Amanda silver over there mentioned that a couple of weeks ago when, uh, uh, a presentation they did that, uh, uh, Microsoft onboarded over 150,000 people since the start of COVID, if you don't have good remote onboarding processes, that's going to be a disaster. Now they're not all developers, but if you think about it, um, everything from how you do the interviewing process, uh, to how you get people, their badges, to how they get their equipment. Um, security is a, is another issue that they called out typically, uh, it security, um, the security of, of developers machines ends at, at, at the corporate desktop. >>But, you know, since we're increasingly using our own machines, our own hardware, um, security organizations kind of have to extend their security policies to cover, uh, employee devices, and that's caused them to scramble a little bit. Uh, so, so the examples are out there. It's not a lot of, like, we have to do everything completely differently, but it's a lot of subtle changes that, that have to be made. Um, I'll give you another example. Um, one of the things that, that we are seeing is that, um, more and more organizations to deal with the challenges around agility, with respect to delivering software, embracing low-code tools. In fact, uh, we see about 50% of firms are using low-code tools right now. We predict it's going to be 75% by the end of next year. So figuring out how your dev ops processes support an organization that might be using Mendix or OutSystems, or, you know, the power platform building the front end of an application, like a track and trace application really, really quickly, but then hooking it up to your backend infrastructure. Does that happen completely outside the dev ops investments that you're making and the agile processes that you're making, or do you adapt your organization? Um, our hybrid teams now teams that not just have professional developers, but also have business users that are doing some development with a low-code tool. Those are the kinds of things that we have to be, um, willing to, um, to entertain in order to shift the focus a little bit more toward the agility side, I think >>Lot of obstacles, but also a lot of opportunities for businesses to really learn, pay attention here, pivot and grow, and hopefully some good opportunities for the developers and the business folks to just get better at what they're doing and learning to embrace spiritual co-location Jeffrey, thank you so much for joining us on the program today. Very insightful conversation. >>My pleasure. It's it's, it's an important thing. Just remember if you're going to run that marathon, break it into 26, 10 minute runs, take a walk break in between each and you'll find that you'll get there. >>Digestible components, wise advice. Jeffery Hammond. Thank you so much for joining for Jeffrey I'm Lisa Martin, you're watching Broadcom's dev ops virtual forum >>From around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom, >>Continuing our conversations here at Broadcom's dev ops virtual forum. Lisa Martin here, please. To welcome back to the program, Serge Lucio, the general manager of the enterprise software division at Broadcom. Hey, Serge. Welcome. Thank you. Good to be here. So I know you were just, uh, participating with the biz ops manifesto that just happened recently. I just had the chance to talk with Jeffrey Hammond and he unlocked this really interesting concept, but I wanted to get your thoughts on spiritual co-location as really a necessity for biz ops to succeed in this unusual time in which we're living. What are your thoughts on spiritual colocation in terms of cultural change versus adoption of technologies? >>Yeah, it's a, it's, it's quite interesting, right? When we, when we think about the major impediments for, uh, for dev ops implementation, it's all about culture, right? And swore over the last 20 years, we've been talking about silos. We'd be talking about the paradox for these teams to when it went to align in many ways, it's not so much about these teams aligning, but about being in the same car in the same books, right? It's really about fusing those teams around kind of the common purpose, a common objective. So to me, the, this, this is really about kind of changing this culture where people start to look at a kind of OKR is instead of the key objective, um, that, that drives the entire team. Now, what it means in practice is really that's, uh, we need to change a lot of behaviors, right? It's not about the Yarki, it's not about roles. It's about, you know, who can do what and when, and, uh, you know, driving a bias towards action. It also means that we need, I mean, especially in this school times, it becomes very difficult, right? To drive kind of a kind of collaboration between these teams. And so I think there there's a significant role that especially tools can play in terms of providing this complex feedback from teams to, uh, to be in that preface spiritual qualification. >>Well, and it talked about culture being, it's something that, you know, we're so used to talking about dev ops with respect to velocity, all about speed here. But of course this time everything changed so quickly, but going from the physical spaces to everybody being remote really does take it. It's very different than you can't replicate it digitally, but there are collaboration tools that can kind of really be essential to help that cultural shift. Right? >>Yeah. So 2020, we, we touch to talk about collaboration in a very mundane way. Like, of course we can use zoom. We can all get into, into the same room. But the point when I think when Jeff says spiritual, co-location, it's really about, we all share the same objective. Do we, do we have a niece who, for instance, our pipeline, right? When you talk about dev ops, probably we all started thinking about this continuous delivery pipeline that basically drives the automation, the orchestration across the team, but just thinking about a pipeline, right, at the end of the day, it's all about what is the meantime to beat back to these teams. If I'm a developer and a commit code, I don't, does it take where, you know, that code to be processed through pipeline pushy? Can I get feedback if I am a finance person who is funding a product or a project, what is my meantime to beat back? >>And so a lot of, kind of a, when we think about the pipeline, I think what's been really inspiring to me in the last year or so is that there is much more of an adoption of the Dora metrics. There is way more of a focus around value stream management. And to me, this is really when we talk about collaboration, it's really a balance. How do you provide the feedback to the different stakeholders across the life cycle in a very timely matter? And that's what we would need to get to in terms of kind of this, this notion of collaboration. It's not so much about people being in the same physical space. It's about, you know, when I checked in code, you know, to do I guess the system to automatically identify what I'm going to break. If I'm about to release some allegation, how can the system help me reduce my change pillar rates? Because it's, it's able to predict that some issue was introduced in the outpatient or work product. Um, so I think there's, there's a great role of technology and AI candidate Lynch to, to actually provide that new level of collaboration. >>So we'll get to AI in a second, but I'm curious, what are some of the, of the metrics you think that really matter right now is organizations are still in some form of transformation to this new almost 100% remote workforce. >>So I'll just say first, I'm not a big fan of metrics. Um, and the reason being that, you know, you can look at a change killer rate, right, or a lead time or cycle time. And those are, those are interesting metrics, right? The trend on metric is absolutely critical, but what's more important is you get to the root cause what is taught to you lean to that metric to degrade or improve or time. And so I'm much more interested and we, you know, fruit for Broadcom. Are we more interested in understanding what are the patterns that contribute to this? So I'll give you a very mundane example. You know, we know that cycle time is heavily influenced by, um, organizational boundaries. So, you know, we talk a lot about silos, but, uh, we we've worked with many of our customers doing value stream mapping. And oftentimes what you see is that really the boundaries of your organization creates a lot of idle time, right? So to me, it's less about the metrics. I think the door metrics are a pretty, you know, valid set metrics, but what's way more important is to understand what are the antiperspirants, what are the things that we can detect through the data that actually are affecting those metrics. And, uh, I mean, over the last 10, 20 years, we've learned a lot about kind of what are, what are the antiperspirants within our large enterprise customers. And there are plenty of them. >>What are some of the things that you're seeing now with respect to patterns that have developed over the last seven to eight months? >>So I think the two areas which clearly are evolving very quickly are on kind of the front end of the life cycle, where DevOps is more and more embracing value stream management value stream mapping. Um, and I think what's interesting is that in many ways the product is becoming the new silo. Uh, the notion of a product is very difficult by itself to actually define people are starting to recognize that a value stream is not its own little kind of Island. That in reality, when I define a product, this product, oftentimes as dependencies on our products and that in fact, you're looking at kind of a network of value streams, if you will. So, so even on that, and there is clearly kind of a new sets, if you will, of anti-patterns where products are being defined as a set of OTRs, they have interdependencies and you have have a new set of silos on the operands, uh, the Abra key movement to Israel and the SRE space where, um, I think there is a cultural clash while the dev ops side is very much embracing this notion of OTRs and value stream mapping and Belgium management. >>On the other end, you have the it operations teams. We still think business services, right? For them, they think about configure items, think about infrastructure. And so, you know, it's not uncommon to see, you know, teams where, you know, the operations team is still thinking about hundreds of thousands, tens of thousands of business services. And so the, the, there is there's this boundary where, um, I think, well, SRE is being put in place. And there's lots of thinking about what kind of metrics can be fined. I think, you know, going back to culture, I think there's a lot of cultural evolution that's still required for true operations team. >>And that's a hard thing. Cultural transformation in any industry pandemic or not is a challenging thing. You talked about, uh, AI and automation of minutes ago. How do you think those technologies can be leveraged by DevOps leaders to influence their successes and their ability to collaborate, maybe see eye to eye with the SRS? >>Yeah. Um, so th you're kind of too. So even for myself, as a leader of a, you know, 1500 people organization, there's a number of things I don't see right. On a daily basis. And, um, I think the, the, the, the technologies that we have at our disposal today from the AI are able to mind a lot of data and expose a lot of, uh, issues that's as leaders we may not be aware of. And some of the, some of these are pretty kind of easy to understand, right? We all think we're agile. And yet when you, when you start to understand, for instance, uh, what is the, what is the working progress right to during the sprint? Um, when you start to analyze the data you can detect, for instance, that maybe the teams are over committed, that there is too much work in progress. >>You can start to identify kind of, interdepencies either from a technology, from a people point of view, which were hidden, uh, you can start to understand maybe the change filler rates he's he is dragging. So I believe that there is a, there's a fundamental role to be played by the tools to, to expose again, these anti parents, to, to make these things visible to the teams, to be able to even compare teams. Right. One of the things that's, that's, uh, that's amazing is now we have access to tons of data, not just from a given customer, but across a large number of customers. And so we start to compare all of these teams kind of operate, and what's working, what's not working >>Thoughts on AI and automation as, as a facilitator of spiritual co-location. >>Yeah, absolutely. Absolutely. It's um, you know, th there's, uh, the problem we all face is the unknown, right? The, the law city, but volume variety of the data, uh, everyday we don't really necessarily completely appreciate what is the impact of our actions, right? And so, um, AI can really act as a safety net that enables us to, to understand what is the impact of our actions. Um, and so, yeah, in many ways, the ability to be informed in a timely matter to be able to interact with people on the basis of data, um, and collaborate on the data. And the actual matter, I think is, is a, is a very powerful enabler, uh, on, in that respect. I mean, I, I've seen, um, I've seen countless of times that, uh, for instance, at the SRE boundary, um, to basically show that we'll turn the quality attributes, so an incoming release, right. And exposing that to, uh, an operations person and a sorry person, and enabling that collaboration dialogue through data is a very, very powerful tool. >>Do you have any recommendations for how teams can use, you know, the SRE folks, the dev ops says can use AI and automation in the right ways to be successful rather than some ways that aren't going to be nonproductive. >>Yeah. So to me, the th there, there's a part of the question really is when, when we talk about data, there are there different ways you can use data, right? Um, so you can, you can do a lot of an analytics, predictive analytics. So I think there is a, there's a tendency, uh, to look at, let's say a, um, a specific KPI, like a, an availability KPI, or change filler rate, and to basically do a regression analysis and projecting all these things, going to happen in the future. To me, that that's, that's a, that's a bad approach. The reason why I fundamentally think it's a better approach is because we are systems. The way we develop software is, is a, is a non-leader kind of system, right? Software development is not linear nature. And so I think there's a D this is probably the worst approach is to actually focus on metrics on the other end. >>Um, if you, if you start to actually understand at a more granular level, what har, uh, which are the things which are contributing to this, right? So if you start to understand, for instance, that whenever maybe, you know, you affect a specific part of the application that translates into production issues. So we, we have, I've actually, uh, a customer who, uh, identified that, uh, over 50% of their unplanned outages were related to specific components in your architecture. And whenever these components were changed, this resulted in these plant outages. So if you start to be able to basically establish causality, right, cause an effect between kind of data across the last cycle. I think, I think this is the right way to, uh, to, to use AI. And so pharma to be, I think it's way more God could have a classification problem. What are the classes of problems that do exist and affect things as opposed to analytics, predictive, which I don't think is as powerful. >>So I mentioned in the beginning of our conversation, that just came off the biz ops manifesto. You're one of the authors of that. I want to get your thoughts on dev ops and biz ops overlapping, complimenting each other, what, from a, the biz ops perspective, what does it mean to the future of dev ops? >>Yeah, so, so it's interesting, right? If you think about DevOps, um, there's no felony document, right? Can we, we can refer to the Phoenix project. I mean, there are a set of documents which have been written, but in many ways, there's no clear definition of what dev ops is. Uh, if you go to the dev ops Institute today, you'll see that they are specific, um, trainings for instance, on value management on SRE. And so in many ways, the problem we have as an industry is that, um, there are set practices between agile dev ops, SRE Valley should management. I told, right. And we all basically talk about the same things, right. We all talk about essentially, um, accelerating in the meantime fee to feedback, but yet we don't have the common framework to talk about that. The other key thing is that we add to wait, uh, for, uh, for jeans, Jean Kim's Lascaux, um, to, uh, to really start to get into the business aspect, right? >>And for value stream mapping to start to emerge for us to start as an industry, right. It, to start to think about what is our connection with the business aspect, what's our purpose, right? And ultimately it's all about driving these business outcomes. And so to me, these ops is really about kind of, uh, putting a lens on this critical element that it's not business and it, that we in fact need to fuse business 19 that I need needs to transform itself to recognize that it's, it's this value generator, right. It's not a cost center. And so the relationship to me, it's more than BizOps provides kind of this Oliver or kind of framework, if you will. That set the context for what is the reason, uh, for it to exist. What's part of the core values and principles that it needs to embrace to, again, change from a cost center to a value center. And then we need to start to use this as a way to start to unify some of the, again, the core practices, whether it's agile, DevOps value, stream mapping SRE. Um, so, so I think over time, my hope is that we start to optimize a lot of our practices, language, um, and, uh, and cultural elements. >>Last question surgeon, the last few seconds we have here talking about this, the relation between biz ops and dev ops, um, what do you think as DevOps evolves? And as you talked to circle some of your insights, what should our audience keep their eyes on in the next six to 12 months? >>So to me, the key, the key, um, challenge for, for the industry is really around. So we were seeing a very rapid shift towards kind of, uh, product to product, right. Which we don't want to do is to recreate kind of these new silos, these hard silos. Um, so that, that's one of the big changes, uh, that I think we need to be, uh, to be really careful about, um, because it is ultimately, it is about culture. It's not about, uh, it's not about, um, kind of how we segment the work, right. And, uh, any true culture that we can overcome kind of silos. So back to, I guess, with Jeffrey's concept of, um, kind of the spiritual co-location, I think it's, it's really about that too. It's really about kind of, uh, uh, focusing on the business outcomes on kind of aligning on driving engagement across the teams, but, but not for create a, kind of a new set of silos, which instead of being vertical are going to be these horizontal products >>Crazy by surge that looking at culture as kind of a way of really, uh, uh, addressing and helping to, uh, re re reduce, replace challenges. We thank you so much for sharing your insights and your time at today's DevOps virtual forum. >>Thank you. Thanks for your time. >>I'll be right back >>From around the globe it's the cube with digital coverage of devops virtual forum brought to you by Broadcom. >>Welcome to Broadcom's DevOps virtual forum, I'm Lisa Martin, and I'm joined by another Martin, very socially distanced from me all the way coming from Birmingham, England is Glynn Martin, the head of QA transformation at BT. Glynn, it's great to have you on the program. Thank you, Lisa. I'm looking forward to it. As we said before, we went live to Martins for the person one in one segment. So this is going to be an interesting segment guys, what we're going to do is Glynn's going to give us a really kind of deep inside out view of devops from an evolution perspective. So Glynn, let's start. Transformation is at the heart of what you do. It's obviously been a very transformative year. How have the events of this year affected the >> transformation that you are still responsible for driving? Yeah. Thank you, Lisa. I mean, yeah, it has been a difficult year. >>Um, and although working for BT, which is a global telecommunications company, um, I'm relatively resilient, I suppose, as a, an industry, um, through COVID obviously still has been affected and has got its challenges. And if anything, it's actually caused us to accelerate our transformation journey. Um, you know, we had to do some great things during this time around, um, you know, in the UK for our emergency and, um, health workers give them unlimited data and for vulnerable people to support them. And that's spent that we've had to deliver changes quickly. Um, but what we want to be able to do is deliver those kinds of changes quickly, but sustainably for everything that we do, not just because there's an emergency. Um, so we were already on the kind of journey to agile, but ever more important now that we are, we are able to do those, that kind of work, do it more quickly. >>Um, and that it works because the, the implications of it not working is, can be terrible in terms of you know, we've been supporting testing centers,  new hospitals to treat COVID patients. So we need to get it right. And then therefore the coverage of what we do, the quality of what we do and how quickly we do it really has taken on a new scale and what was already a very competitive market within the telco industry within the UK. Um, you know, what I would say is that, you know, we are under pressure to deliver more value, but we have small cost challenges. We have to obviously, um, deal with the fact that, you know, COVID 19 has hit most industries kind of revenues and profits. So we've got this kind of paradox between having less costs, but having to deliver more value quicker and  to higher quality. So yeah, certainly the finances is, um, on our minds and that's why we need flexible models, cost models that allow us to kind of do growth, but we get that growth by showing that we're delivering value. Um, especially in these times when there are financial challenges on companies. So one of the things that I want to ask you about, I'm again, looking at DevOps from the inside >>Out and the evolution that you've seen, you talked about the speed of things really accelerating in this last nine months or so. When we think dev ops, we think speed. But one of the things I'd love to get your perspective on is we've talked about in a number of the segments that we've done for this event is cultural change. What are some of the things that you've seen there as, as needing to get, as you said, get things right, but done so quickly to support essential businesses, essential workers. How have you seen that cultural shift? >>Yeah, I think, you know, before test teams for themselves at this part of the software delivery cycle, um, and actually now really our customers are expecting that quality and to deliver for our customers what they want, quality has to be ingrained throughout the life cycle. Obviously, you know, there's lots of buzzwords like shift left. Um, how do we do shift left testing? Um, but for me, that's really instilling quality and given capabilities shared capabilities throughout the life cycle that drive automation, drive improvements. I always say that, you know, you're only as good as your lowest common denominator. And one thing that we were finding on our dev ops journey was that we  would be trying to do certain things quick, we had automated build, automated tests. But if we were taking a weeks to create test scripts, or we were taking weeks to manually craft data, and even then when we had taken so long to do it, that the coverage was quite poor and that led to lots of defects later on in the life cycle, or even in our production environment, we just couldn't afford to do that. >>And actually, focusing on continuous testing over the last nine to 12 months has really given us the ability to deliver quickly across the whole life cycle. And therefore actually go from doing a kind of semi agile kind of thing, where we did the user stories, we did a few of the kind of agile ceremonies, but we weren't really deploying any quicker into production because our stakeholders were scared that we didn't have the same control that we had when we had more waterfall releases. And, you know, when we didn't think of ourselves. So we've done a lot of work on every aspect, um, especially from a testing point of view, every aspect of every activity, rather than just looking at automated tests, you know, whether it is actually creating the test in the first place, whether it's doing security testing earlier in the lot and performance testing in the life cycle, et cetera. So, yeah,  it's been a real key thing that for CT, for us to drive DevOps, >>Talk to me a little bit about your team. What are some of the shifts in terms of expectations that you're experiencing and how your team interacts with the internal folks from pipeline through life cycle? >>Yeah, we've done a lot of work on this. Um, you know, there's a thing that I think people will probably call it a customer experience gap, and it reminds me of a Gilbert cartoon, where we start with the requirements here and you're almost like a Chinese whisper effects and what we deliver is completely different. So we think the testing team or the delivery teams, um, know in our teeth has done a great job. This is what it said in the acceptance criteria, but then our customers are saying, well, actually that's not working this isn't working and there's this kind of gap. Um, we had a great launch this year of agile requirements, it's one of the Broadcom tools. And that was the first time in, ever since I remember actually working within BT, I had customers saying to me, wow, you know, we want more of this. >>We want more projects to have extra requirements design on it because it allowed us to actually work with the business collaboratively. I mean, we talk about collaboration, but how do we actually, you know, do that and have something that both the business and technical people can understand. And we've actually been working with the business , using agile requirements designer to really look at what the requirements are, tease out requirements we hadn't even thought of and making sure that we've got high levels of test coverage. And what we actually deliver at the end of it, not only have we been able to generate tests more quickly, but we've got much higher test coverage and also can more smartly, using the kind of AI within the tool and then some of the other kinds of pipeline tools, actually deliver to choose the right tasks, and actually doing a risk based testing approach. So that's been a great launch this year, but just the start of many kinds of things that we're doing >>Well, what I hear in that, Glynn is a lot of positives that have come out of a very challenging situation. Talk to me about it. And I liked that perspective. This is a very challenging time for everybody in the world, but it sounds like from a collaboration perspective you're right, we talk about that a lot critical with devops. But those challenges there, you guys were able to overcome those pretty quickly. What other challenges did you face and figure out quickly enough to be able to pivot so fast? >>I mean, you talked about culture. You know, BT is like most companies  So it's very siloed. You know we're still trying to work to become closer as a company. So I think there's a lot of challenges around how would you integrate with other tools? How would you integrate with the various different technologies. And BT, we have 58 different IT stacks. That's not systems, that's stacks, all of those stacks can have hundreds of systems. And we're trying to, we've got a drive at the moment, a simplified program where we're trying to you know, reduce that number to 14 stacks. And even then there'll be complexity behind the scenes that we will be challenged more and more as we go forward. How do we actually highlight that to our users? And as an it organization, how do we make ourselves leaner, so that even when we've still got some of that legacy, and we'll never fully get rid of it and that's the kind of trade off that we have to make, how do we actually deal with that and hide that from our users and drive those programs, so we can, as I say, accelerate change,  reduce that kind of waste and that kind of legacy costs out of our business. You know, the other thing as well, I'm sure telecoms is probably no different to insurance or finance. When you take the number of products that we do, and then you combine them, the permutations are tens and hundreds of thousands of products. So we, as a business are trying to simplify, we are trying to do that in an agile way. >>And haven't tried to do agile in the proper way and really actually work at pace, really deliver value. So I think what we're looking more and more at the moment is actually  more value focused. Before we used to deliver changes sometimes into production. Someone had a great idea, or it was a great idea nine months ago or 12 months ago, but actually then we ended up deploying it and then we'd look at the users, the usage of that product or that application or whatever it is, and it's not being used for six months. So we haven't got, you know, the cost of the last 12 months. We certainly haven't gotten room for that kind of waste and, you know, for not really understanding the value of changes that we are doing. So I think that's the most important thing of the moment, it's really taking that waste out. You know, there's lots of focus on things like flow management, what bits of our process are actually taking too long. And we've started on that journey, but we've got a hell of a long way to go. But that involves looking at every aspect of the software delivery cycle. >> Going from, what 58 IT stacks down to 14 or whatever it's going to be, simplifying sounds magical to everybody. It's a big challenge. What are some of the core technology capabilities that you see really as kind of essential for enabling that with this new way that you're working? >>Yeah. I mean, I think we were started on a continuous testing journey, and I think that's just the start. I mean as I say, looking at every aspect of, you know, from a QA point of view is every aspect of what we do. And it's also looking at, you know, we've started to branch into more like AI, uh, AI ops and, you know, really the full life cycle. Um, and you know, that's just a stepping stone to, you know, I think autonomics is the way forward, right. You know, all of this kind of stuff that happens, um, you know, monitoring, uh, you know, watching the systems what's happening in production, how do we feed that back? How'd you get to a point where actually we think about change and then suddenly it's in production safely, or if it's not going to safety, it's automatically backing out. So, you know, it's a very, very long journey, but if we want to, you know, in a world where the pace is in ever-increasing and the demands for the team, and, you know, with the pressures on, at the moment where we're being asked to do things, uh, you know, more efficiently and as lean as possible, we need to be thinking about every part of the process and how we put the kind of stepping stones in place to lead us to a more automated kind of, um, you know, um, the future. >>Do you feel that that planned outcomes are starting to align with what's delivered, given this massive shift that you're experiencing? >>I think it's starting to, and I think, you know, as I say, as we look at more of a value based approach, um, and, um, you know, as I say, print, this was a kind of flow management. I think that that will become ever, uh, ever more important. So, um, I think it starting to people certainly realize that, you know, teams need to work together, you know, the kind of the cousin between business and it, especially as we go to more kind of SAS based solutions, low code solutions, you know, there's not such a gap anymore, actually, some of our business partners that expense to be much more tech savvy. Um, so I think, you know, this is what we have to kind of appreciate what is its role, how do we give the capabilities, um, become more of a centers of excellence rather than actually doing mounds amounts of work. And for me, and from a testing point of view, you know, mounds and mounds of testing, actually, how do we automate that? How do we actually generate that instead of, um, create it? I think that's the kind of challenge going forward. >>What are some, as we look forward, what are some of the things that you would like to see implemented or deployed in the next, say six to 12 months as we hopefully round a corner with this pandemic? >>Yeah, I think, um, you know, certainly for, for where we are as a company from a QA perspective, we are, um, you let's start in bits that we do well, you know, we've started creating, um, continuous delivery and DevOps pipelines. Um, there's still manual aspects of that. So, you know, certainly for me, I I've challenged my team with saying how do we do an automated journey? So if I put a requirement in JIRA or rally or wherever it is and why then click a button and, you know, with either zero touch for one such, then put that into production and have confidence that, that has been done safely and that it works and what happens if it doesn't work. So, you know, that's, that's the next, um, the next few months, that's what our concentration, um, is, is about. But it's also about decision-making, you know, how do you actually understand those value judgments? >>And I think there's lots of the things dev ops, AI ops, kind of that always ask aspects of business operations. I think it's about having the information in one place to make those kinds of decisions. How does it all try and tie it together? As I say, even still with kind of dev ops, we've still got elements within my company where we've got lots of different organizations doing some, doing similar kinds of things, but they're all kind of working in silos. So I think having AI ops as it comes more and more to the fore as we go to cloud, and that's what we need to, you know, we're still very early on in our cloud journey, you know, so we need to make sure the technologies work with cloud as well as you can have, um, legacy systems, but it's about bringing that all together and having a full, visible pipeline, um, that everybody can see and make decisions. >>You said the word confidence, which jumped out at me right away, because absolutely you've got to have be able to have confidence in what your team is delivering and how it's impacting the business and those customers. Last question then for you is how would you advise your peers in a similar situation to leverage technology automation, for example, dev ops, to be able to gain the confidence that they're making the right decisions for their business? >>I think the, the, the, the, the approach that we've taken actually is not started with technology. Um, we've actually taken a human centered design, uh, as a core principle of what we do, um, within the it part of BT. So by using human centered design, that means we talk to our customers, we understand their pain points, we map out their current processes. Um, and then when we mapped out what this process does, it also understand their aspirations as well, you know? Um, and where do they want to be in six months? You know, do they want it to be, um, more agile and, you know, or do they want to, you know, is, is this a part of their business that they want to do one better? We actually then looked at why that's not running well, and then see what, what solutions are out there. >>We've been lucky that, you know, with our partnership, with Broadcom within the payer line, lots of the tools and the PLA have directly answered some of the business's problems. But I think by having those conversations and actually engaging with the business, um, you know, especially if the business hold the purse strings, which in, in, uh, you know, in some companies include not as they do there is that kind of, you know, almost by understanding their, their pain points and then starting, this is how we can solve your problem. Um, is we've, we've tended to be much more successful than trying to impose something and say, well, here's the technology that they don't quite understand. It doesn't really understand how it kind of resonates with their problems. So I think that's the heart of it. It's really about, you know, getting, looking at the data, looking at the processes, looking at where the kind of waste is. >>And then actually then looking at the right solutions. Then, as I say, continuous testing is massive for us. We've also got a good relationship with Apple towards looking at visual AI. And actually there's a common theme through that. And I mean, AI is becoming more and more prevalent. And I know, you know, sometimes what is AI and people have kind of this semantics of, is it true AI or not, but it's certainly, you know, AI machine learning is becoming more and more prevalent in the way that we work. And it's allowing us to be much more effective, be quicker in what we do and be more accurate. And, you know, whether it's finding defects running the right tests or, um, you know, being able to anticipate problems before they're happening in a production environment. >>Well, thank you so much for giving us this sort of insight outlook at dev ops sharing the successes that you're having, taking those challenges, converting them to opportunities and forgiving folks who might be in your shoes, or maybe slightly behind advice enter. They appreciate it. We appreciate your time. >>Well, it's been an absolute pleasure, really. Thank you for inviting me. I have a extremely enjoyed it. So thank you ever so much. >>Excellent. Me too. I've learned a lot for Glenn Martin. I'm Lisa Martin. You're watching the cube >>Driving revenue today means getting better, more valuable software features into the hands of your customers. If you don't do it quickly, your competitors as well, but going faster without quality creates risks that can damage your brand destroy customer loyalty and cost millions to fix dev ops from Broadcom is a complete solution for balancing speed and risk, allowing you to accelerate the flow of value while minimizing the risk and severity of critical issues with Broadcom quality becomes integrated across the entire DevOps pipeline from planning to production, actionable insights, including our unique readiness score, provide a three 60 degree view of software quality giving you visibility into potential issues before they become disasters. Dev ops leaders can manage these risks with tools like Canary deployments tested on a small subset of users, or immediately roll back to limit the impact of defects for subsequent cycles. Dev ops from Broadcom makes innovation improvement easier with integrated planning and continuous testing tools that accelerate the flow of value product requirements are used to automatically generate tests to ensure complete quality coverage and tests are easily updated. >>As requirements change developers can perform unit testing without ever leaving their preferred environment, improving efficiency and productivity for the ultimate in shift left testing the platform also integrates virtual services and test data on demand. Eliminating two common roadblocks to fast and complete continuous testing. When software is ready for the CIC CD pipeline, only DevOps from Broadcom uses AI to prioritize the most critical and relevant tests dramatically improving feedback speed with no decrease in quality. This release is ready to go wherever you are in your DevOps journey. Broadcom helps maximize innovation velocity while managing risk. So you can deploy ideas into production faster and release with more confidence from around the globe. It's the queue with digital coverage of dev ops virtual forum brought to you by Broadcom. >>Hi guys. Welcome back. So we have discussed the current state and the near future state of dev ops and how it's going to evolve from three unique perspectives. In this last segment, we're going to open up the floor and see if we can come to a shared understanding of where dev ops needs to go in order to be successful next year. So our guests today are, you've seen them all before Jeffrey Hammond is here. The VP and principal analyst serving CIO is at Forester. We've also Serge Lucio, the GM of Broadcom's enterprise software division and Glenn Martin, the head of QA transformation at BT guys. Welcome back. Great to have you all three together >>To be here. >>All right. So we're very, we're all very socially distanced as we've talked about before. Great to have this conversation. So let's, let's start with one of the topics that we kicked off the forum with Jeff. We're going to start with you spiritual co-location that's a really interesting topic that we've we've uncovered, but how much of the challenge is truly cultural and what can we solve through technology? Jeff, we'll start with you then search then Glen Jeff, take it away. >>Yeah, I think fundamentally you can have all the technology in the world and if you don't make the right investments in the cultural practices in your development organization, you still won't be effective. Um, almost 10 years ago, I wrote a piece, um, where I did a bunch of research around what made high-performance teams, software delivery teams, high performance. And one of the things that came out as part of that was that these teams have a high level of autonomy. And that's one of the things that you see coming out of the agile manifesto. Let's take that to today where developers are on their own in their own offices. If you've got teams where the team itself had a high level of autonomy, um, and they know how to work, they can make decisions. They can move forward. They're not waiting for management to tell them what to do. >>And so what we have seen is that organizations that embraced autonomy, uh, and got their teams in the right place and their teams had the information that they needed to make the right decisions have actually been able to operate pretty well, even as they've been remote. And it's turned out to be things like, well, how do we actually push the software that we've created into production that would become the challenge is not, are we writing the right software? And that's why I think the term spiritual co-location is so important because even though we may be physically distant, we're on the same plane, we're connected from a, from, from a, a shared purpose. Um, you know, surgeon, I worked together a long, long time ago. So it's been what almost 15, 16 years since we were at the same place. And yet I would say there's probably still a certain level of spiritual co-location between us, uh, because of the shared purposes that we've had in the past and what we've seen in the industry. And that's a really powerful tool, uh, to build on. So what do tools play as part of that, to the extent that tools make information available, to build shared purpose on to the extent that they enable communication so that we can build that spiritual co-location to the extent that they reinforce the culture that we want to put in place, they can be incredibly valuable, especially when, when we don't have the luxury of physical locate physical co-location. Okay. That makes sense. >>It does. I shouldn't have introduced us. This last segment is we're all spiritually co-located or it's a surge, clearly you're still spiritually co located with jump. Talk to me about what your thoughts are about spiritual of co-location the cultural impact and how technology can move it forward. >>Yeah. So I think, well, I'm going to sound very similar to Jeff in that respect. I think, you know, it starts with kind of a shared purpose and the other understanding, Oh, individuals teams, uh, contributed to kind of a business outcome, what is our shared goal or shared vision? What's what is it we're trying to achieve collectively and keeping it kind of aligned to that? Um, and so, so it's really starts with that now, now the big challenge, always these over the last 20 years, especially in large organization, there's been specialization of roles and functions. And so we, we all that started to basically measure which we do, uh, on a daily basis using metrics, which oftentimes are completely disconnected from kind of a business outcome or purpose. We, we kind of reverted back to, okay, what is my database all the time? What is my cycle time? >>Right. And, and I think, you know, which we can do or where we really should be focused as an industry is to start to basically provide a lens or these different stakeholders to look at what they're doing in the context of kind of these business outcomes. So, um, you know, probably one of my, um, favorites experience was to actually weakness at one of a large financial institution. Um, you know, Tuesday Golder's unquote development and operations staring at the same data, right. Which was related to, you know, in calming changes, um, test execution results, you know, Coverity coverage, um, official liabilities and all the all ran. It could have a direction level links. And that's when you start to put these things in context and represent that to you in a way that these different stakeholders can, can look at from their different lens. And, uh, and it can start to basically communicate and, and understand have they joined our company to, uh, to, to that kind of common view or objective. >>And Glen, we talked a lot about transformation with you last time. What are your thoughts on spiritual colocation and the cultural part, the technology impact? >>Yeah, I mean, I agree with Jeffrey that, you know, um, the people and culture, the most important thing, actually, that's why it's really important when you're transforming to have partners who have the same vision as you, um, who, who you can work with, have the same end goal in mind. And w I've certainly found that with our, um, you know, continuing relationship with Broadcom, what it also does though, is although, you know, tools can accelerate what you're doing and can join consistency. You know, we've seen within simplify, which is BTS flagship transformation program, where we're trying to, as it can, it says simplify the number of systems stacks that we have, the number of products that we have actually at the moment, we've got different value streams within that program who have got organizational silos. We were trying to rewrite, rewrite the wheel, um, who are still doing things manually. >>So in order to try and bring that consistency, we need the right tools that actually are at an enterprise grade, which can be flexible to work with in BT, which is such a complex and very dev, uh, different environments, depending on what area of BT you're in, whether it's a consumer, whether it's a mobile area, whether it's large global or government organizations, you know, we found that we need tools that can, um, drive that consistency, but also flex to Greenfield brownfield kind of technologies as well. So it's really important that as I say, for a number of different aspects, that you have the right partner, um, to drive the right culture, I've got the same vision, but also who have the tool sets to help you accelerate. They can't do that on their own, but they can help accelerate what it is you're trying to do in it. >>And a really good example of that is we're trying to shift left, which is probably a, quite a bit of a buzz phrase in their kind of testing world at the moment. But, you know, I could talk about things like continuous delivery direct to when a ball comes tools and it has many different features to it, but very simply on its own, it allows us to give the visibility of what the teams are doing. And once we have that visibility, then we can talk to the teams, um, around, you know, could they be doing better component testing? Could they be using some virtualized services here or there? And that's not even the main purpose of continuous delivery director, but it's just a reason that tools themselves can just give greater visibility of have much more intuitive and insightful conversations with other teams and reduce those organizational silos. >>Thanks, Ben. So we'd kind of sum it up, autonomy collaboration tools that facilitate that. So let's talk now about metrics from your perspectives. What are the metrics that matter? Jeff, >>I'm going to go right back to what Glenn said about data that provides visibility that enables us to, to make decisions, um, with shared purpose. And so business value has to be one of the first things that we look at. Um, how do we assess whether we have built something that is valuable, you know, that could be sales revenue, it could be net promoter score. Uh, if you're not selling what you've built, it could even be what the level of reuse is within your organization or other teams picking up the services, uh, that you've created. Um, one of the things that I've begun to see organizations do is to align value streams with customer journeys and then to align teams with those value streams. So that's one of the ways that you get to a shared purpose, cause we're all trying to deliver around that customer journey, the value with it. >>And we're all measured on that. Um, there are flow metrics which are really important. How long does it take us to get a new feature out from the time that we conceive it to the time that we can run our first experiments with it? There are quality metrics, um, you know, some of the classics or maybe things like defect, density, or meantime to response. Um, one of my favorites came from a, um, a company called ultimate software where they looked at the ratio of defects found in production to defects found in pre production and their developers were in fact measured on that ratio. It told them that guess what quality is your job to not just the test, uh, departments, a group, the fourth level that I think is really important, uh, in, in the current, uh, situation that we're in is the level of engagement in your development organization. >>We used to joke that we measured this with the parking lot metric helpful was the parking lot at nine. And how full was it at five o'clock. I can't do that anymore since we're not physically co-located, but what you can do is you can look at how folks are delivering. You can look at your metrics in your SCM environment. You can look at, uh, the relative rates of churn. Uh, you can look at things like, well, are our developers delivering, uh, during longer periods earlier in the morning, later in the evening, are they delivering, uh, you know, on the weekends as well? Are those signs that we might be heading toward a burnout because folks are still running at sprint levels instead of marathon levels. Uh, so all of those in combination, uh, business value, uh, flow engagement in quality, I think form the backbone of any sort of, of metrics, uh, a program. >>The second thing that I think you need to look at is what are we going to do with the data and the philosophy behind the data is critical. Um, unfortunately I see organizations where they weaponize the data and that's completely the wrong way to look at it. What you need to do is you need to say, you need to say, how is this data helping us to identify the blockers? The things that aren't allowing us to provide the right context for people to do the right thing. And then what do we do to remove those blockers, uh, to make sure that we're giving these autonomous teams the context that they need to do their job, uh, in a way that creates the most value for the customers. >>Great advice stuff, Glenn, over to your metrics that matter to you that really make a big impact. And, and, and also how do you measure quality kind of following onto the advice that Jeff provided? >>That's some great advice. Actually, he talks about value. He talks about flow. Both of those things are very much on my mind at the moment. Um, but there was this, I listened to a speaker, uh, called me Kirsten a couple of months ago. It taught very much around how important flow management is and removing, you know, and using that to remove waste, to understand in terms of, you know, making software changes, um, what is it that's causing us to do it longer than we need to. So where are those areas where it takes long? So I think that's a very important thing for us. It's even more basic than that at the moment, we're on a journey from moving from kind of a waterfall to agile. Um, and the problem with moving from waterfall to agile is with waterfall, the, the business had a kind of comfort that, you know, everything was tested together and therefore it's safer. >>Um, and with agile, there's that kind of, you know, how do we make sure that, you know, if we're doing things quick and we're getting stuff out the door that we give that confidence, um, that that's ready to go, or if there's a risk that we're able to truly articulate what that risk is. So there's a bit about release confidence, um, and some of the metrics around that and how, how healthy those releases are, and actually saying, you know, we spend a lot of money, um, um, an investment setting up our teams, training our teams, are we actually seeing them deliver more quickly and are we actually seeing them deliver more value quickly? So yeah, those are the two main things for me at the moment, but I think it's also about, you know, generally bringing it all together, the dev ops, you know, we've got the kind of value ops AI ops, how do we actually bring that together to so we can make quick decisions and making sure that we are, um, delivering the biggest bang for our buck, absolutely biggest bang for the buck, surge, your thoughts. >>Yeah. So I think we all agree, right? It starts with business metrics, flow metrics. Um, these are kind of the most important metrics. And ultimately, I mean, one of the things that's very common across a highly functional teams is engagements, right? When, when you see a team that's highly functioning, that's agile, that practices DevOps every day, they are highly engaged. Um, that that's, that's definitely true. Now the, you know, back to, I think, uh, Jeff's point on weaponization of metrics. One of the key challenges we see is that, um, organizations traditionally have been kind of, uh, you know, setting up benchmarks, right? So what is a good cycle time? What is a good lead time? What is a good meantime to repair? The, the problem is that this is very contextual, right? It varies. It's going to vary quite a bit, depending on the nature of application and system. >>And so one of the things that we really need to evolve, um, as an industry is to understand that it's not so much about those flow metrics is about our, these four metrics ultimately contribute to the business metric to the business outcome. So that's one thing. The second aspect, I think that's oftentimes misunderstood is that, you know, when you have a bad cycle time or, or, or what you perceive as being a buy cycle time or better quality, the problem is oftentimes like all, do you go and explore why, right. What is the root cause of this? And I think one of the key challenges is that we tend to focus a lot of time on metrics and not on the eye type patterns, which are pretty common across the industry. Um, you know, if you look at, for instance, things like lead time, for instance, it's very common that, uh, organizational boundaries are going to be a key contributor to badly time. >>And so I think that there is, you know, the only the metrics there is, I think a lot of work that we need to do in terms of classifying, descend type patterns, um, you know, back to you, Jeff, I think you're one of the cool offers of waterscrumfall as a, as, as a key pattern, the industry or anti-spatter. Um, but waterscrumfall right is a key one, right? And you will detect that through kind of a defect arrival rates. That's where that looks like an S-curve. And so I think it's beyond kind of the, the metrics is what do you do with those metrics? >>Right? I'll tell you a search. One of the things that is really interesting to me in that space is I think those of us had been in industry for a long time. We know the anti-patterns cause we've seen them in our career maybe in multiple times. And one of the things that I think you could see tooling do is perhaps provide some notification of anti-patterns based on the telemetry that comes in. I think it would be a really interesting place to apply, uh, machine learning and reinforcement learning techniques. Um, so hopefully something that we'd see in the future with dev ops tools, because, you know, as a manager that, that, you know, may be only a 10 year veteran or 15 year veteran, you may be seeing these anti-patterns for the first time. And it would sure be nice to know what to do, uh, when they start to pop up, >>That would right. Insight, always helpful. All right, guys, I would like to get your final thoughts on this. The one thing that you believe our audience really needs to be on the lookout for and to put on our agendas for the next 12 months, Jeff will go back to you. Okay. >>I would say look for the opportunities that this disruption presents. And there are a couple that I see, first of all, uh, as we shift to remote central working, uh, we're unlocking new pools of talent, uh, we're, it's possible to implement, uh, more geographic diversity. So, so look to that as part of your strategy. Number two, look for new types of tools. We've seen a lot of interest in usage of low-code tools to very quickly develop applications. That's potentially part of a mainstream strategy as we go into 2021. Finally, make sure that you embrace this idea that you are supporting creative workers that agile and dev ops are the peanut butter and chocolate to support creative, uh, workers with algorithmic capabilities, >>Peanut butter and chocolate Glen, where do we go from there? What are, what's the one silver bullet that you think folks to be on the lookout for now? I, I certainly agree that, um, low, low code is, uh, next year. We'll see much more low code we'd already started going, moving towards a more of a SAS based world, but low code also. Um, I think as well for me, um, we've still got one foot in the kind of cow camp. Um, you know, we'll be fully trying to explore what that means going into the next year and exploiting the capabilities of cloud. But I think the last, um, the last thing for me is how do you really instill quality throughout the kind of, um, the, the life cycle, um, where, when I heard the word scrum fall, it kind of made me shut it because I know that's a problem. That's where we're at with some of our things at the moment we need to get beyond that. We need >>To be releasing, um, changes more frequently into production and actually being a bit more brave and having the confidence to actually do more testing in production and go straight to production itself. So expect to see much more of that next year. Um, yeah. Thank you. I haven't got any food analogies. Unfortunately we all need some peanut butter and chocolate. All right. It starts to take us home. That's what's that nugget you think everyone needs to have on their agendas? >>That's interesting. Right. So a couple of days ago we had kind of a latest state of the DevOps report, right? And if you read through the report, it's all about the lost city, but it's all about sweet. We still are receiving DevOps as being all about speed. And so to me, the key advice is in order to create kind of a spiritual collocation in order to foster engagement, we have to go back to what is it we're trying to do collectively. We have to go back to tie everything to the business outcome. And so for me, it's absolutely imperative for organizations to start to plot their value streams, to understand how they're delivering value into aligning everything they do from a metrics to deliver it, to flow to those metrics. And only with that, I think, are we going to be able to actually start to really start to align kind of all these roles across the organizations and drive, not just speed, but business outcomes, >>All about business outcomes. I think you guys, the three of you could write a book together. So I'll give you that as food for thought. Thank you all so much for joining me today and our guests. I think this was an incredibly valuable fruitful conversation, and we appreciate all of you taking the time to spiritually co-located with us today, guys. Thank you. Thank you, Lisa. Thank you. Thank you for Jeff Hammond serves Lucio and Glen Martin. I'm Lisa Martin. Thank you for watching the broad cops Broadcom dev ops virtual forum.

>> Narrator: From theCUBE Studios in Palo Alto, in Boston, connecting with thought leaders all around the world, this is a CUBE conversation. >> Hey, welcome back, everybody. Jeff Frick here with theCUBE. We're in our Palo Alto Studios today, talking about a pretty interesting topic. You probably haven't heard of it, but you're going to know a lot of the attributes and it's going to sound very familiar. And that's BizOps, the concept of BizOps. We've heard about DevOps and DevSecOps and a whole bunch of ops, but BizOps is really a new twist and a new way to think about this. And we're excited to have the woman who actually wrote the book on the topic. She's Laureen Knudsen. She's a Chief Transformation Officer from Broadcom. She's also the co-author of the "Modern Business Management: Creating a Built-to-Change Organization", and a founding member of the BizOps Coalition. Laureen, great to see you. >> Great to be here. Thanks so much for having me. >> Absolutely. For people that aren't familiar with BizOps, give us kind of the quick high level. What is BizOps? >> BizOps is a new way of doing business. Just like Agile changed engineering and DevOps changed how we put things into production, BizOps is changing from soup to nuts. So from concept to cash or strategy to execution, right. There's a lot of... This has been talked about for a few years now, but this is formalizing that structure. So what do you need to do to truly have your strategy linked to your customer base? And so it's creating that umbrella over all of these other ops processes that brings it all together to tie the top to the bottom. >> Right. So, DevOps, right, fundamentally changed, the way the software gets developed. There used to be waterfall, it used to be data market's development document and then a product requirements document, then you put together a plan and you code it for six months or nine months, threw it over the Wall operations, and then hopefully they delivered. That doesn't happen anymore. And that was really set forth about 20 years ago when this kind of revolution happened on the software side. So what's been happening on the business side, and why now do they need their own ops to be pulled into this process? >> Well, sort of in the same way that things happened in the late 1990s, where certain organizations started to realize that that wasn't the most efficient way to create software and came together and created the Agile Manifesto. We've realized that there's certain things in doing business that make us much more effective and efficient. Things like bringing a data stream from the top to the bottom so every level of the organization has the data they need to run their business. Having that trust run throughout the organization, having that communication and that transparency from the strategy to the execution. You know, the global economy is just in dire straits right now, and the world is moving faster than ever. And so being able to respond to that change is vital at all levels of the organization. >> So you wrote the book years ago, I'm sure you've speaking to ton of business leaders, you know, as an author of the book, what were the biggest inhibitors to kind of the adoption of these ops and there must have been something, because why then did you found this coalition? What was the, you know, kind of the founding principle behind the coalition? >> Well, a bunch of industry leaders have come together to realize that in the same way that development needed to change in the early 2000s, really business needs to change today. And to your point, we've been talking about this for a while. Different companies are doing it better than others. And the ones that are doing this well are really heads and tails succeeding above the others. So, it's not easy though. It's not easy to change an entire organization and to change the way you do business. So, the coalition is bringing together some principles and values. We've come together to talk about how we're doing business differently and what actually works. And the main things you need to focus on in order to ensure success. >> Right. But you did it loud and proud with this declarative manifesto and then an event, actually, later this month that you're going to have to really unveil the manifesto, October 15th. I think it's 9:00, or excuse me, >> 11: 00 AM Eastern, 8:00 AM Pacific. Manifesto, right? Just the word manifesto, elicits all types of, kind of emotional response and really strong declarative statement of purpose and mission. So, why the manifesto and what's really the key pieces of the manifesto? >> You know, you need the principles that go along to help you change people, process and technology. And a lot of folks are focusing only on the technology and the data that comes from that technology and all that is key and vital to the way that you run your business differently. It's not the only piece. And so we need to focus on how do we get to bring the people along with us, how do we change our processes to be more efficient and effective. And the four values and the principles that we've created as this coalition, really help companies to do that more easily and to know they're on the right track, in the same way that the Agile principles and the values that brought out in the Agile Manifesto did. >> Right. So, I have a preview version here of the values. And I think it is really important for people to stay kind of fundamental values. 'Cause then everything builds from that and if there's ever a question, you can go back to the values as of a reference point. But just to read a few of, you know, business outcomes over individual projects, trust and collaboration over siloed teams and organizations, data-driven decisions over opinions and judgment calls, and finally, learn responded pivot over following a documented plan. And that seems so, right, so simple and so foundational and so fundamental to the way business works today. But the fact that you have to put this coalition together, and the fact that you're publishing this manifesto, tells me that the adoption really isn't where it should be. And this is really a new way to try to drive the adoption of these values. >> Absolutely. I mean, everybody seems to understand that they need to focus on their customers and that they need to focus on outcomes, but you can't just take something, you know, once you have work in progress and say, well, what's the value of this one piece of work. You have to have started at the beginning to come with the right outcome you're trying to meet, and then ensure that you're doing that all along the path to creating that and to bringing that to your customer base. It's focusing on your customers and creating the trust with your customers as well as through your organization. The data is really vital. Being able to run our businesses on real data and know the reality of the situation rather than at status reports that were created by people saying, yeah, I'm done, but there's no definition of done, right? It's fundamentally changing how we do business, which sounds easy. But as we know because of the Agile transformations that we've done and DevOps transformations that we've done, it's not as easy as it sounds. >> Right. So, why not just try to include more of the business people in the DevOps process? Why the strategy to have BizOps as kind of a standalone activity and again, to have the coalition and manifesto, that means it's super important. Can't the business people participate in the DevOps, or why has that not really been effective? >> It's really a different part of the business. And BizOps is a framework that pulls together all of these other operational pieces. So, security, operations, you know, how do you get something from engineering out to your customers, really were DevOps focuses, right? So, that's great. But running your business includes a lot more than your IT organization or your engineering teams. So this really expands out and brings in all of the rest of the business for how you sell software, how you plan, how you fund your teams, how you look at the work from that high strategic level and ensuring that you create that solid pipeline of data so that you truly know the status of any strategy in your organization. I was working with one group who had really good strategies and they had really good execution and they found that they spent over $100 million annually rolling up that data to try and understand the true status of their strategies. So companies are spending and are being very inefficient in, you know, they're spending millions of dollars on trying to do this link where if you just fundamentally change the way you do business a little bit, day to day, you can have that as a natural outcome of your processes. >> Right. 'Cause you've talked about on some of this stuff about using it as a way to do prioritization and to make sure you're not spending money places that you shouldn't. Another thing that strikes me as I go through the principles are, again, things that in 2020 should not be new information, you know, frequent changes, which was not part of the old paradigm. Trust and transparency. And I think you even tied it back into one of the articles I saw, tying trust and transparency really back to employee engagement, which then drives profitability and productivity. So I wonder if you can talk about the role of trust and in your conversations with people, as you've been kind of developing this idea over the years since the book, getting leaders to, you know, to trust their people, to do what that needs to be done rather than managing tasks, you know, manage the outcome, not manage tasks. >> Right. This is really important. Having trust in your organizations, especially today when everyone's remote, right? And in almost every company in the globe right now, most of their employees are working from their houses. You can't really do command and control well when no one is sitting in your building with you. So being able to have that trust to truly trust in your employees, you know, we spend a lot of money on all of these technical folks that we hire, and then we put people in place to try and direct them what to do on a daily basis. And so having... Building that trust within your organization, and it goes both ways, right? Employees need to trust the leadership, leadership needs to trust the employees, but it's not just from the top level to the end level, right? To the team level. It's actually every level in the middle. So this is truly pulling the pieces of work that we've done over the past few years through the entire organization. It's getting rid of what we call that frozen middle, of middle management and making sure that trust is aligned in there as well. And that the communication and transparency is working through that part of the organization. >> Right. Another principle I want to highlight is talking about the role of machine learning and artificial intelligence. Clearly, we all know, right, data's exploding, et cetera, et cetera, and we want to get the data driven decisions. But what this really calls out is that there's probably more data, both in terms of frequency and complexity, than people can really sift through, in terms of finding what they should be working on and what's important and what's not, you know, the classic separating the signal from the noise. I wonder if you can speak to a little bit about the role of machine learning and artificial intelligence, as an enhancer to productivity in this BizOps world versus a threat to people's jobs. >> Absolutely. I mean, like I said already that there's some companies spending $100 million rolling up data on things that computers can do today, even without machine learning and an AI. But when we put that into place, it really doesn't replace people any more than DevOps removed people from the organization. We automated a lot in testing yet we still have test organizations. It's just a different focus and a way of doing business. And this is no different. I'm seeing a lot of companies though start to try and throw all of their data together. And I've recently started saying that they're creating data land fields when they're attempting to create data lakes. And so you really need to understand your data that you're collecting and why you're collecting it and what outcomes you're trying to get from that data so that you can understand your business and you're not just creating, to your point, more noise. >> Right. So let's shift gears a little bit and talk about the event that's coming up on the 15th, about, you know, kind of, what is the role of the coalition? How should people get involved, what's membership all about, and then what can they expect to happen on the 15th? >> We have 10 industry leaders that have come together to author the BizOps Manifesto. And it's everyone from influencers, transformation experts, CEOs of a lot of companies or of organizations. We have people like Evan Leybourn of the Business Agility Institute and Sally Elatta from AgilityHealth, who have come to help author this and are really transformational leaders across the globe. And to get involved, you can go to bizopsmanifesto.org. and you can sign the manifesto. You can align to that if, you know, if you want to bring this into your own organization, we're happy to help work with that as well. So it's a group of industry leaders who are here to help the globe get more efficient and effective in how they do business. >> It's really interesting, right. It's not really an open source project, but it is kind of a co-opetition in terms of, you know, you're reaching out to lots of different companies and lots of different leaders to participate. They may or may not be competitive, but really this is more kind of an industry, kind of productivity thing, if you will, to bring all these people together at the coalition. Would that be accurate? >> It is accurate, but we're also looking to have competitors. I mean, we've... Competitors is an interesting thing today because there's no company just uses one company software, for example, to automate all of their pieces, right? There's all of these products that have to come together and share data today in the same way that we needed to share, you know, access to software. In the past, integrations were really difficult and now, you know, everyone's got open APIs. It's a very similar thing with data today. And so we are working with our competitors and we're working with, you know, like you said, industry leaders. We have Mik Kersten from Tasktop as part of this as well. We're looking at how we can benefit the companies of the world today, much more efficiently and effectively than we have in the past. So it is a group of people who compete with each other, maybe on a daily basis, but also have the same customers and have the need to help companies today, especially in this economy with the pandemic, right. There's a lot of companies in dire straits right now and we all need to come together as business leaders to help those companies get through this time. And anything that we can do to do that is going to benefit us all in the long run. >> Right. You know, it is really interesting co-opetition, is like you say, most companies have everybody's, you know, a lot of different products and people compete as well as having API connections and having all kinds of interesting relationships. So the lines are not so clean, like they used to be. And as we've seen with DevOps, you know, significant delta in the productivity and the responsiveness and the way software is delivered. So, sounds super exciting. We'll look forward to the event on the 15th. I give you the last word. What are you looking most forward to for the big launch in a couple of weeks? >> I'm really excited for people to give us their feedback on what they think and how this benefits them. And I'm excited to help our customers and help the, you know, the big companies of the world get through these next 18 months. I think we're all in for a bit more of a struggled time, you know, at a difficult time, and anything that we can all do to work together. So I'm looking forward to working with other industry leaders on this as well, and to the benefit of, you know, the global economy. >> Right. Well, great. Well, Laureen, thank you for giving us the one on one on BizOps. Really appreciate it. And best of luck to you and good luck to you and the team on the 15th. >> Thanks so much. >> Alrighty. Thank you. All right, she's Laureen, I'm Jeff. You're watching theCUBE. Thanks for watching. We'll see you next time. (upbeat music begins)

>> Narrator: From theCUBE studios in Palo Alto in Boston, connecting with thought leaders all around the world, this is theCUBE conversation. >> Hi, I'm Stu Miniman. And welcome to this CUBE conversation. Of course during the COVID-19 pandemic, lots of businesses and industries have been upended. One area where there's been real acceleration of the use of online technology, of course, has been telehealth and telemedicine. To help us look into what is happening in that space. We have Eric Gray, he is the chief Solutions Architect with NetScout. Eric, thanks so much for joining us. >> Thanks to you, it's great to be here. >> All right, so as I teed it up, obviously, telehealth, telemedicine. I've had most of my family have done virtual visits, if you will, you know talking to doctors in the like online has been a real shift not something that is pervasive today. Help us understand a little bit how your customers are dealing with this, and the changes that are happening in their world? >> Well, it's certainly becoming a significant paradigm shift in our industry, you think over, over the history of medicine, people have been going in and seeing a doctor sitting in that waiting room and going through all of the, the permutations to spend, 10 minutes with the doctor to diagnose their symptoms. The shift that we have and driven in the fact that has been driven by a global pandemic is, maybe it's unfortunate, but at the same time, it is pushing the industry, strongly in that direction. They say that by by 2021, this is a $66 billion industry or business. So, healthcare organizations be at hospitals and clinics, local providers, anybody that's having to deal with medicine back and forth, in an interaction with their patients. He's going to make this shift over, over a very short period of time. >> In general Eric, how prepared was the typical practitioner to be able to support this kind of environment? You know, we've seen what's happened with local elementary school education, most of them aren't set up for remote as opposed to if I looked at, secondary schools, universities usually had some component of online learning. But when it comes to the medical industry, do you have any thing you can share as to, what segments of the market were ready? How many just had to scramble and say, oh, my gosh, I need this by Monday. >> So there were certainly the larger healthcare providers that I spend my time with, here in the Western US they were ready to go. They had been looking forward into this field for quite a while they had the technology in place, but not was certainly not the case for all. I've spent more time in the last three months talking to university healthcare organizations, local healthcare organizations, who weren't at all ready to roll out the technology necessary to be able to provide that doctor-patient interaction in a successful and high quality way. >> All right, well, let's let's drill in a little bit because most people think, oh, I'm going to move to an online experience. It doesn't just mean, if I was a restaurant, it doesn't just mean that, I have an app or an online, portal. If I was school, it's not just let's throw zoom at the solution. If you're talking, telehealth and telemedicine. I'm sure there's a lot that needs to be done, ahead of the any visits, obviously, heavily regulated industry. So let's walk through a free quick could the, the full landscape there. >> So the the biggest concerns that a lot of the healthcare organizations have they're trying to roll this out. Probably the biggest one by far is maintaining a level of HIPAA compliance. So that the data that's been moving back and forth between the doctor-patient is staying exactly there it's private. It's not exposed, even though it's going across public internet, in many cases, from someone's home to the the location of the physician, that that information remains confidential. Second, it really needs to be high quality, as the doctor is interacting with the patient now in his, kind of the same fashion that you and I are right now, over a webcam over their local ISP, the quality might vary. So, if a doctor is going to make an accurate assessment of a patient, and assess their symptoms without actually having them come into an office, they need to have an exceptional experience, the quality of the audio needs to be great quality, the video needs to be excellent. The entire interaction needs to be pristine. And then there's the things that wrap around that patient doctor experience, the things that give us the call it the infrastructure that makes it happen. That's the DNS connections in the underlying network, but it's also prior to the call making sure that you have the ability to set it up, access medical records, after the call, being able to get to pharmacy to get to your prescription, or see the test results that came from the experience. Even billing, I'm going to go pay my bill, I need to be able to get on, get to something reliably and have a secure transaction. All of this stuff together sort of makes up what is modern telemedicine. Though, most of the time, the telehealth experiences what's considered everything, whereas telemedicine is really looked at as the doctor patient conversation, across that new digital media. >> Yeah, what if companies had to deal with if they had really a toe or they were starting down this path and all of a sudden they need to go from something that they do as an exception to now this is what they've been doing for the last few months. How do they scale that up? >> That was a shock for many of them. Some had some, basic level of interaction capability. But I've had customers that have talked to me about a 20 to 30 x increase in the amount of bandwidth necessary and the amount of technology needed in order to facilitate these conversations. The market is skyrocketing. Doctors are you know, they're making this dramatic shift because they need to protect their patients they need to protect themselves. And as the need has gone up exponentially, IT teams are really scrambling. They're having to provide this technology very, very quickly standing up new concentrators, for VPN connections. Lots of new service provider connections, so that they have additional bandwidth capable. And then going out to the different companies who provide direct telemedicine and telehealth connectivity, so that they are maintaining, that high level of security as well. So all of this together has just created this explosion in this industry as people rush to deploy the stuff. >> It definitely sounds very challenging. I've talked to, government agencies that get emergency funding for this. What's the impact on from a financial standpoint? I think from a patient standpoint, you say, it's not like all of a sudden you're going to be able to bill more. If anything, they're like, hey, I'm not coming to the office. I'm you know, is a little bit less to go there. So what are the financial implications of all this? >> That's really interesting. So, as many healthcare companies especially the hospitals ramped up to fight COVID-19, and the coronavirus epidemic, getting access to the appropriate PPE and emergency room technology, making sure they have enough ventilators. All that stuff was a big drain on the emergency funds. When they looked at what was going on with telemedicine. It's really a dramatic savings. So the survey say that somewhere in the order of the United States healthcare industry overall. As we shift into a primarily telemedicine based system, it save up to $4 billion a year. So it's significantly less expensive for those health care companies to be able to provide this kind of interaction. Not only money, but also from a quality of the interaction as well. Now, as I said it kind of in the beginning, I know when I would go in and talk to a doctor, maybe I would get 10 minutes. There's a lot of time that you spend sitting in the waiting room, waiting in the in the actual room, and the interaction is very short, and maybe not such great quality. Now, as I've been spending a few sessions with doctors online, it's really great. I've got no waiting. I've got a longer window of time with my physician. I think it's probably, a better interaction for me and overall, it's going to save the healthcare company significant amount of money. Seems like it makes a lot of sense. >> Yeah, that's an interesting silver lining, if you will, that we can right really kind of, change it from, it was almost done. Just in time manufacturing methodology, as we've maximized the utilization of everything with all the scheduling and the like, and we're really building it more like a distributed system now. So I'm curious, Eric, what is the thinking around these people, these companies, if you're scaling this up for remote, eventually, there will be the new normal, let's say we have, you know, a vaccine and, going back to the office visits will be more prevalent. What is the thinking about, what this will look like and hybrid mode or what will the telemedicine dial back a little bit, in the next year or so? >> I think the general consensus is that it's here to stay. This isn't the first pandemic, it won't be the last and putting the proper technology in place right now, that's available. I mean, this is not something that's years in the making, it's out there. It's just that a lot of companies, weren't quite ready to take the lead, either from an investment standpoint or just doing things the same way and making that paradigm shift. I believe not only are we seeing the significant shift just in this timeframe, but it's going to be here for a long period of time. They're going to be certainly people that will want to go back to the old way of visiting the doctor. And as at home diagnostics become more, more prevalent things from like a blood pressure monitor or pulse ox monitor, various ways that you can actually take vital readings from your home and have that data transmitted into your EMR, EHR system. That makes it even more sticky. So I believe the time is going to come where we'll set up a couple of steps back, but those 10 steps that we've made forward, it's something that the industry has been waiting for for a long time. And now we're going to get there really quickly. >> Yeah, it's fascinating to think, Eric, if this had been 10 years ago, that we would be having a very different conversation. If you would take us in a little bit the learnings that you had, whereas NetScout finding that it's helping its clients the most when it comes to the telehealth and telemedicine solutions? >> Well, one of the things that's really gotten us excited at NetScout, we've been in this business of being able to secure and monitor, enterprise and service provider networks for the last 35 plus years. NetScout has been in this business to keep the customers networks alive, keep them healthy, and help them to troubleshoot problems when they occur. So as we look at applying, our technology towards this telemedicine experience, it seemed like a perfect fit for us. We can break it down in kind of three categories. First, what happens prior to the experience? We want to make sure that we can maintain a high level of availability for the the healthcare organizations network to make sure that the telehealth software is functional, that the network is robust that the response times are low. So understanding what that experience is like in advance of the call, is probably a little bit of a slam dunk. But we want to make sure that we're always ready and able to handle the load. Second is, and probably most important is during the call. Once that patient is talking to the doctor, and they're ongoing through video, audio chat, we want to make sure that, the quality of that experience is exceptional. About 10 years ago, NetScout acquired some technology that gave us the insight into how unified communication protocols function, and gave us the ability to measure my scores jitter and loss, even in a secure RTP kind of payload environment. So even with encryption, we can still give you a high understanding of how good that session is to make sure that the patient and doctor, are seeing each other, they're hearing each other and it's pristine. Then finally on the back end, what happens after the call. So once the physician and the patient are done, I still need to go see my records and the bill. As I said before, we want to make sure that all the systems that make that happen are up, functional and capable of being used every day. Our ability to monitor these sessions baseline their performance and triage in the event of an issue helps us to keep EMR systems like Epic and Cerner and McKesson up and running. The billing systems that make things happen. HL seven protocol tying everything together. Giving the patient access to their records, their medical images, et cetera. And the network that makes all this happen, probably already monitored by NetScout as our customers are very loyal and have been for many years. >> Alright, Eric, I'll give you the final word. If customers want to learn more about what you're doing in this space, what would you recommend for them? >> Well, we are very excited about what we're doing with all of these solutions for our customers. First we published a white paper that you can find it at netscout.com. We show up on a telemedicine landing page you can read all about how NetScout products are being used to help in all of these areas of telemedicine. Also on the July 21st, at 10:00 am Pacific, we're going to be offering a live webinar, demonstrating how our technology can be used before, during and after a telemedicine call for the customer. >> All right, well, Eric Gray, thank you so much for joining us really important stuff around the telehealth and telemedicine. Really appreciate all the updates. >> Thanks to have a great day. >> All right, and thank you for joining. I'm Stu Miniman, thank you for watching theCUBE. (bright upbeat music)

>> Narrator: Live from Las Vegas it's theCUBE! Covering AWS Reinvent 2017 presented by AWS, Intel, and our ecosystem of partners. >> And we are back live here in Las Vegas. We are at the Sands as Reinvent day two wraps up. AWS here for four days and we'll be here live again tomorrow, by the way on theCUBE continuing our coverage. We're wrapping up here today saving the best for last. No doubt about that. Stu Miniman and John Walls were joined by a couple of folks from SoftwareONE. Mike Gersten, immediately to my right. Global Innovation and Strategy. Mike, good to see you, sir. >> Yeah, likewise, thanks for having us. >> And Lawrence Schwartz on the far side there. The CMO at SoftwareONE, Lawrence, good to see you. >> Pleasure to see you again. >> Tell us a little bit, first off, about not only what you do but why you're here. >> Sure, yeah. So, SoftwareONE, we're in the business of making sure that people are spending the right amount of money on their software. Not too much, and you know making sure they're not underspending either. So all that optimization to spend. We also help customer implement their technology. We're at this show because we help customers do that in a Cloud environment so obviously natural to be at AWS. We help them do that on premise as well and we find a lot of our customers here. People we've been talking to for a long time. New people as well trying to move over. So it's a great environment for us. We kind of see what's going on in the innovation side from AWS too so... >> And so Mike, what do you make of the show in general. I'm just curious about your take, two day take. We've been here a couple of days. By now it's sunk in a little bit, I would think. >> It's incredible. You know, we were here last year and there was roughly 30,000 attendees. We're here again, there's over 45,000 attendees. I walk the floor here, I see dozens of providers that I didn't see here last year. Some of them are one guy with a six foot table, who's got some pretty interesting technology. So I would say that the pace with which this ecosystem is growing isn't showing any signs of slowing down. >> One of the things we've been hearing for a while is, boy, Amazon keeps innovating but it's been adding to complexity. The Amazon catalog itself, you know, 30,000 line items there. Then you add on top of that the marketplace and I'm like I'm pretty sure it's infinite at this point. And marketplace has been, you know, a huge driver in growth. I have to expect that marketplace impacted your business quite a bit in a good way, I would think. >> Yeah, for us that's probably one of the most exciting announcements that we've heard this week is around the marketplace specifically. We have a 30 year history selling software so to see now AWS taking what they did in the online retail business and in essence doing it for software in the marketplace is a good opportunity for us to connect our global catalog to their global catalog and provide a much wider range of software options for customers. >> And Mike, I wonder if I could... Can I ask you? Explain to our audience that might not have been in the keynote, gone to a break, what is it about this announcement that is so special from Amazon? So today they have about 4,000 or a little bit over 4,000 applications that sit in the marketplace and the idea is that any customer ought to be able to go up to the marketplace and with the click of a button, procure some software, have it put into a, maybe a reserved instance or some hosted infrastructure from Amazon and then be off to the races. No configuration, no support, no installation. >> Yeah, that was my tongue in cheek "so it's an infinite marketplace." >> It's an infinite-- >> As of today, right? Lawrence, one of the things we've been kind of wrestling with the last few years is this term, multi-Cloud. We hear an update from Andy every year as to how they look at it. Of course, I think it's, "everything is everything" is Amazon >> Right, right, yeah. and of course I think they've made an announcement about everything. And we still have a couple of days left. How are customers, you know, your customers looking at that multi Cloud, what are you seeing? Give us a little insight as to... and how does SoftwareONE help? >> Yeah, what's interesting for us is because that's what we see all the time, and we were talking about this a little bit beforehand and all of the customers that we have, we have tens of thousands of customers, we don't have any real conversations where people are talking about a single Cloud that they're working with, right. They're working with multiple. Maybe they're more in one today than another, but they're kind of looking at multiple solutions. So it's part of the business of what they do. And in a lot of cases it's just another extension of what's happening, you know, on premise today or in the data center. And so they always have to think about that. So, it's a part of it. There's lots of reasons for them to kind of go into that multi Cloud environment. Some of it might be for redundancy, some of it for flexibility on contracting. Some of it is for things like GDPR where they're worried about where their data sits and some of the local requirements. So it's part of the conversations that we always have and it's good to see some of the solutions that they're doing here where they think about that and they've really thought about how different ways customers can go in there and look at it, a really dispersed environment, so... >> So I mean how does the... In a multi Cloud world, I mean, how does that changing in terms of people optimizing their software, making these decisions that you're trying to steer, you're trying to advise them on. What's the impact that that's having kind of on what they're deciding to do going forward? >> There's a couple of things. First of all, I couldn't agree more with Lawrence. Tens of thousands of customers. I can't think of a time where I've talked to a customer that wasn't multi Cloud. So it's almost I would say all customers are multi Cloud. The challenge for our customers is how do you take all the different Cloud environments you're working with that use different vernacular, bring it into one system that then is using a common language to, you know, a common language around the resources that sit in those Cloud environments. A common language around how you map those resources to your organizational structure or how you manage your business. So that you're looking at your Cloud resources, you're planning for them financially but in the context of your business, using a common vernacular across all Cloud providers. That's a difficult thing to do if you're just going to a point solution from one of the publishers. >> And so why do that? I mean, why not simplify? Why not, why not just keep it in one environment or maybe two environment? Why branch out? Why take it to a different sphere? >> Well, I think, Lawrence touched on some of the reasons which is some of it's price Some of it is I've got legacy applications that don't work in one Cloud infrastructure but will work in another Cloud infrastructure. Some of it is security. Some of it is just simply I don't have any control over it. People are acquiring Cloud infrastructure via credit card within departments. You can't say, "You have to use Amazon "or you have to use Microsoft." You're not controlling that. >> You know we've talked before. SoftwareONE has a different legacy than say, what I'd say the Cloud management providers that are out there. But some of the things are very similar that you're attacking because everybody, I think, has identified this multi Cloud and it's the big elephant and everybody's trying to take bites out of it. Can you maybe give us a little bit of a compare contrast about... Think there's all these companies out there that start with Cloud and have one or two other words with them. How's SoftwareONE? What's similar and what's different? >> Yeah, I think when you look at that, there's certainly a lot of vendors here do a good job of starting to think about that. But a lot of vendors have started with the Cloud and kind of built around that and for companies that are born in the Cloud and that's all they focus on, that might be a great solution. But a lot of the enterprises that we deal with, our larger customers, that mix of what's on premise and what's in the Cloud, is still a minority is going to be in the Cloud. It is gonna be definitely less than 50 percent for a lot of the companies. Even smaller for some of the big places. So if you get really really good visibility, even in a multi Cloud environment, on only 20 or 30 percent of your environment, then you're not getting the whole big picture of what's happening, particularly on the expense side and where you're spending the money. So what we bring to the table that's a little bit unique here is we come from the history of doing this for many years on premise. We give a good visibility into what they're doing, how their assets are being utilized, giving them thoughts and contributions on how they price it and what they buy to it. So we give them that good view of on premise as well as what's happening in the Cloud. So now when they make decisions, they're getting that wholistic view. So marketing might come in there and they might have a software catalog that's part in Cloud and part of it's on premise. Their CRM might be on premise. So if they're looking at, What's my overall budget and spend there? How do I consolidate it? How do I make it better? You just can't look at the SAS applications and what's in the Cloud or what's in Amazon. You've really gotta get that full picture and that's what we can bring to bare. And the other thing is, a lot of the solutions that you see, a lot of the vendors are very focused on one particular country or environment. SoftwareONE is really spread out across the globe. We're in 80 plus countries. So when you're looking at okay, now I've gotta figure out who's buying whatever in Switzerland and the UK and the US, how do I simplify procurement for it? How do I get visibility across all of that? I'm prim in the Cloud. That becomes a much more complex question and those are the things that we can help enterprises with that's a little bit above and beyond what you might see on some of the kind of pure Cloud focused players. >> Mike, I had a interesting session I got to sit in with Amazon talking about how they are helping customers with innovation. And one of the things they put forth is, you know, companies have usually hundreds or thousands of applications but at least the premise they put forth is there's usually a handful of companies that are the strategic ones. There's maybe a next tier that are kind of important and then there's whole lots of other stuff. Maybe they're not all applications But they're putting out. But they're coming to Amazon and saying, "You're innovating. "You're moving fast. "How do we do that? "How do we help with the digital transformation?" How does SoftwareONE get involved in kind of the innovation, helping them along that journey? >> It's true that we work with over 9,000 publishers and I would say that the top seven or eight make up the lion's share of both our revenue and our customer's spend. However, if you take the what we call the tail end spend, if you take not just those top tier providers and not the middle tier but all of those small little applications they're using departmentally, they don't seem to add up to a lot when you look at them each department or each geography. When you bring them together for the enterprise, it's a large spend and it's very hard for our customers to get control of that. So when we talk about innovation, I would not suggest that you innovate just around the top five to 10 publishers. You have to be able to provide a cost management solution across the entire portfolio for customers, across the entire life cycle, that's on premises, that's multi Cloud, hybrid Cloud, and that's from acquisition through disposition and that includes the tail end spend. >> So is a lot of that when you give the visibility into the client as to what they're doing across the enterprise? They might not realize how deeply involved they are and that could give them leverage for pricing and then the lend. >> The best example is SAS Today, because SAS has enabled shadow IT in a way that we've never seen. Now people are just buying whatever application they want in their department with their credit card. Well, the IT department, the procurement department, the compliance group, they have no idea where that spend is coming from. They haven't discovered those unknown SAS subscriptions and you can't budget for it. You can't manage what you can't see. >> What's the shocker then? I mean, when you come in, just in general, to a business, what's the eye opening moment for them you think in terms of what you're uncovering or what you're showing them about their own process that you think they would know but you give them a little aha. >> Yeah, I think one example... we work with a lot of the companies, but one not that far from here. We work with LA Metro, the county and what they do there and on transportation. And they were looking at their environment and their spend in the Cloud and once we gave them our platform to kind of see what was going on, give them full visibility on what's happening with their VMs, what's happening in the Cloud, they saw that they're basically spending two to three X what they needed to before hand. Right before they really took a look, a good look at it. I think they were surprised by that, but it was a really good opportunity to take a good hard look at what VMs I might need to spend down, what ones do I need to throttle, what ones aren't being active, and again, you want to make sure you're right sized for what you're doing, instead of just over provisioning and kind of taking a guess at it if you will and overspending at the end of the day. >> Interestingly enough, just to add on to that, I was talking to a CIO that we work with maybe a month ago. We surveilled all of our customers on their spend. By and large, most of them know they're overspending on software. The estimates go anywhere between 25 and 35 percent, especially in the Cloud. But they know they're overspending. So some of them don't... they just can't solve the problem so they just budget for the overspend. So I was meeting with her and I said to her, "Do you know you're overspending?" She said, "Yeah, we're definitely overspending about 30%." I said, "What are you doing about that?" She says, "I just put it into my budget." I says, "That's crazy. Why wouldn't you want to solve that problem?" And the irony is that she's actually the CIO for a tax and accounting firm. (laughter) But they're just burying it. So, yeah. >> Yeah, that is funny. So, Andy Jassy did quite an extensive keynote. Lots of announcements. Kind of gave you the wild card. Something that jumped out at you, you wanna build on, any announcements or any pieces of Andy's presentation. >> Yeah, I think overall he kept talking about innovation and what they're doing. I think for us, we were talking a little bit about the marketplace. But you also see things they're doing on the database side. Are they strengthening aurora? Making post gray more attractive as an alternative to some of the legacy systems. And that complements well what we see as a migration and people wanna do, right. If you really strengthen the offering, it makes it more attractive and then certainly makes it interesting for the ecosystem which we're part of to help contribute that for people to move over there. I also really liked his analogy there. He kept going back to the musicians and kind of the relationship they have or thinking that compared to developers. Having all the instruments that you need to kind of build what you want and having that flexibility of choice and I think it's a great analogy. I think the flip side to that is: Hey, if you've got a jam session here and you're bringing in all of these different instruments: a guitar player, a tuba player, whatever it is to build this up, you also have to look at what do you have in house? You probably already have a pretty good size orchestra. Some of it might be legacy. Some people might be playing the harpsichord, the recorder, whatever it is. And you've got to figure out how does, In some cases, how do I blend that together? If I'm bringing them in, is it just for a jam session or do I need them here for a full set or for like a year long concert? And if you're not careful, you could end up spending a lot of money on bringing in all of these different players and I think that's an interesting way to think about this is it's great to have the flexibility but how do you make sure you have the visibility, the cost controls, so that doesn't overdo or overspend what you wanna do to get that creativity that you need. >> Everybody's gotta play in the same key too, right? Alright, let's all get on the same page Same page of music. >> So SoftwareONE is a global company. I know you both do a lot of travel. Give us kind of... We're here in the center of AWS. 43,000 devoted, super excited, passionate people but what are you seeing out there in the globe? Where does AWS specifically, Public Cloud in general, still need to push? What are some of the concerns, challenges, things that you see out there? >> Geographically? >> Yeah. >> Well, I mean certainly in APAC there's the challenge that any Cloud infrastructure provider is gonna face and that's Jolicloud. Jolicloud is a fairly dominant player in China and Japan, and they have a good presence throughout APAC. I think that AWS is making strong strides in Asia. Clearly the market leader worldwide. Everybody is chasing them. With over a thousand features and enhancements announced just within the last year, the pricing changes happening at such rapid fire that I think it's difficult for the other guys to keep up with them. So I'd say there's no question they're the market leader globally. Asia Pacific is probably their biggest challenge. >> Lawrence? >> I concur with a lot of that. I mean, we've seen in our own company a lot of activities starting in India and even in South America. Some good relationships with AWS. Some early adoption there. But yeah, I think those comments on APAC, I think you've got some good experiences there. >> Well, gentlemen, thanks for joining us. We appreciate the time here on theCUBE. Good luck with the rest of the show and I'll be kind of curious to see where this goes. The vibe is good, right? >> The vibe's unbelievable. >> Fantastic. >> And you've got a big runway so good luck with that. >> Thank you. >> Alright. >> Alright gentlemen. That's it for our coverage here on theCUBE. Today, day two here at Reinvent. Back with more live tomorrow morning 11:00 Pacific time. We'll be with you 2:00 on the East Coast and we'll see you right here. Until then have a good night. (upbeat music)