>>Oh, MABA. It so good. It, so, so. Oh my, to make my move a in nothing. Lose open. Let go. Here I go. Are ready to go. >>Ready. >>Ready for more? Are you >>Ready? Coverage of AWS Reinvent 22 continues in a moment.

(energetic music) >> Okay, we're back with Justin Borgman, CEO of Starburst, Richard Jarvis is the CTO of EMIS Health, and Teresa Tung is the cloud first technologist from Accenture. We're on to lie number three. And that is the claim that today's "Modern Data Stack" is actually modern. So (chuckles), I guess that's the lie. Or, is that it's not modern. Justin, what do you say? >> Yeah, I think new isn't modern. Right? I think it's the new data stack. It's the cloud data stack, but that doesn't necessarily mean it's modern. I think a lot of the components actually, are exactly the same as what we've had for 40 years. Rather than Teradata, you have Snowflake. Rather than Informatica, you have Fivetran. So, it's the same general stack, just, y'know, a cloud version of it. And I think a lot of the challenges that have plagued us for 40 years still maintain. >> So, let me come back to you Justin. Okay, but there are differences, right? You can scale. You can throw resources at the problem. You can separate compute from storage. You really, there's a lot of money being thrown at that by venture capitalists, and Snowflake you mentioned, its competitors. So that's different. Is it not? Is that not at least an aspect of modern dial it up, dial it down? So what do you say to that? >> Well, it is. It's certainly taking, y'know what the cloud offers and taking advantage of that. But it's important to note that the cloud data warehouses out there are really just separating their compute from their storage. So it's allowing them to scale up and down, but your data's still stored in a proprietary format. You're still locked in. You still have to ingest the data to get it even prepared for analysis. So a lot of the same structural constraints that exist with the old enterprise data warehouse model on-preem still exist. Just yes, a little bit more elastic now because the cloud offers that. >> So Teresa, let me go to you, 'cause you have cloud-first in your title. So, what's say you to this conversation? >> Well, even the cloud providers are looking towards more of a cloud continuum, right? So the centralized cloud as we know it, maybe data lake, data warehouse in the central place, that's not even how the cloud providers are looking at it. They have use query services. Every provider has one that really expands those queries to be beyond a single location. And if we look at a lot of where our- the future goes, right? That's going to very much fall the same thing. There was going to be more edge. There's going to be more on-premise, because of data sovereignty, data gravity, because you're working with different parts of the business that have already made major cloud investments in different cloud providers, right? So, there's a lot of reasons why the modern, I guess, the next modern generation of the data stack needs to be much more federated. >> Okay, so Richard, how do you deal with this? You've obviously got, you know, the technical debt, the existing infrastructure, it's on the books. You don't want to just throw it out. A lot of conversation about modernizing applications, which a lot of times is, you know, of microservices layer on top of legacy apps. How do you think about the Modern Data Stack? >> Well, I think probably the first thing to say is that the stack really has to include the processes and people around the data as well is all well and good changing the technology. But if you don't modernize how people use that technology, then you're not going to be able to, to scale because just 'cause you can scale CPU and storage doesn't mean you can get more people to use your data to generate you more value for the business. And so what we've been looking at is really changing in very much aligned to data products and, and data mesh. How do you enable more people to consume the service and have the stack respond in a way that keeps costs low? Because that's important for our customers consuming this data but also allows people to occasionally run enormous queries and then tick along with smaller ones when required. And it's a good job we did because during COVID all of a sudden we had enormous pressures on our data platform to answer really important life threatening queries. And if we couldn't scale both our data stack and our teams we wouldn't have been able to answer those as quickly as we had. So I think the stack needs to support a scalable business not just the technology itself. >> Well thank you for that. So Justin let's, let's try to break down what the critical aspects are of the modern data stack. So you think about the past, you know, five seven years cloud obviously has given a different pricing model. Derisked experimentation, you know that we talked about the ability to scale up scale down, but it's, I'm taking away that that's not enough. Based on what Richard just said, the modern data stack has to serve the business and enable the business to build data products. I buy that. I'm you a big fan of the data mesh concepts, even though we're early days. So what are the critical aspects if you had to think about you know, the, maybe putting some guardrails and definitions around the modern data stack, what does that look like? What are some of the attributes and, and principles there >> Of how it should look like or, or how >> Yeah. What it should be? >> Yeah. Yeah. Well, I think, you know, in, in Theresa mentioned this in in a previous segment about the data warehouse is not necessarily going to disappear. It just becomes one node, one element of the overall data mesh. And I certainly agree with that. So by no means, are we suggesting that, you know Snowflake or what Redshift or whatever cloud data warehouse you may be using is going to disappear, but it's it's not going to become the end all be all. It's not the, the central single source of truth. And I think that's the paradigm shift that needs to occur. And I think it's also worth noting that those who were the early adopters of the modern data stack were primarily digital, native born in the cloud young companies who had the benefit of of idealism. They had the benefit of starting with a clean slate that does not reflect the vast majority of enterprises. And even those companies, as they grow up, mature out of that ideal state, they go by a business. Now they've got something on another cloud provider that has a different data stack and they have to deal with that heterogeneity that is just change and change is a part of life. And so I think there is an element here that is almost philosophical. It's like, do you believe in an absolute ideal where I can just fit everything into one place or do I believe in reality? And I think the far more pragmatic approach is really what data mesh represents. So to answer your question directly, I think it's adding you know, the ability to access data that lives outside of the data warehouse, maybe living in open data formats in a data lake or accessing operational systems as well. Maybe you want to directly access data that lives in an Oracle database or a Mongo database or, or what have you. So creating that flexibility to really future proof yourself from the inevitable change that you will you won't encounter over time. >> So thank you. So Theresa, based on what Justin just said, I I might take away there is it's inclusive whether it's a data mart, data hub, data lake, data warehouse, just a node on the mesh. Okay. I get that. Does that include Theresa on, on Preem data? Obviously it has to. What are you seeing in terms of the ability to, to take that data mesh concept on Preem I mean most implementations I've seen and data mesh, frankly really aren't, you know adhering to the philosophy there. Maybe, maybe it's data lake and maybe it's using glue. You look at what JPMC is doing, HelloFresh, a lot of stuff happening on the AWS cloud in that, you know, closed stack, if you will. What's the answer to that Theresa? >> I mean, I think it's a killer case for data mesh. The fact that you have valuable data sources on Preem, and then yet you still want to modernize and take the best of cloud. Cloud is still, like we mentioned, there's a lot of great reasons for it around the economics and the way ability to tap into the innovation that the cloud providers are giving around data and AI architecture. It's an easy button. So the mesh allows you to have the best of both world. You can start using the data products on Preem, or in the existing systems that are working already. It's meaningful for the business. At the same time, you can modernize the ones that make business sense because it needs better performance. It needs, you know, something that is, is cheaper or or maybe just tapping into better analytics to get better insights, right? So you're going to be able to stretch and really have the best of both worlds. That, again, going back to Richard's point, that is meaningful by the business. Not everything has to have that one size fits all set a tool. >> Okay. Thank you. So Richard, you know, talking about data as product wonder if we could give us your perspectives here what are the advantages of treating data as a product? What, what role do data products have in the modern data stack? We talk about monetizing data. What are your thoughts on data products? >> So for us, one of the most important data products that we've been creating is taking data that is healthcare data across a wide variety of different settings. So information about patients, demographics about their their treatment, about their medications and so on, and taking that into a standards format that can be utilized by a wide variety of different researchers because misinterpreting that data or having the data not presented in the way that the user is expecting means that you generate the wrong insight and in any business that's clearly not a desirable outcome but when that insight is so critical as it might be in healthcare or some security settings you really have to have gone to the trouble of understanding the data, presenting it in a format that everyone can clearly agree on. And then letting people consume in a very structured managed way, even if that data comes from a variety of different sources in the first place. And so our data product journey has really begun by standardizing data across a number of different silos through the data mesh. So we can present out both internally and through the right governance externally to, to researchers. >> So that data product through whatever APIs is is accessible, it's discoverable, but it's obviously got to be governed as well. You mentioned appropriately provided to internally. >> Yeah. >> But also, you know, external folks as well. So the, so you've, you've architected that capability today? >> We have and because the data is standard it can generate value much more quickly and we can be sure of the security and value that that's providing, because the data product isn't just about formatting the data into the correct tables, it's understanding what it means to redact the data or to remove certain rows from it or to interpret what a date actually means. Is it the start of the contract or the start of the treatment or the date of birth of a patient? These things can be lost in the data storage without having the proper product management around the data to say in a very clear business context what does this data mean, and what does it mean to process this data for a particular use case. >> Yeah, it makes sense. It's got the context. If the, if the domains on the data, you know you got to cut through a lot of the, the centralized teams, the technical teams that that data agnostic, they don't really have that context. All right, let's end. Justin. How does Starburst fit into this modern data stack? Bring us home. >> Yeah. So I think for us it's really providing our customers with, you know the flexibility to operate and analyze data that lives in a wide variety of different systems. Ultimately giving them that optionality, you know and optionality provides the ability to reduce costs store more in a data lake rather than data warehouse. It provides the ability for the fastest time to insight to access the data directly where it lives. And ultimately with this concept of data products that we've now, you know incorporated into our offering as well you can really create and, and curate, you know data as a product to be shared and consumed. So we're trying to help enable the data mesh, you know model and make that an appropriate compliment to you know, the modern data stack that people have today. >> Excellent. Hey, I want to thank Justin, Teresa, and Richard for joining us today. You guys are great. Big believers in the in the data mesh concept, and I think, you know we're seeing the future of data architecture. So thank you. Now, remember, all these conversations are going to be available on the cube.net for on demand viewing. You can also go to starburst.io. They have some great content on the website and they host some really thought provoking interviews and they have awesome resources. Lots of data mesh conversations over there and really good stuff in, in the resource section. So check that out. Thanks for watching the "Data Doesn't Lie... or Does It?" made possible by Starburst data. This is Dave Vellante for the Cube, and we'll see you next time. (upbeat music)

>>All right. Let's kick things off. I'm Lisa Martin with Richard Hummel manager of threat intelligence at NetScout. We're going to be talking about the vertical industries where attackers really zeroed in for DDoSs attacks. Richard. This is some interesting findings in the second half of 20 21, 20 21. >>It is in it's unfortunate because I never liked to see individuals or organizations specifically targeted by DDoS attacks and often this kind of individualistic targeting isn't so individual. And what I mean by that is DDoS attacks. Almost always have some form of ripple effects. It collateral damage that extends far beyond who the adversary is going after. We've got an example of this. There's there's been a lot of reports recently about, uh, various void providers, um, starting in Eastern Europe and expanding even to north America and various other parts of the world that have reported this DDoS extortion campaign or crew or whoever it might be copycatting as our eval, which is a notable ransomware group that all those publicly no, no that they were successful. Well, these guys are, are copycatting that unfortunately they've been very successful in some of these attacks and some of the companies have gone on record saying that, look, this didn't just impact us. >>They didn't just take our services offline. None of our customers could make calls. They could not do the reputation damage alone. How many of you users or subscribers did they lose as a result of them not being able to meet phone calls, how much revenue loss during that time period that they're losing out on. I go back all the way back to last year, and we saw something similar with another DDoSs extortion campaign against the New Zealand stock exchange. It was down for almost four days. Just think about the sheer amount of revenue loss and just all of the things that domino effect from there, right? It's not just the exchange commission that had problems. It's not just them. It's all of the stockholders and anybody that couldn't make a trade. And so yes, adversaries absolutely single out organizations, but the damage that it causes to those around them can be astronomical. >>Right? The downstream effects are just go, as you said, the ripple effect just goes on and on. And on. One of the things that I found interesting in the second half of 2021 threat intelligence report was that telecommunications verticals, which are usually a popular target for attackers actually saw fewer attacks in second half. Why is that? What are your thoughts there? >>So I think a lot of this goes back to why we saw a decrease in the second half of the year. Yeah. That decrease is almost exclusively attributed to a decrease in DNS amplification in CLD, DNS being like the predominant, uh, the us attack factor for many, many years, uh, TCP attacks, these direct path attacks that we talked about in our last segment, where they are direct from button ads or they're source from high powered, we're seeing a rebalancing, the scales here. So we're seeing about equal parts of both of these kinds of attacks now versus the reflection amplification that the amplification stuff being predominant. Um, and so that's one of the reasons why we saw that decrease. And when we look at the telecommunications and wired it and wireless, um, these are your consumers. These are your gamers. These are just individuals sitting at home, minding their own business that are getting DDoS attack. >>Then we've talked about it on previous interviews that we've done, that gamers are predominantly the targets of DDoS attacks. And so if we're seeing a decrease in like the preferred method for these attacks to occur, naturally, we're going to see a decrease in some of the attacks against these consumers. But what's notable here in, in telecommunication is considered like this big umbrella, right? You have wired, you have wireless, you have mobile, you have satellite, you have all other telecommunications, which is where your work providers fall. Um, so most of them we saw decreases, but in wireless and all other telecommunications now wireless, remember this 5g advent, and then the other telecommunications with this digital extortion stuff against the void providers. Those are two areas where we saw increases wireless, saw 32% increase and all other telecommunications think void saw at 93% increase. And so we are seeing some increases here, but the higher kind of frequency or attack counts in the wired, um, in a mobile, those saw decreases. >>Let's talk about 5g where, you know, everybody is so excited about it. The adoption is coming. What's going to be the amplification implication of 5g in terms of feeling increased attacks, >>Just the sheer volume. I mean, when we start to introduce 5g, now we're talking about every single device that we have potentially having its own space on the internet. You may have high bandwidth, high throughput capacity. And so we're not talking about just in the home, right. 5g is going to be everywhere. So now just take all of your IOT devices that maybe either be isolated to your home network are now going to be across the entire globe, outside the home on 5g networks that have the capability to launch really fast, really, really potent attacks. And so just the, the footprint really of what we've got to think about from a security perspective and from a defense perspective, it's going to flip things on its head quite a bit, because you're going from here's everything that I'm going to secure inside. My let's just use the castle representation again, everything's inside the castle. >>I put my boundaries in place. I've got my firewalls. I've got my IDs is I've got my access control lists. So anything outside of my sphere or my domain is irrelevant because I don't care about it. Well, 5g is going to blast that away because not only do you not have it on prem anymore, everything starts to get its own direct connection to the internet. How do you secure 5g? Does your organization have that in practice? I mean, ISP is, are still rolling 5g out and are still trying to figure things out as they go, how much more do enterprises and others that are gonna be consumers of this need to figure out how we're going to secure against these. And so, yeah, it's gonna introduce a whole new realm of how we need to think about security, >>A whole new realm, lots to consider there. Another thing besides the wireless telecommunications that that report uncovered was that closely related related software and computer and manufacturing verticals also saw massive increases in attacks. Why talk to us about that? >>You know, I think it's a logical progression of attacks. Um, the last report we put out, we talked about the conduct of the supply chain and what we meant by that was how do we communicate? How do we talk to each other? How do we get into our work, uh, assets? We use a VPN, we use DNS servers. We use internet exchanges to resolve our websites, adversaries increase the tax against those. And that was kind of like the connectivity piece. Well, let's, let's take a step back. What do you need to be able to get online? You need a computer, you need software and you need the ability to store some of this data on your computer. So what we saw 606% increase in attacks against software publishers, 260 and 253% increases against computer manufacturing, computer storage manufacturing together. To me, these are the digital supply chain. >>These are the things that allow us to do what we need to do. And so it's almost like a natural progression. And we see this a lot with DDoS extortion. So take the Lazarus Paramatta guys. We talked about last time, you know, they've initially started against financial organizations going after banks. Then they moved to the stock markets and they moved to insurance brokerages accounts. They moved to travel exchanges, currency exchanges, and they started this domino effect that, you know what, let's go where the money is. Maybe we'll get a payday that didn't work so slowly. They started to expand. Eventually LBA started targeting anybody and everybody in every single industry in vertical. And so what we see here is kind of like a logical progression of, you know, what our, to the supply chain attacks didn't work. And there's a good reason for them because these devices that they're going after are usually very, very secure. And so DDoS attacks, they can absorb them. They can mitigate them that they just bounce off. So can we succeed by going a little bit more upstream or downstream? However you want to look at this by targeting the actual manufacturers themselves, the people who create the software, we need to be able to conduct our business. And so that's kind of the logical progression of what we're seeing here. >>So Richard, how can companies prepare, defend against the attacks against the digital supply chain? That's pretty critical >>Preparation. Preparation is the key. And if we follow best current or best industry practices, um, there's this 80 20 rule that a colleague of mine likes to use. If you do 80% of the recommended things, the best current practices, it solves 80% of your problem and not just for the DDoS problem, but also for things like ransomware, various other, it's really that 20% in there that you have to worry about. And that's going to be being aware, knowing the adversaries are actually going after software publishers who would have funk it. Right. Who would think that they're actually going after the manufacturer of the applications that I'm using to talk to you right now, Lisa, right. Yeah. And so these are the kinds of things that people just aren't always aware of. And so making sure that we're cognizant of the actual targets of these attacks, and then from there, figuring out is there is our business involved in any kind of, of the software publishing. >>Should we be concerned about that? And if not, what about where we're getting our software from? Do they have to worry about this? Is there a risk for them not being able to deliver something to us because they're under bombardment by detail sometimes. And so it's just being aware and taking steps to be able to handle prepare. And I will say it again. You must have some sort of DDoS protections in place. It's not when, or it's not, if it's when you're going to get attacked and everybody, even if you are not the direct target, there's collateral damage as we talked about in the last segment. >>Yep. It's a matter of, if not, when, and that's something that businesses of any size in any industry have to be prepared for, as you said, preparation is really number one. Richard, thank you for sharing some of the really interesting findings and the verticals that have saw massive increases in the second half of 2021. And we look forward to what you're going to uncover next. >>Absolutely. Thanks again for having me. It's been a pleasure. >>Likewise. We want to thank you for watching the program today. Remember all these videos are available@thecubedotnetandyoucancheckoutthenewsfromtodayonsiliconangledotcomandofcoursenetscout.com many thanks to NetScout for making this program possible and sponsoring the cube. This is Lisa Martin signing off. Thanks for watching and bye for now.

>>Welcome to this exclusive discussion. IBM, the next three years of life sciences, innovation, precision medicine, advanced clinical data management and beyond. My name is Dave Volante from the Cuban today, we're going to take a deep dive into some of the most important trends impacting the life sciences industry in the next 60 minutes. Yeah, of course. We're going to hear how IBM is utilizing Watson and some really important in life impacting ways, but we'll also bring in real world perspectives from industry and the independent analyst view to better understand how technology and data are changing the nature of precision medicine. Now, the pandemic has created a new reality for everyone, but especially for life sciences companies, one where digital transformation is no longer an option, but a necessity. Now the upside is the events of the past 22 months have presented an accelerated opportunity for innovation technology and real world data are coming together and being applied to support life science, industry trends and improve drug discovery, clinical development, and treatment commercialization throughout the product life cycle cycle. Now I'd like to introduce our esteemed panel. Let me first introduce Lorraine Marshawn, who is general manager of life sciences at IBM Watson health. Lorraine leads the organization dedicated to improving clinical development research, showing greater treatment value in getting treatments to patients faster with differentiated solutions. Welcome Lorraine. Great to see you. >>Dr. Namita LeMay is the research vice-president of IDC, where she leads the life sciences R and D strategy and technology program, which provides research based advisory and consulting services as well as market analysis. The loan to meta thanks for joining us today. And our third panelist is Greg Cunningham. Who's the director of the RWE center of excellence at Eli Lilly and company. Welcome, Greg, you guys are doing some great work. Thanks for being here. Thanks >>Dave. >>Now today's panelists are very passionate about their work. If you'd like to ask them a question, please add it to the chat box located near the bottom of your screen, and we'll do our best to answer them all at the end of the panel. Let's get started. Okay, Greg, and then Lorraine and meta feel free to chime in after one of the game-changers that you're seeing, which are advancing precision medicine. And how do you see this evolving in 2022 and into the next decade? >>I'll give my answer from a life science research perspective. The game changer I see in advancing precision medicine is moving from doing research using kind of a single gene mutation or kind of a single to look at to doing this research using combinations of genes and the potential that this brings is to bring better drug targets forward, but also get the best product to a patient faster. Um, I can give, uh, an example how I see it playing out in the last decade. Non-oncology real-world evidence. We've seen an evolution in precision medicine as we've built out the patient record. Um, as we've done that, uh, the marketplace has evolved rapidly, uh, with, particularly for electronic medical record data and genomic data. And we were pretty happy to get our hands on electronic medical record data in the early days. And then later the genetic test results were combined with this data and we could do research looking at a single mutation leading to better patient outcomes. But I think where we're going to evolve in 2022 and beyond is with genetic testing, growing and oncology, providing us more data about that patient. More genes to look at, uh, researchers can look at groups of genes to analyze, to look at that complex combination of gene mutations. And I think it'll open the door for things like using artificial intelligence to help researchers plow through the complex number of permutations. When you think about all those genes you can look at in combination, right? Lorraine yes. Data and machine intelligence coming together, anything you would add. >>Yeah. Thank you very much. Well, I think that Greg's response really sets us up nicely, particularly when we think about the ability to utilize real-world data in the farm industry across a number of use cases from discovery to development to commercial, and, you know, in particular, I think with real world data and the comments that Greg just made about clinical EMR data linked with genetic or genomic data, a real area of interest in one that, uh, Watson health in particular is focused on the idea of being able to create a data exchange so that we can bring together claims clinical EMR data, genomics data, increasingly wearables and data directly from patients in order to create a digital health record that we like to call an intelligent patient health record that basically gives us the digital equivalent of a real life patient. And these can be used in use cases in randomized controlled clinical trials for synthetic control arms or natural history. They can be used in order to track patients' response to drugs and look at outcomes after they've been on various therapies as, as Greg is speaking to. And so I think that, you know, the promise of data and technology, the AI that we can apply on that is really helping us advance, getting therapies to market faster, with better information, lower sample sizes, and just a much more efficient way to do drug development and to track and monitor outcomes in patients. >>Great. Thank you for that now to meta, when I joined IDC many, many years ago, I really didn't know much about the industry that I was covering, but it's great to see you as a former practitioner now bringing in your views. What do you see as the big game-changers? >>So, um, I would, I would agree with what both Lorraine and Greg said. Um, but one thing that I'd just like to call out is that, you know, everyone's talking about big data, the volume of data is growing. It's growing exponentially actually about, I think 30% of data that exists today is healthcare data. And it's growing at a rate of 36%. That's huge, but then it's not just about the big, it's also about the broad, I think, um, you know, I think great points that, uh, Lorraine and Greg brought out that it's, it's not just specifically genomic data, it's multi omic data. And it's also about things like medical history, social determinants of health, behavioral data. Um, and why, because when you're talking about precision medicine and we know that we moved away from the, the terminology of personalized to position, because you want to talk about disease stratification and you can, it's really about convergence. >>Um, if you look at a recent JAMA paper in 2021, only 1% of EHS actually included genomic data. So you really need to have that ability to look at data holistically and IDC prediction is seeing that investments in AI to fuel in silico, silicone drug discovery will double by 20, 24, but how are you actually going to integrate all the different types of data? Just look at, for example, diabetes, you're on type two diabetes, 40 to 70% of it is genetically inherited and you have over 500 different, uh, genetic low side, which could be involved in playing into causing diabetes. So the earlier strategy, when you are looking at, you know, genetic risk scoring was really single trait. Now it's transitioning to multi rate. And when you say multi trade, you really need to get that integrated view that converging for you to, to be able to drive a precision medicine strategy. So to me, it's a very interesting contrast on one side, you're really trying to make it specific and focused towards an individual. And on the other side, you really have to go wider and bigger as well. >>Uh, great. I mean, the technology is enabling that convergence and the conditions are almost mandating it. Let's talk about some more about data that the data exchange and building an intelligent health record, as it relates to precision medicine, how will the interoperability of real-world data, you know, create that more cohesive picture for the, for the patient maybe Greg, you want to start, or anybody else wants to chime in? >>I think, um, the, the exciting thing from, from my perspective is the potential to gain access to data. You may be weren't aware of an exchange in implies that, uh, some kind of cataloging, so I can see, uh, maybe things that might, I just had no idea and, uh, bringing my own data and maybe linking data. These are concepts that I think are starting to take off in our field, but it, it really opens up those avenues to when you, you were talking about data, the robustness and richness volume isn't, uh, the only thing is Namita said, I think really getting to a rich high-quality data and, and an exchange offers a far bigger, uh, range for all of us to, to use, to get our work done. >>Yeah. And I think, um, just to chime, chime into that, uh, response from Greg, you know, what we hear increasingly, and it's pretty pervasive across the industry right now, because this ability to create an exchange or the intelligent, uh, patient health record, these are new ideas, you know, they're still rather nascent and it always is the operating model. Uh, that, that is the, uh, the difficult challenge here. And certainly that is the case. So we do have data in various silos. Uh, they're in patient claims, they're in electronic medical records, they might be in labs, images, genetic files on your smartphone. And so one of the challenges with this interoperability is being able to tap into these various sources of data, trying to identify quality data, as Greg has said, and the meta is underscoring as well. Uh, we've gotta be able to get to the depth of data that's really meaningful to us, but then we have to have technology that allows us to pull this data together. >>First of all, it has to be de-identified because of security and patient related needs. And then we've gotta be able to link it so that you can create that likeness in terms of the record, it has to be what we call cleaned or curated so that you get the noise and all the missing this out of it, that's a big step. And then it needs to be enriched, which means that the various components that are going to be meaningful, you know, again, are brought together so that you can create that cohort of patients, that individual patient record that now is useful in so many instances across farm, again, from development, all the way through commercial. So the idea of this exchange is to enable that exact process that I just described to have a, a place, a platform where various entities can bring their data in order to have it linked and integrated and cleaned and enriched so that they get something that is a package like a data package that they can actually use. >>And it's easy to plug into their, into their studies or into their use cases. And I think a really important component of this is that it's gotta be a place where various third parties can feel comfortable bringing their data together in order to match it with other third parties. That is a, a real value, uh, that the industry is increasingly saying would be important to them is, is the ability to bring in those third-party data sets and be able to link them and create these, these various data products. So that's really the idea of the data exchange is that you can benefit from accessing data, as Greg mentioned in catalogs that maybe are across these various silos so that you can do the kind of work that you need. And that we take a lot of the hard work out of it. I like to give an example. >>We spoke with one of our clients at one of the large pharma companies. And, uh, I think he expressed it very well. He said, what I'd like to do is have like a complete dataset of lupus. Lupus is an autoimmune condition. And I've just like to have like the quintessential lupus dataset that I can use to run any number of use cases across it. You know, whether it's looking at my phase one trial, whether it's selecting patients and enriching for later stage trials, whether it's understanding patient responses to different therapies as I designed my studies. And so, you know, this idea of adding in therapeutic area indication, specific data sets and being able to create that for the industry in the meta mentioned, being able to do that, for example, in diabetes, that's how pharma clients need to have their needs met is through taking the hard workout, bringing the data together, having it very therapeutically enriched so that they can use it very easily. >>Thank you for that detail and the meta. I mean, you can't do this with humans at scale in technology of all the things that Lorraine was talking about, the enrichment, the provenance, the quality, and of course, it's got to be governed. You've got to protect the privacy privacy humans just can't do all that at massive scale. Can it really tech that's where technology comes in? Doesn't it and automation. >>Absolutely. >>I, couldn't more, I think the biggest, you know, whether you talk about precision medicine or you talk about decentralized trials, I think there's been a lot of hype around these terms, but what is really important to remember is technology is the game changer and bringing all that data together is really going to be the key enabler. So multimodal data integration, looking at things like security or federated learning, or also when you're talking about leveraging AI, you're not talking about things like bias or other aspects around that are, are critical components that need to be addressed. I think the industry is, uh, it's partly, still trying to figure out the right use cases. So it's one part is getting together the data, but also getting together the right data. Um, I think data interoperability is going to be the absolute game changer for enabling this. Uh, but yes, um, absolutely. I can, I can really couldn't agree more with what Lorraine just said, that it's bringing all those different aspects of data together to really drive that precision medicine strategy. >>Excellent. Hey Greg, let's talk about protocols decentralized clinical trials. You know, they're not new to life silences, but, but the adoption of DCTs is of course sped up due to the pandemic we've had to make trade-offs obviously, and the risk is clearly worth it, but you're going to continue to be a primary approach as we enter 2022. What are the opportunities that you see to improve? How DCTs are designed and executed? >>I see a couple opportunities to improve in this area. The first is, uh, back to technology. The infrastructure around clinical trials has, has evolved over the years. Uh, but now you're talking about moving away from kind of site focus to the patient focus. Uh, so with that, you have to build out a new set of tools that would help. So for example, one would be novel trial, recruitment, and screening, you know, how do you, how do you find patients and how do you screen them to see if are they, are they really a fit for, for this protocol? Another example, uh, very important documents that we have to get is, uh, you know, the e-consent that someone's says, yes, I'm, well, I understand this study and I'm willing to do it, have to do that in a more remote way than, than we've done in the past. >>Um, the exciting area, I think, is the use of, uh, eco, uh, E-Pro where we capture data from the patient using apps, devices, sensors. And I think all of these capabilities will bring a new way of, of getting data faster, uh, in, in this kind of model. But the exciting thing from, uh, our perspective at Lily is it's going to bring more data about the patient from the patient, not just from the healthcare provider side, it's going to bring real data from these apps, devices and sensors. The second thing I think is using real-world data to identify patients, to also improve protocols. We run scenarios today, looking at what's the impact. If you change a cut point on a, a lab or a biomarker to see how that would affect, uh, potential enrollment of patients. So it, it definitely the real-world data can be used to, to make decisions, you know, how you improve these protocols. >>But the thing that we've been at the challenge we've been after that this probably offers the biggest is using real-world data to identify patients as we move away from large academic centers that we've used for years as our sites. Um, you can maybe get more patients who are from the rural areas of our countries or not near these large, uh, uh, academic centers. And we think it'll bring a little more diversity to the population, uh, who who's, uh, eligible, but also we have their data, so we can see if they really fit the criteria and the probability they are a fit for the trial is much higher than >>Right. Lorraine. I mean, your clients must be really pushing you to help them improve DCTs what are you seeing in the field? >>Yes, in fact, we just attended the inaugural meeting of the de-central trials research Alliance in, uh, in Boston about two weeks ago where, uh, all of the industry came together, pharma companies, uh, consulting vendors, just everyone who's been in this industry working to help define de-central trials and, um, think through what its potential is. Think through various models in order to enable it, because again, a nascent concept that I think COVID has spurred into action. Um, but it is important to take a look at the definition of DCT. I think there are those entities that describe it as accessing data directly from the patient. I think that is a component of it, but I think it's much broader than that. To me, it's about really looking at workflows and processes of bringing data in from various remote locations and enabling the whole ecosystem to work much more effectively along the data continuum. >>So a DCT is all around being able to make a site more effective, whether it's being able to administer a tele visit or the way that they're getting data into the electronic data captures. So I think we have to take a look at the, the workflows and the operating models for enabling de-central trials and a lot of what we're doing with our own technology. Greg mentioned the idea of electronic consent of being able to do electronic patient reported outcomes, other collection of data directly from the patient wearables tele-health. So these are all data acquisition, methodologies, and technologies that, that we are enabling in order to get the best of the data into the electronic data capture system. So edit can be put together and processed and submitted to the FDA for regulatory use for clinical trial type submission. So we're working on that. I think the other thing that's happening is the ability to be much more flexible and be able to have more cloud-based storage allows you to be much more inter-operable to allow API APIs in order to bring in the various types of data. >>So we're really looking at technology that can make us much more fluid and flexible and accommodating to all the ways that people live and work and manage their health, because we have to reflect that in the way we collect those data types. So that's a lot of what we're, what we're focused on. And in talking with our clients, we spend also a lot of time trying to understand along the, let's say de-central clinical trials continuum, you know, w where are they? And I know Namita is going to talk a little bit about research that they've done in terms of that adoption curve, but because COVID sort of forced us into being able to collect data in more remote fashion in order to allow some of these clinical trials to continue during COVID when a lot of them had to stop. What we want to make sure is that we understand and can codify some of those best practices and that we can help our clients enable that because the worst thing that would happen would be to have made some of that progress in that direction. >>But then when COVID is over to go back to the old ways of doing things and not bring some of those best practices forward, and we actually hear from some of our clients in the pharma industry, that they worry about that as well, because we don't yet have a system for operationalizing a de-central trial. And so we really have to think about the protocol it's designed, the indication, the types of patients, what makes sense to decentralize, what makes sense to still continue to collect data in a more traditional fashion. So we're spending a lot of time advising and consulting with our patients, as well as, I mean, with our clients, as well as CRS, um, on what the best model is in terms of their, their portfolio of studies. And I think that's a really important aspect of trying to accelerate the adoption is making sure that what we're doing is fit for purpose, just because you can use technology doesn't mean you should, it really still does require human beings to think about the problem and solve them in a very practical way. >>Great, thank you for that. Lorraine. I want to pick up on some things that Lorraine was just saying. And then back to what Greg was saying about, uh, uh, DCTs becoming more patient centric, you had a prediction or IDC, did I presume your fingerprints were on it? Uh, that by 20 25, 70 5% of trials will be patient-centric decentralized clinical trials, 90% will be hybrid. So maybe you could help us understand that relationship and what types of innovations are going to be needed to support that evolution of DCT. >>Thanks, Dave. Yeah. Um, you know, sorry, I, I certainly believe that, uh, you know, uh, Lorraine was pointing out of bringing up a very important point. It's about being able to continue what you have learned in over the past two years, I feel this, you know, it was not really a digital revolution. It was an attitude. The revolution that this industry underwent, um, technology existed just as clinical trials exist as drugs exist, but there was a proof of concept that technology works that this model is working. So I think that what, for example, telehealth, um, did for, for healthcare, you know, transition from, from care, anywhere care, anytime, anywhere, and even becoming predictive. That's what the decentralized clinical trials model is doing for clinical trials today. Great points again, that you have to really look at where it's being applied. You just can't randomly apply it across clinical trials. >>And this is where the industry is maturing the complexity. Um, you know, some people think decentralized trials are very simple. You just go and implement these centralized clinical trials, but it's not that simple as it it's being able to define, which are the right technologies for that specific, um, therapeutic area for that specific phase of the study. It's being also a very important point is bringing in the patient's voice into the process. Hey, I had my first telehealth visit sometime last year and I was absolutely thrilled about it. I said, no time wasted. I mean, everything's done in half an hour, but not all patients want that. Some want to consider going back and you, again, need to customize your de-centralized trials model to, to the, to the type of patient population, the demographics that you're dealing with. So there are multiple factors. Um, also stepping back, you know, Lorraine mentioned they're consulting with, uh, with their clients, advising them. >>And I think a lot of, um, a lot of companies are still evolving in their maturity in DCTs though. There's a lot of boys about it. Not everyone is very mature in it. So it's, I think it, one thing everyone's kind of agreeing with is yes, we want to do it, but it's really about how do we go about it? How do we make this a flexible and scalable modern model? How do we integrate the patient's voice into the process? What are the KPIs that we define the key performance indicators that we define? Do we have a playbook to implement this model to make it a scalable model? And, you know, finally, I think what organizations really need to look at is kind of developing a de-centralized mature maturity scoring model, so that I assess where I am today and use that playbook to define, how am I going to move down the line to me reach the next level of maturity. Those were some of my thoughts. Right? >>Excellent. And now remember you, if you have any questions, use the chat box below to submit those questions. We have some questions coming in from the audience. >>At one point to that, I think one common thread between the earlier discussion around precision medicine and around decentralized trials really is data interoperability. It is going to be a big game changer to, to enable both of these pieces. Sorry. Thanks, Dave. >>Yeah. Thank you. Yeah. So again, put your questions in the chat box. I'm actually going to go to one of the questions from the audience. I get some other questions as well, but when you think about all the new data types that are coming in from social media, omics wearables. So the question is with greater access to these new types of data, what trends are you seeing from pharma device as far as developing capabilities to effectively manage and analyze these novel data types? Is there anything that you guys are seeing, um, that you can share in terms of best practice or advice >>I'll offer up? One thing, I think the interoperability isn't quite there today. So, so what's that mean you can take some of those data sources. You mentioned, uh, some Omix data with, uh, some health claims data and it's the, we spend too much time and in our space putting data to gather the behind the scenes, I think the stat is 80% of the time is assembling the data 20% analyzing. And we've had conversations here at Lilly about how do we get to 80% of the time is doing analysis. And it really requires us to think, take a step back and think about when you create a, uh, a health record, you really have to be, have the same plugins so that, you know, data can be put together very easily, like Lorraine mentioned earlier. And that comes back to investing in as an industry and standards so that, you know, you have some of data standard, we all can agree upon. And then those plugs get a lot easier and we can spend our time figuring out how to make, uh, people's lives better with healthcare analysis versus putting data together, which is not a lot of fun behind the scenes. >>Other thoughts on, um, on, on how to take advantage of sort of novel data coming from things like devices in the nose that you guys are seeing. >>I could jump in there on your end. Did you want to go ahead? Okay. So, uh, I mean, I think there's huge value that's being seen, uh, in leveraging those multiple data types. I think one area you're seeing is the growth of prescription digital therapeutics and, um, using those to support, uh, you know, things like behavioral health issues and a lot of other critical conditions it's really taking you again, it is interlinking real-world data cause it's really taking you to the patient's home. Um, and it's, it's, there's a lot of patients in the city out here cause you can really monitor the patient real-time um, without the patient having coming, you know, coming and doing a site visit once in say four weeks or six weeks. So, um, I, and, uh, for example, uh, suicidal behavior and just to take an example, if you can predict well in advance, based on those behavioral parameters, that this is likely to trigger that, uh, the value of it is enormous. Um, again, I think, uh, Greg made a valid point about the industry still trying to deal with resolving the data interoperability issue. And there are so many players that are coming in the industry right now. There are really few that have the maturity and the capability to address these challenges and provide intelligence solutions. >>Yeah. Maybe I'll just, uh, go ahead and, uh, and chime into Nikita's last comment there. I think that's what we're seeing as well. And it's very common, you know, from an innovation standpoint that you have, uh, a nascent industry or a nascent innovation sort of situation that we have right now where it's very fragmented. You have a lot of small players, you have some larger entrenched players that have the capability, um, to help to solve the interoperability challenge, the standards challenge. I mean, I think IBM Watson health is certainly one of the entities that has that ability and is taking a stand in the industry, uh, in order to, to help lead in that way. Others are too. And, uh, but with, with all of the small companies that are trying to find interesting and creative ways to gather that data, it does create a very fragmented, uh, type of environment and ecosystem that we're in. >>And I think as we mature, as we do come forward with the KPIs, the operating models, um, because you know, the devil's in the detail in terms of the operating models, it's really exciting to talk these trends and think about the future state. But as Greg pointed out, if you're spending 80% of your time just under the hood, you know, trying to get the engine, all the spark plugs to line up, um, that's, that's just hard grunt work that has to be done. So I think that's where we need to be focused. And I think bringing all the data in from these disparate tools, you know, that's fine, we need, uh, a platform or the API APIs that can enable that. But I think as we, as we progress, we'll see more consolidation, uh, more standards coming into play, solving the interoperability types of challenges. >>And, um, so I think that's where we should, we should focus on what it's going to take and in three years to really codify this and make it, so it's a, it's a well hum humming machine. And, you know, I do know having also been in pharma that, uh, there's a very pilot oriented approach to this thing, which I think is really healthy. I think large pharma companies tend to place a lot of bets with different programs on different tools and technologies, to some extent to see what's gonna stick and, you know, kind of with an innovation mindset. And I think that's good. I think that's kind of part of the process of figuring out what is going to work and, and helping us when we get to that point of consolidating our model and the technologies going forward. So I think all of the efforts today are definitely driving us to something that feels much more codified in the next three to five years. >>Excellent. We have another question from the audience it's sort of related to the theme of this discussion, given the FDA's recent guidance on using claims and electronic health records, data to support regulatory decision-making what advancements do you think we can expect with regards to regulatory use of real-world data in the coming years? It's kind of a two-parter so maybe you guys can collaborate on this one. What role that, and then what role do you think industry plays in influencing innovation within the regulatory space? >>All right. Well, it looks like you've stumped the panel there. Uh, Dave, >>It's okay to take some time to think about it, right? You want me to repeat it? You guys, >>I, you know, I I'm sure that the group is going to chime into this. I, so the FDA has issued a guidance. Um, it's just, it's, it's exactly that the FDA issues guidances and says that, you know, it's aware and supportive of the fact that we need to be using real-world data. We need to create the interoperability, the standards, the ways to make sure that we can include it in regulatory submissions and the like, um, and, and I sort of think about it akin to the critical path initiative, probably, I don't know, 10 or 12 years ago in pharma, uh, when the FDA also embrace this idea of the critical path and being able to allow more in silico modeling of clinical trial, design and development. And it really took the industry a good 10 years, um, you know, before they were able to actually adopt and apply and take that sort of guidance or openness from the FDA and actually apply it in a way that started to influence the way clinical trials were designed or the in silico modeling. >>So I think the second part of the question is really important because while I think the FDA is saying, yes, we recognize it's important. Uh, we want to be able to encourage and support it. You know, when you look for example, at synthetic control arms, right? The use of real-world data in regulatory submissions over the last five or six years, all of the use cases have been in oncology. I think there've been about maybe somewhere between eight to 10 submissions. And I think only one actually was a successful submission, uh, in all those situations, the real-world data arm of that oncology trial that synthetic control arm was actually rejected by the FDA because of lack of completeness or, you know, equalness in terms of the data. So the FDA is not going to tell us how to do this. So I think the second part of the question, which is what's the role of industry, it's absolutely on industry in order to figure out exactly what we're talking about, how do we figure out the interoperability, how do we apply the standards? >>How do we ensure good quality data? How do we enrich it and create the cohort that is going to be equivalent to the patient in the real world, uh, in the end that would otherwise be in the clinical trial and how do we create something that the FDA can agree with? And we'll certainly we'll want to work with the FDA in order to figure out this model. And I think companies are already doing that, but I think that the onus is going to be on industry in order to figure out how you actually operationalize this and make it real. >>Excellent. Thank you. Um, question on what's the most common misconception that clinical research stakeholders with sites or participants, et cetera might have about DCTs? >>Um, I could jump in there. Right. So, sure. So, um, I think in terms of misconceptions, um, I think the communist misconceptions that sites are going away forever, which I do not think is really happening today. Then the second, second part of it is that, um, I think also the perspective that patients are potentially neglected because they're moving away. So we'll pay when I, when I, what I mean by that neglected, perhaps it was not the appropriate term, but the fact that, uh, will patients will, will, will patient engagement continue, will retention be strong since the patients are not interacting in person with the investigator quite as much. Um, so site retention and patient retention or engagement from both perspectives, I think remains a concern. Um, but actually if you look at, uh, look at, uh, assessments that have been done, I think patients are more than happy. >>Majority of the patients have been really happy about, about the new model. And in fact, sites are, seem to increase, have increased investments in technology by 50% to support this kind of a model. So, and the last thing is that, you know, decentralized trials is a great model and it can be applied to every possible clinical trial. And in another couple of weeks, the whole industry will be implementing only decentralized trials. I think we are far away from that. It's just not something that you would implement across every trial. And we discussed that already. So you have to find the right use cases for that. So I think those were some of the key misconceptions I'd say in the industry right now. Yeah. >>Yeah. And I would add that the misconception I hear the most about is, uh, the, the similar to what Namita said about the sites and healthcare professionals, not being involved to the level that they are today. Uh, when I mentioned earlier in our conversation about being excited about capturing more data, uh, from the patient that was always in context of, in addition to, you know, healthcare professional opinion, because I think both of them bring that enrichment and a broader perspective of that patient experience, whatever disease they're faced with. So I, I think some people think is just an all internet trial with just someone, uh, putting out there their own perspective. And, and it's, it's a combination of both to, to deliver a robust data set. >>Yeah. Maybe I'll just comment on, it reminds me of probably 10 or 15 years ago, maybe even more when, um, really remote monitoring was enabled, right? So you didn't have to have the study coordinator traveled to the investigative site in order to check the temperature of the freezer and make sure that patient records were being completed appropriately because they could have a remote visit and they could, they could send the data in a via electronic data and do the monitoring visit, you know, in real time, just the way we're having this kind of communication here. And there was just so much fear that you were going to replace or supplant the personal relationship between the sites between the study coordinators that you were going to, you know, have to supplant the role of the monitor, which was always a very important role in clinical trials. >>And I think people that really want to do embrace the technology and the advantages that it provided quickly saw that what it allowed was the monitor to do higher value work, you know, instead of going in and checking the temperature on a freezer, when they did have their visit, they were able to sit and have a quality discussion for example, about how patient recruitment was going or what was coming up in terms of the consent. And so it created a much more high touch, high quality type of interaction between the monitor and the investigative site. And I think we should be looking for the same advantages from DCT. We shouldn't fear it. We shouldn't think that it's going to supplant the site or the investigator or the relationship. It's our job to figure out where the technology fits and clinical sciences always got to be high touch combined with high-tech, but the high touch has to lead. And so getting that balance right? And so that's going to happen here as well. We will figure out other high value work, meaningful work for the site staff to do while they let the technology take care of the lower quality work, if you will, or the lower value work, >>That's not an, or it's an, and, and you're talking about the higher value work. And it, it leads me to something that Greg said earlier about the 80, 20, 80% is assembly. 20% is actually doing the analysis and that's not unique to, to, to life sciences, but, but sort of question is it's an organizational question in terms of how we think about data and how we approach data in the future. So Bamyan historically big data in life sciences in any industry really is required highly centralized and specialized teams to do things that the rain was talking about, the enrichment, the provenance, the data quality, the governance, the PR highly hyper specialized teams to do that. And they serve different constituencies. You know, not necessarily with that, with, with context, they're just kind of data people. Um, so they have responsibility for doing all those things. Greg, for instance, within literally, are you seeing a move to, to, to democratize data access? We've talked about data interoperability, part of that state of sharing, um, that kind of breaks that centralized hold, or is that just too far in the future? It's too risky in this industry? >>Uh, it's actually happening now. Uh, it's a great point. We, we try to classify what people can do. And, uh, the example would be you give someone who's less analytically qualified, uh, give them a dashboard, let them interact with the data, let them better understand, uh, what, what we're seeing out in the real world. Uh, there's a middle user, someone who you could give them, they can do some analysis with the tool. And the nice thing with that is you have some guardrails around that and you keep them in their lane, but it allows them to do some of their work without having to go ask those centralized experts that, that you mentioned their precious resources. And that's the third group is those, uh, highly analytical folks that can, can really deliver, uh, just value beyond. But when they're doing all those other things, uh, it really hinders them from doing what we've been talking about is the high value stuff. So we've, we've kind of split into those. We look at people using data in one of those three lanes and it, and it has helped I think, uh, us better not try to make a one fit solution for, for how we deliver data and analytic tools for people. Right. >>Okay. I mean, DCT hot topic with the, the, the audience here. Another question, um, what capabilities do sponsors and CRS need to develop in-house to pivot toward DCT? >>Should I jump in here? Yeah, I mean, um, I think, you know, when, when we speak about DCTs and when I speak with, uh, folks around in the industry, I, it takes me back to the days of risk-based monitoring. When it was first being implemented, it was a huge organizational change from the conventional monitoring models to centralize monitoring and risk-based monitoring, it needs a mental reset. It needs as Lorraine had pointed out a little while ago, restructuring workflows, re redefining processes. And I think that is one big piece. That is, I think the first piece, when, you know, when you're implementing a new model, I think organizational change management is a big piece of it because you are disturbing existing structures, existing methods. So getting that buy-in across the organization towards the new model, seeing what the value add in it. And where do you personally fit into that story? >>How do your workflows change, or how was your role impacted? I think without that this industry will struggle. So I see organizations, I think, first trying to work on that piece to build that in. And then of course, I also want to step back for the second to the, uh, to the point that you brought out about data democratization. And I think Greg Greg gave an excellent point, uh, input about how it's happening in the industry. But I would also say that the data democratization really empowerment of, of, of the stakeholders also includes the sites, the investigators. So what is the level of access to data that you know, that they have now, and is it, uh, as well as patients? So see increasingly more and more companies trying to provide access to patients finally, it's their data. So why shouldn't they have some insights to it, right. So access to patients and, uh, you know, the 80, 20 part of it. Uh, yes, he's absolutely right that, uh, we want to see that flip from, uh, 20%, um, you know, focusing on, on actually integrating the data 80% of analytics, but the real future will be coming in when actually the 20 and 18 has gone. And you actually have analysts the insights out on a silver platter. That's kind of wishful thinking, some of the industries is getting there in small pieces, but yeah, then that's just why I should, why we share >>Great points. >>And I think that we're, we're there in terms that like, I really appreciate the point around democratizing the data and giving the patient access ownership and control over their own data. I mean, you know, we see the health portals that are now available for patients to view their own records, images, and labs, and claims and EMR. We have blockchain technology, which is really critical here in terms of the patient, being able to pull all of their own data together, you know, in the blockchain and immutable record that they can own and control if they want to use that to transact clinical trial types of opportunities based on their data, they can, or other real world scenarios. But if they want to just manage their own data because they're traveling and if they're in a risky health situation, they've got their own record of their health, their health history, uh, which can avoid, you know, medical errors occurring. So, you know, even going beyond life sciences, I think this idea of democratizing data is just good for health. It's just good for people. And we definitely have the technology that can make it a reality. Now >>You're here. We have just about 10 minutes left and now of course, now all the questions are rolling in like crazy from the crowd. Would it be curious to know if there would be any comments from the panel on cost comparison analysis between traditional clinical trials in DCTs and how could the outcome effect the implementation of DCTs any sort of high-level framework you can share? >>I would say these are still early days to, to drive that analysis because I think many companies are, um, are still in the early stages of implementation. They've done a couple of trials. The other part of it that's important to keep in mind is, um, is for organizations it's, they're at a stage of, uh, of being on the learning curve. So when you're, you're calculating the cost efficiencies, if ideally you should have had two stakeholders involved, you could have potentially 20 stakeholders involved because everyone's trying to learn the process and see how it's going to be implemented. So, um, I don't think, and the third part of it, I think is organizations are still defining their KPIs. How do you measure it? What do you measure? So, um, and even still plugging in the pieces of technology that they need to fit in, who are they partnering with? >>What are the pieces of technology they're implementing? So I don't think there is a clear cut as answered at this stage. I think as you scale this model, the efficiencies will be seen. It's like any new technology or any new solution that's implemented in the first stages. It's always a little more complex and in fact sometimes costs extra. But as, as you start scaling it, as you establish your workflows, as you streamline it, the cost efficiencies will start becoming evident. That's why the industry is moving there. And I think that's how it turned out on the long run. >>Yeah. Just make it maybe out a comment. If you don't mind, the clinical trials are, have traditionally been costed are budgeted is on a per patient basis. And so, you know, based on the difficulty of the therapeutic area to recruit a rare oncology or neuromuscular disease, there's an average that it costs in order to find that patient and then execute the various procedures throughout the clinical trial on that patient. And so the difficulty of reaching the patient and then the complexity of the trial has led to what we might call a per patient stipend, which is just the metric that we use to sort of figure out what the average cost of a trial will be. So I think to point, we're going to have to see where the ability to adjust workflows, get to patients faster, collect data more easily in order to make the burden on the site, less onerous. I think once we start to see that work eases up because of technology, then I think we'll start to see those cost equations change. But I think right now the system isn't designed in order to really measure the economic benefit of de-central models. And I think we're going to have to sort of figure out what that looks like as we go along and since it's patient oriented right now, we'll have to say, well, you know, how does that work, ease up? And to those costs actually come down and then >>Just scale, it's going to be more, more clear as the media was saying, next question from the audiences, it's kind of a best fit question. You all have touched on this, but let me just ask it is what examples in which, in which phases suit DCT in its current form, be it fully DCT or hybrid models, none of our horses for courses question. >>Well, I think it's kind of, uh, it's, it's it's has its efficiencies, obviously on the later phases, then the absolute early phase trials, those are not the ideal models for DCTs I would say so. And again, the logic is also the fact that, you know, when you're, you're going into the later phase trials, the volume of number of patients is increasing considerably to the point that Lorraine brought up about access to the patients about patient selection. The fact, I think what one should look at is really the advantages that it brings in, in terms of, you know, patient access in terms of patient diversity, which is a big piece that, um, the cities are enabling. So, um, if you, if, if you, if you look at the spectrum of, of these advantages and, and just to step back for a moment, if you, if you're looking at costs, like you're looking at things like remote site monitoring, um, is, is a big, big plus, right? >>I mean, uh, site monitoring alone accounts for around a third of the trial costs. So there are so many pieces that fall in together. The challenge actually that comes when you're in defining DCTs and there are, as Rick pointed out multiple definitions of DCTs that are existing, uh, you know, in the industry right now, whether you're talking of what Detroit is doing, or you're talking about acro or Citi or others. But the point is it's a continuum, it's a continuum of different pieces that have been woven together. And so how do you decide which pieces you're plugging in and how does that impact the total cost or the solution that you're implementing? >>Great, thank you. Last question we have in the audience, excuse me. What changes have you seen? Are there others that you can share from the FDA EU APAC, regulators and supporting DCTs precision medicine for approval processes, anything you guys would highlight that we should be aware of? >>Um, I could quickly just add that. I think, um, I'm just publishing a report on de-centralized clinical trials should be published shortly, uh, perspective on that. But I would say that right now, um, there, there was a, in the FDA agenda, there was a plan for a decentralized clinical trials guidance, as far as I'm aware, one has not yet been published. There have been significant guidances that have been published both by email and by, uh, the FDA that, um, you know, around the implementation of clinical trials during the COVID pandemic, which incorporate various technology pieces, which support the DCD model. Um, but I, and again, I think one of the reasons why it's not easy to publish a well-defined guidance on that is because there are so many moving pieces in it. I think it's the Danish, uh, regulatory agency, which has per se published a guidance and revised it as well on decentralized clinical trials. >>Right. Okay. Uh, we're pretty much out of time, but I, I wonder Lorraine, if you could give us some, some final thoughts and bring us home things that we should be watching or how you see the future. >>Well, I think first of all, let me, let me thank the panel. Uh, we really appreciate Greg from Lily and the meta from IDC bringing their perspectives to this conversation. And, uh, I hope that the audience has enjoyed the, uh, the discussion that we've had around the future state of real world data as, as well as DCT. And I think, you know, some of the themes that we've talked about, number one, I think we have a vision and I think we have the right strategies in terms of the future promise of real-world data in any number of different applications. We certainly have talked about the promise of DCT to be more efficient, to get us closer to the patient. I think that what we have to focus on is how we come together as an industry to really work through these very vexing operational issues, because those are always the things that hang us up and whether it's clinical research or whether it's later stage, uh, applications of data. >>We, the healthcare system is still very fragmented, particularly in the us. Um, it's still very, state-based, uh, you know, different states can have different kinds of, uh, of, of cultures and geographic, uh, delineations. And so I think that, you know, figuring out a way that we can sort of harmonize and bring all of the data together, bring some of the models together. I think that's what you need to look to us to do both industry consulting organizations, such as IBM Watson health. And we are, you know, through DTRA and, and other, uh, consortia and different bodies. I think we're all identifying what the challenges are in terms of making this a reality and working systematically on those. >>It's always a pleasure to work with such great panelists. Thank you, Lorraine Marshawn, Dr. Namita LeMay, and Greg Cunningham really appreciate your participation today and your insights. The next three years of life sciences, innovation, precision medicine, advanced clinical data management and beyond has been brought to you by IBM in the cube. You're a global leader in high tech coverage. And while this discussion has concluded, the conversation continues. So please take a moment to answer a few questions about today's panel on behalf of the entire IBM life sciences team and the cube decks for your time and your feedback. And we'll see you next time.

>>Oh, here we go again. Yeah. Here we go again. >>Cube Coverage of AWS re invent continues in a moment.

>>Oh, here we go again. Yeah. Here we go again. >>Cube Coverage of AWS re invent continues in a moment.

(upbeat music) >> We're back at AWS re:Invent 2021. It's the biggest hybrid event of the year. One of the few physical events and we're psyched to be here. My name is Dave Vellante, and I'm really pleased to bring back the host emeritus, Stu Miniman, somebody I worked with side-by-side, Stu, for 10 years in a setting much like this, many like this. So, good to have you back. >> Dave, it's great to be here with theCUBE team, family here and re:Invent, Dave. I mean, this show, I remember back, Dave, going to you after the first re:Invent we talked, we were like, "We got to be there." Dave, remember the first year we came, the second year of re:Invent, this is the 10th year now, little card tables, gaming companies, all this stuff. You had Jerry Chen on yesterday and Jerry was comparing like, this is going to be like the next Microsoft. And we bet heavy on this ecosystem. And yeah, we all think this cloud thing, it might be real. 20,000 people here, it's not the 50 or 75,000 that we had in like 2018, 2019, but this ecosystem, what's happening in the cloud, multiple versions of hybrid going on with the event and the services, but yeah, phenomenal stuff. And yeah, it's so nice to see people. >> That's for sure. It's something that we've talked about a lot over the years is, and you remember the early days of re:Invent and to this day, just very a strong developer affinity that AWS has done a tremendous job of building that up and it's their raison d'etre, it's how they approach the market. But now you've been at Red Hat for a bit, obviously as well, developer affinity, what have you learned? Specifically as it relates to the cloud, Kubernetes, hottest thing going, you don't want to do an OpenShift commercial, but it's there, you're in the middle of that mix. What have you learned generally? >> Well, Dave, to the comment that you made about developers here, it's developers and the enterprise. We used to have a joke and say, enterprise developer is an oxymoron, but that line between developers doing stuff, early as a cloud, it was stealth computing. It's they're often doing this stuff and central IT is not managing it. So how do the pieces come together? How do apps and infrastructure, how do those pieces come together? And it's something that Red Hat has been doing a long time. Think about the Linux developer. They might've not have been the app developers, the people building Linux and everything, but they had a decent close tie to it. I'm on the OpenShift team. What we do is cloud, Dave, and we've got a partnership here with Amazon. We GAed our native cloud service earlier this year. Andy Jassy helped name it. It is the beautifully named Red Hat OpenShift Service on AWS or ROSA. But we've done OpenShift on AWS for more than five years, basically since we were doing Kubernetes, it's been here because of course customers doing cloud, where are they? A lot of them are here in Amazon. So I've been loving talking to a lot of customers, understanding how enterprise adoption is increasing, how we can enable developers and help them move faster. And yeah, I mean the quick plug on OpenShift is our service. We've got an SRE team that is going to manage all of that. A friend of the program, Corey Quinn, says, "Hey, an SRE team like that, because you don't want to manage as an enterprise." You don't want to manage Kubernetes. Yeah, you need to understand some of the pieces, but what is important to your business is the applications, your data and all those things and managing the undifferentiated heavy lifting. That's one of the reasons you went to the cloud. So therefore changing your model as to how you consume services in the cloud. And what are we seeing with Amazon, Dave? They're trying to build more solutions, simplify deployments, and offer more solutions including with their ecosystem. >> So I want to ask you. You said enterprise developer is kind of an oxymoron, and I remember, years ago I used to hang around with a lot of heads of application development and insurance companies and financial services, pharmaceutical, and they didn't wear hoodies, but they didn't wear suits either. And then when I talked to guys like Jeff Clark, for instance. He talks about we're building an abstraction layer across clouds, blah, blah, blah, which by the way, I think it is the right strategy. I'm like, "Okay, I'll drink some of that Kool-Aid." And then when I come here, we talked to Adam Selipsky. John flew out and I was on the chime. He goes, "Yeah, that's not hybrid. No, this is nothing like, it's not AWS, AWS is cloud." So, square that circle for me, 'cause you're in both worlds and certainly your strategy is to connect those words. Is that cloud? >> Yeah, right. I mean, Dave, we spent years talking about like is private cloud really a cloud? And when we started coming to the show, there is only one cloud. It is the public cloud and Amazon is the paragon of, I don't know what it was. >> Dave: Fake clouds, cloud washing. >> So today, Amazon's putting lots of things into your data center and extending the cloud out to that environment. >> So that's cloud. >> That's cloud. >> What do we call that cloud? What about the reverse? >> What's happening at the edge is that cloud is that extension of what we said from Amazon. If you look at not only Outpost, but Wavelengths and Local Zones and everything else like that. >> Let's say, yes, that's cloud. The APIs, primitives, check. >> Dave, I've always thought cloud is an operating model, not a location. And the hybrid definition is not the old, I did an ebook on this, Dave earlier this year. It's not the decade old NIS definition of an application that spans because I don't get up in the morning as an enterprise and say, "Oh, let me look at the table of how much Google is charging me or Microsoft or Amazon," or wake up one morning and move from one cloud to the other. Portability, follow the sun type stuff, does it ever happen? Yes, but it is rare thing. Applications oftentimes get pulled apart. So we've seen if you talk about AI, training the cloud, then transact and do things at the edge. If I'm in an autonomous vehicle or in a geosynchronous satellite, I can't be going back to the cloud to process stuff. So I get what I need and I process there. The same thing hybrid, oftentimes I will do my transactional activity in the public cloud because I've got unlimited compute capability, but I might have my repository of data for many different reasons, governance or security, all these things in my own data center. So parts of an application might live there, but I don't just span to go between the public cloud in my data center or the edge, it's specific architectural decisions as to how we do this. And by the way the developer, they don't want to have to think about location. I mean, my background, servers, storage, virtualization, all that stuff, that was very much an infrastructure up look of things. Developers want to worry about their code and make sure that it works in production. >> Okay, let me test that. If it's in the AWS cloud and I think it's true for the other hyperscale clouds too, they don't have to think about location, but they still have to think about location on-prem, don't they? >> Well, Dave, even in a public cloud, you do need to worry about sometimes it's like, "Okay, do I split it between availability zones? How do I build that? How do I do that?" So there are things that we build on top of it. So we've seen Amazon. >> I think that's fair, data sovereignty, you have to think about okay. >> Absolutely, a lot of those things. >> Okay, but the experience in Germany is going to be the same as it is in DC, is it not? >> More or less? There are some differences we'll see off and Amazon will roll things out over time and what's available, you've got cloud. >> For sure, though that's definitely true. That's a maturity thing, right? You've talked a bit, but ultimately they all sort of catch up. I guess my question would be is the delta between, let's say, Fed adoption and East Coast, is that delta narrower, significantly narrow than what you might see on-prem? >> The services are the same, sometimes for financial or political things, there might be some slight differences, but yes, the cloud experience should be the same everywhere from Amazon. >> Is it from a standpoint of hybrid, on-prem to cloud, across cloud? >> Many of the things when they go outside of the Amazon data centers are limited or a little bit different or you might have latency considerations that you have to consider. >> Now it's a tug of war. >> So it's not totally seamless because, David Foyer would tell us there, "You're not going to fight physics." There are certain things that we need to have and we've changed the way we architect things because it's no longer the bottleneck of the local scuzzy connection that you have there, it is now (indistinct). >> But the point I'm making is that gets into a tug of war of "Our way is better than your way." And the answer is depends in terms of your workload and the use case. >> You've looked at some of these new databases that span globes and do things of the like. >> Another question, I don't know if you saw the Goldman Sachs deal this morning, Goldman Sachs is basically turning its business into a SaaS and pointing it to their hedge funds and allowing people to access their data, their tools, their software that they built for their own purposes. And now they're outselling it. Similar to what NASDAQ has done. I can't imagine doing that without containers. >> Yeah, so interesting point, I think. At least six years ago now, Amazon launched serverless and serverless was going to take over the world. I dug into the space for a couple of years. And you had the serverless with camp and you had the container camp. Last year at re:Invent, I really felt a shift from Amazon's positioning that many of the abstraction layers and the tools that help you support those environments will now span between Lambda and containers. The container world has been adding serverless functionality. So Amazon does Fargate. The open-source community uses something called Knative, and just breaking this week. Knative was a project that Google started and it looks like that is going to move over to the CNCF. So be part of the whole Kubernetes ecosystem and everything like that. Oracle, VMware, IBM, Red Hat, all heavily involved in Knative, and we're all excited to see that go into the CNCF. So the reason I say that, I've seen from Amazon, I actually, John and I, when we interviewed Andy Jassy back in 2017, I asked him a follow-up question because he said if he was to build AWS in 2017, "I would start with everything underneath it serverless." I would wonder if following up with Adam or Andy today, I'd said, "Would it be all serverless or would containers be a piece of it?" Because sometimes underneath it doesn't matter or sometimes it can be containers and serverless. It's a single unit in Amazon and when they position things, it's now that spectrum of unit, everything from the serverless through the containers, through... James Hamilton wrote a blog post today about running Xen-on-Nitro and they have a migration service for a mainframe. So what do we know? That one of the only things about IT is almost nothing ever goes away. I mean, it sounded like Amazon declared coming soon the end of life of mainframe. My friends over at IBM might not be quite ready to call that era over but we shall see. All these things take time. Everything in IT is additive. I'm happy to see. It is very much usually an end world when I look at the container and Kubernetes space. That is something that you can have a broad spectrum of applications. So some of my more monolithic applications can move over, my cool new data, AI things, I can build on it, microservices in between. And so, it's a broad platform that spans the cloud, the edge, the data center. So that cloud operating model is easier to have consistency all the places that I go. >> Mainframe is in the cloud. Well, we'll see. Big banks by the next site unseen. So I think Amazon will be able to eat away at the edges of that, but I don't think there's going to be a major migration. They claim it. Their big thing is that you can't get COBOL programmers. So I'm like, "Yeah, call DXC, you'll get plenty." Let's talk about something more interesting. (Stu laughs softly) So the last 10 years was a lot of, a lot about IT transformation and there was a lot more room to grow there. I mean, the four big hyperscalers are going to do 120 billion this year. They're growing at 35%. Maybe it's not a trillion, but there's a $500 billion market that they're going after, maybe more. It looks like there's a real move. You saw that with NASDAQ, the Goldman deal, to really drive into business, deeper business integration in addition to IT transformation. So how do you see the next decade of cloud? What should we be watching? >> So, one of the interesting trends, I mean, Dave, for years we covered big data and big data felt very horizontal in it's approach thing. Hadoop take over the world. When I look at AI solutions, when I look at the edge computing technologies that happen, they're very vertically driven. So, our early customers in edge adoption tend to be like telco with the 5G rollout manufacturing in some of their environments. AI, every single industry has a whole set of use cases that they're using that go very deep. So I think cloud computing goes from, we talked about infrastructure as a service to it needs to be more, it is solution, some of these pieces go together. When Adam got up on stage and talked about how many instance types they have on Amazon, Dave, it's got to be 2X or 4X more different instant types than if I went to go to HPE or Dell and buy a physical server for my environment. So we need to have areas and guidance and blueprints and heck, use some of that ML and AI to help drive people to the right solutions because we definitely have the paradox of choice today. So I think you will find some gravity moving towards some of these environments. Gravatar has been really interesting to watch. Obviously that Annapurna acquisition should be down as one of the biggest ones in the cloud era. >> No lack of optionality to your point. So I guess to the point of deeper business integration, that's the big question, will Amazon provide more solution abstractions? They certainly do with Connect. We didn't hear a ton of that this show. >> Interestingly. (Dave speaking indistinctly) So the article that you and John Furrier wrote after meeting with Adam, the thing that caught my eye is discussion of community and ecosystems. And one of the things coming after, some, big communities out there like, you and I lived through the VMware ecosystem in that very tight community. There are forming little areas of community here in this group, but it's not a single cloud community. There are those focus areas that they have. And I do love to see, I mean, obviously working for Red Hat, talking about the ecosystem support. I was very happy to hear Adam mention Red Hat in the keynote as one of the key hybrid partners there. So, for Amazon to get from the 60 million, the 60 billion to the trillion dollar mark down the road, it's going to take a village and we're happy to be a part of it. >> Hey, great to have you back, enjoy the rest of the show. This is, let's see, day three, we're wrapping up. We're here again tomorrow so check it out. Special thanks to obviously AWS is our anchor sponsor and of course, AMD for sponsoring the editorial segments of our event. You're watching theCUBE, the leader in tech coverage. See you tomorrow. (bright upbeat music)

>>Good morning. Welcome to the cubes. Continuous coverage of AWS reinvent 2021. I'm Lisa Martin with Dave Nicholson morning, Dave. Good morning. Here we are two sets, two live sets, two remote sets, a hundred guests on the program this week. This is day, second full day for us. We were here. Exactly. The number of announcements has my head spinning typical for AWS. So many. We won't even recap all the pre, but really from a thematic perspective. One of the things that I think we've both noticed is that the theme here is if you're not a data-driven company, if you don't have a data company at your core, you're going to be left behind. >>Yeah, yeah. That was that. That came through in a Swami's keynote this morning. That was the key setup for the announcements of all of the tools that are being rolled out to be able to achieve those goals. >>And we talked to, you know, we've had a number of AWS folks on the program this week, customers partners, and from another theme is this customer obsession, this customer centricity, really working backwards from the customer first to help them understand how do we, how do we build a data core? How do we build our redefine reinvent, Hubba reinvent our business around a data strategy because there is a competitor in the rear view mirror in every industry waiting to take place. If somebody doesn't have. And they really were very clear with, if you don't, if you're not a data company, now you're going to be left behind. >>Yeah, absolutely. The whole, it, especially the keynote this morning, it felt like a, it felt like they were very, very careful to set up the pins and then knock them all down. Talking about exactly what customers want to do, what customers have said in surveys they need to do. Uh, and then, oh, by the way, we have all of these tools. So exactly what you mentioned, not talking about technology first, but talking about the requirements the customers have, where customers say they want to be, where customers need to be, and then proving out that in fact, AWS has the entire tool kit. Uh, I, I mean it is mind boggling. Uh, I, uh, I wouldn't want to be another cloud this morning, waking up and seeing that it looks like I fell months behind somehow. Why not? While I was sleeping. >>One of the things that you and I were talking about as we walked over here this morning is that it's no wonder AWS. And they say they don't look in their rear view mirror. They don't need to. One of the things the magically that Adam talked about in his keynote yesterday was I kept saying the phrase, but you wanted more, you being, the customer did this, but you wanted more. So we did this, but you wanted more. So we did this, the fact that they start backwards, that customer flywheel. The thing that another theme that got me is all of the innovation that's coming out of AWS that comes from the customers, enables AWS to do so much more enables customers across every industry to do so much more and to competitively differentiate themselves if they truly lean in to the power of AWS and the partnership that can deliver. >>Yeah, no, no completely agree. And you know, from, from sort of an analysis perspective, um, I think it's interesting, uh, part of the theme this morning had to do with artificial intelligence and, um, and I, I just personally like to think of it more as augmented intelligence, uh, cause essentially what they're doing is they're building tools that are powerful, that, uh, that amplify human intelligence, it's not that AI replaces human intelligence, maybe it does for certain things, but machine learning, most of that is really machine training, humans, training the machines. Yeah. We'll get to a point. Yes. There are science experiments where, um, where the computer has learned on its own. Uh, but essentially all of the pieces that Swami was talking about putting together today, sources of data, various databases, everything together that equals an amazing amount of opportunity for human beings moving forward. So this isn't about machine technology, replacing the value of what people do, data scientists. I mean, I'm, I'm, I'm texting my kids data science used to be plastics right during their graduate. Now it's all about data science be a data scientist. Well, >>Another thing from a data perspective is the idea of data democratization. We've talked. We talk about that all the time. It's interesting. We were, I was talking about data mesh yesterday as, as a really facilitator and unlocking technical debt facilitating the democratization. So that because data is so scattered these days, I mean, it's one of the things that we saw when the pandemic stuck was people scattered. We're still scattered data sources are only proliferating data volume is only proliferating. Customers need to be able to harness the value of it in real time. I think another thing we learned in the pandemic is of there is no real-time is no longer a nice to have. It is absolutely essential. We've also seen the acceleration and talked a lot in the last couple of days about the acceleration of cloud adoption. And just thinking about the sheer volume of announcements. The last couple of days that AWS has achieved during a global pandemic is phenomenal. Obviously we saw every business have no choice, but to go digital and those that aren't here anymore, weren't able to do that fast enough. And with the scale at which they need to know, meet customer demand. >>Yeah. When you talk democratization often, it's a question of not only can you do it technically, but what is the cost associated with doing it? So when you lower the barrier to entry and you lower the friction associated with all sorts of transactions, not just financial transactions, but a transaction being teasing important information out of a pile of otherwise meaningless data, when you lower the cost of that, like what AWS is doing with all of these tools. I mean, really that's the, that's the end result of this? Uh, I look at these things through the lens of an economist and uh, and it's about decreasing friction and increasing efficiency. And when you do that, you can't imagine what the effect of that is on the entire world, just by lowering the cost of teasing out important information from otherwise useless data. Yeah. >>And another thing that we talked about the vision of this company, this is a 10th reinvent, 15 years of AWS, but the 10th reinvent and wanted to get your take on the last, you know, the first 15 years we'll say of AWS division, obviously this is the first year with a new head, new CEO, Adam. So Lipski, and, uh, we're, we're used to the, the JASSI era. This is now we're looking at like, what's the next 10 years of cloud innovation gonna look like at AWS? >>Yeah. I get a chill when you ask that question. I like, I was there working with Amazon before there was an AWS. So in the early two thousands, uh, working at a very large data storage company and we're working with then Amazon to help them figure out how they might someday put together something that they could not only use for themselves, but also rent out to others, sort of this novel idea. Um, and so that feels like yesterday to me, um, the idea that AWS was ECE to S3 science experiment, not ready for enterprise. All of that feels like yesterday. Those claims most people at the time realized that they were going to be disruptive. I don't think a lot of people realized how relatively quickly it was going to happen. And I don't think anyone could have predicted that in 2021, they would be as dominant as they are with such a small share of the overall it market. >>So onstage, they'll say five to 15% of it is in the cloud, right? Not five to 15% is in AWS, right. Five to 15% is in the cloud. So you think that AWS is a large organization firing on all cylinders. Now they're just at the beginning of addressing the total addressable market. So I, you know, um, uh, Dave Vellante, uh, had a great article on the subject of the natural inclination for people to worry when certain things get to a certain size. Um, I think that that's going to be a part of the conversation moving forward. Uh, uh, you know, just what does it mean when so much critical infrastructure is being handled by a single entity? Uh, from a user's perspective though, Hey, it's all good. The more they integrate, the more they bring things together, the easier they make my life as an it practitioner. So I, I don't, I have no idea five years from now what we're going to be looking at. >>No, and they, you know, one of the things that, that I think I've done maybe three or four reinvents and the, one of the themes always, or, or really kind of taglines is early innings. We're S we're still early days. You talked about, you know, when the 15% of it spending, being in the cloud, they have their, and they're so massive, but there's so much the Tam is, is enormous, absolutely enormous. And we're seeing, you know, the need for as data volumes, we're only going to continue to proliferate. We have to have the artificial intelligence and the machine learning to help the humans process the data, to be able to unlock all that value. Otherwise organizations, risk being left behind when they simply cannot get value out of the data either at all, or fast enough. >>Well, you know, you know, let me make one prediction actually. Okay. Okay. Um, I think that the definition of cloud, which is different in a lot of people's minds already is increasingly going to be broad broadened out to include what AWS at a certain period of time said, didn't matter. And that was what's going on on premises. It was clear that in the early days, AWS adopted an attitude that their, their stratospheric growth could be maintained on the back of net. New stuff. Only let the stuff on, on premises die. The stickiness associated with that on premises stuff has caused them to change direction over time in a very, very smart way. And so my prediction is that in five years, we might not be using the term cloud at all. We might be back to just calling it all it, because when you think about it, cloud represents this idea that the physical location isn't important, it's all virtualized. >>So if that's the case, why does it matter if it's in a data center that I own, or a colo or data center that AWS owns, it doesn't matter. So when you look at things like snowball and outpost and all of these other things where they're taking essentially AWS, and they're sticking it wherever it needs to be fit for function, you have latency, sovereignty, govern governance issues, fine. We'll run it as a black box in your data center. You don't touch it, you consume it, you interact with it just like it's in an AWS data center. So, so that's my prediction is that it's not that, you know, w we're going to start thinking about this comment that X percentage is in the cloud as being sort of like a, well, what do you even mean by that? That's a good point. Is it in the cloud? If it's in an outpost in my data center, right. Is that included in the 15% or does it not? Is that on-prem or isn't it? I think, I think within five years, we're all going to agree that it just doesn't matter. It's all it. And I think AWS will be still dominating the it space. >>Interesting prediction. I liked that you were bold in that, you know it, but it also shows what you talked about, shows how focused AWS is on the customer, really leaning into everything in the public cloud, you know, no more on-prem years ago. And, and then what, what is outpost announced two years ago? So it's, I think it's been a couple years to, >>Yeah, I confused. I can. I confuse announcements with deployments and things like that, but it's been years it's been years now. And I guarantee you that inside the halls of AWS, there were people who said, no, no, no, we are not going to be involved with boxes being shipped to raise data center floors. But like you said, customer it's about the customer, what the customer needs. And the reason why there is, you know, if the number is 85%, whatever the number is, um, there, there's a reason why that still remains in on premises data centers. And it's not primarily because it, people are stupid slash fearful. There are good reasons for it. You know, it has to do with the ROI of going through modernization and migration. And so what does AWS do? They figure out how to make the math work so that it makes sense for people to do things. >>And we've been talking for the last couple of days with a lot of the really, really important we'll continue to the really, really important service providers that are partners because AWS and the rest of the cloud providers can come up with as much technology as they want to, but they need partners to bridge the divide, right? To tease out the human value, the business value from the technology they provide. So again, opportunity people who look at AWS and say, wow, that's scary. They're so big. No, no, no, no, no. They're creating an ecosystem of opportunity around them. So that's just, >>Yeah, no, their ecosystem is huge. We talked about this cube being sponsored by AMD. What we talked about, the ecosystem of partners at the beginning of the segment, it's, it's, it's massive. We've talked with a lot of different partners, huge partners, smaller partners, overall, at the end of the day, what we see, what they create, all of the technologies that they're creating in response to customer needs. You know, we have to address it there. That the more data we have, the more things grow, the more complex and complexity has to be dealt with. And there's a lot of their ecosystem partners that are really have business models designed around helping customers. What does seamless actually mean? How do you actually become efficient? How do we do this in a frictionless way? Those are all great marketing terms, but they need this ecosystem of partners to help the customers actually make those reality. Right? And otherwise they're just words on a, on a piece of paper >>And it's not easy. Implementation is not easy. Sales is easy, implementation, not easy, right. And, uh, you know, the, the, the risk associated with failure can be very, very high, especially from a financial perspective. You know, the endless project that you're sinking money into that doesn't deliver any ROI just doesn't work. No, you can't, you can't do it. So, yeah, I, I I'm, I'm blown away by just today. Today's keynote going through knocking down those pins one just one after the other. Oh, CQL got it now, you know, for, for, for certain environments where before, yeah. We've got Oracle database support for this function. Now we have SQL also, it's just, you know, it's like this thousand by thousand grid check box, right? Check, check, check, check, check. And I can imagine all of those service providers just salivating, because they're thinking, oh my gosh, practice new practice that we need to spin up to be able to do this. Yep. Which points to a huge challenge. And that is actual practitioners where the rubber meets the road. All of these things require smart people to be able to implement. >>And the good news is a lot of those smart people are on the program this week here, you're going to be able to get, to hear so much more about the innovations that AWS and its ecosystem of partners are doing. We have a LA a full day again today, Dave, and again, tomorrow looking forward to hearing about the conversations that you had, I'm looking forward to interviewing guests and we look forward to having you continue to watch the cube, the global leader in live tech coverage.

>>Back cubes back on day three here in cloud city, global world Congress. This is where all the action is. And this is the cube set. I'm John. We're here with Dr., who is the CEO of telco VR, as well as the CEO of Tacoma. Great to see you again. Hey, Hey, how are you guys? Good, great time. Great boat last night. Good industry executives. A lot of intimate high player players here in the industry, even though not a lot of attendance, but the right people are here and events are back. >>Yeah. Yeah. I think, um, you know, MWC was the first event to cancel with COVID in, uh, February of 20 end of February, 2020. So first big event to come back. It's such a nice symmetry. Um, yeah, typically you have big delegations, hundreds of people from the big groups coming to the show, we're seeing the executives are coming smaller delegations, but they're all in the booth and that we're having great conversations and it's awesome. >>Yeah. And the thing I will say is that the cubes back to we'd like them to be fee in here in the action. He says, one of the things that's happened with this hybrid events is that people are watching. And so there's a virtual space and the physical space and cloud city has built out paradise. It's beautiful and spectacular behind this. Look, you look around for the people who can't see, it's really made for the combination of onsite and virtual experience, the content, the people Bon Jovi last night, just, it's just the top of mobile world Congress. And it's translating to the industry. This has been amazing. So congratulations. Well, >>I think I got to say you have a lot to say as we all know. Yeah. But I think it was easy for the big guys. That's why we love you in the queue, but I think it was easy for the big guys to tap out and say, Hey, we can save a bunch more money. We don't really have much to talk about. Right. We're going to talk about again, let's talk about 5g revolution. Whereas, whereas the narrative here is all about the future and it's not about the future of blah, blah, blah. It's about the future. You know, this is the journey that we're taking and here's where it starting and with leaving the boat. >>Yeah. And I think what's really interesting about cloud city is the fact that we've brought these different players together that are all focused, as you said, on the, on the future. And I'm starting to see these connections where they're collaborating right vendors that didn't know each other probably would never have partnered before. Totally different areas. I'm hearing the conversation in the booth about like, Hey, I talked to Pete when security or I went and talked to, you know, LMX and we're putting deals together because we're complimentary and it's amazing. >>And integration partnership heard that from Google yesterday on our, our news exclusive break in there, they see integration. And they're talking about Android with Android, for mobile. They're seeing a whole new software paradigm coming into telephones it's partnership ecosystem and open. These are new kind of dynamics >>For you guys. When you say integration and open, I think those things are really paired and they're important. A lot of times telco people will hear integration, I'll think customization, right? Coding it up and customizing it so that they talk to each other. But I think the open part of that is really important where we're connecting via API APIs. And I think that's bringing in the hyperscalers, that's what they do. Right? They provide these systems and the software, it's all API base and you can use it very quickly and you can unravel it if you need to. It's feature velocity that we talked about a couple of days and automation >>Is the underpinning. I mean, that's really the theme, right? It's not like a one-off hardcore custom integration. That's gonna be, I have >>To upgrade every 18 months or whatever it is. Yeah. It's, it's alive. Yeah. >>How about Musk yesterday? I mean, he's always a crowd pleaser. First of all, my kids love him. He's crazy. >>I mean, he is amazing. >>He's he's, he's a builder. He takes no prisoners. He's just, you know what? My goal was not to go bankrupt. That's what he said a couple of years ago, which >>Was brilliant because everybody's gone bankrupt in that business. And he's just, you know, and he's just like, look it, we're here to >>Just want to chip away at it. And we're just going to keep striving, not making up excuses. He takes the failures, he takes the face plant. He gets back up and he keeps going. He's focused on buildings, >>Some one thing, right? He's not focused on everything. He's focused on getting to Mars. And I think that's what I like to compare myself to Elon Musk, right? Not that I'm building rockets or getting to Mars, but that the hard problem that I'm solving is getting telco to the public cloud. And that's going to take a decade. It might've been accelerated because of COVID, it might've taken 20 years and now it might take 10, but you look at what he does and that guy, he has, he has haters on Twitter there. Pew, pew always like throwing their bars, but he's like, I got, I got my rocket company. I got my, you know, communication and space company. We're going to need the bore holes, the boring company. I need batteries. I got my Tesla company. And so this guy focuses. He's got >>Some haters, but he's got a lot more lovers on his other side because people might not know this, but he fires entire PR department because he's like, I don't need PR. I'm just going to go do my own, his own PR actually the crypto, stuff's always fun. Goats, coins, >>Always a laugh. >>And it's just more of like playing watch this. >>I said, I live was interesting thing he did, but I think he illustrates the point of a new generation. And I think my young kids, not young, they're in their twenties. Now they look at him and they say, that's aspirational because he's building and he's not, he's focused on that one thing. And again, the growth that you mentioned telco to the cloud, thinking back to that, I want to ask you this growth question. It used to be like, okay, growth was there, people expand itself? Howard's networks were networks. Now it seems like the growth of telco for telco is going into what's the edge and all the open ranch stuff, which means that we need more infrastructure. Yeah. We need more stuff. There's more and more needed and there's grow. Find them. >>What's your, what's your, I think we need more software. Right. Software eats the world. Right. And it's, I mean, there was a lot of hardware to Trump in telco and it's just gonna keep eating it. Um, and that's just gonna accelerate. I think that's where tacos needs to start to build that muscle. They don't have great software capability. They don't have public cloud building capability. And so that's a big upskilling. That's a new hiring. And I think it's a, it's an executive conversation. It's not just an it thing or just a marketing thing. I got to chime >>In here for a second because there are a lot of parallels with how the data center transition has occurred. And what's happening here. We talk about all the time. It was a mainframe, et cetera. There are parallels. Yeah. And what happened when the data center went to software defined a whole bunch of hardware was allocated to run all the software defined stuff. It wasn't built for that. But the cloud, what you guys are doing with Togi and taken advantage of AWS is nitro and graviton. That's built to two V software defined. Correct. And so the telcos are going to go through the same thing. If they just virtualized, they're going to say, oh wow, we're wasting 30% of our power, our compute power on just supporting all this software defined stuff, because it wasn't built for that. But the cloud is built for that. And that is going to be a huge >>Difference. And I keep trying to make this distinction. And I think people in telcos still don't get this about the public cloud. They think of it as a place. It's a place to run a workload. And that tells me, they think of it as infrastructure. I think of it as servers still like, well, I'm going to run it in my closet. Or AWS has closet. I'm like, I was just having a conversation about this with a senior person from DSMA. I'm like, it's actually about the software. That's there. It's about the databases they're building and the analytics and the AI and ML that they let you buy by the minutes by the API call. And that is my, like, you need to think about that. Cause it's mindblowing, it's a totally different way. And you're >>Totally right. And just spend it again, give you props on this. I've had many one on with Andy gesture the past seven years, not for exclusives, but over the years it's been consistent. Each platform lifting and shift. Wasn't the end game. Okay. Replatforming in the cloud. Certainly a great advantage, a great starting point. It was the refactoring. And that's why you see Amazon web services. For instance, keep adding more services because that's the model. They keep offering more goodness so that the businesses could refactor, not just replatform. Yeah. And that's what you're getting. I think with the AI and machine learning where you start getting into these new use cases, but why couldn't do that before? Right? Right. This is going to be a huge >>Game changer. Forest Brazil, right? A great guy. A cloud guru wrote a great blog called a lift and shift is a ticking time bomb. And it's a great start to get your stuff over there. It forces your team to start to interact with like an AWS or GCP in a real way. Like now they, they gotta use it. You take it away. And I'm like, but once you move it, you got to read factor. You got to rewrite. And then that's why it's a ticking time bomb. You got to get, move it over and get you're going >>To rush him. Dr. Digital revolution of you are one. You got it here, tells the VR. And this has been a great experience for the cube. As we get back to business with real life events and virtual, the folks who couldn't make it here, Barcelona is still a great city, obviously a great place to come and events. We'll be back. There'll be hybrid. There'll be different. Certainly the queue we'll wait doubling down, but, but we've got a great video. I want to share with the group, the Barcelona and cloud city. This is a montage of what it's like here and a little experiential video. So they get away and run that video. >>Hi, I'm Katie Goldfinch here in Barcelona for an action packed day two at telco DER's cloud setting this morning, the focus was firmly on Dr. And her MWC keynote, which told telco exactly in no uncertain terms that now is the time to act on embracing public clouds back in cloud city content ruled the day with both the cube and cloud city life stages, hosting public cloud, thought-leaders covering a wide range of topics to educate and inspire attendees and in the beautiful space of cloud city, the excitement grew throughout the day. As we streamed MW cities, exclusive keynotes from Elon Musk and preparations got underway for tonight. Star performer, Jon Bon Jovi. Wow. What an amazing day from groundbreaking keynotes into space and back to a star studded performance. Don't forget. You can catch up on anything you missed and join us for the rest of the week@cloudcitydottelcor.com or following hashtag cloud Ceci. >>Yeah, that was a great look at what's going on here in cloud city. This next video, Dr. You're going to love this. Your keynote highlights and some Bon Jovi highlights, which by the way, was the most epic thing people were packed. It was excited. It was packed. It had the security flicking, peoples counting. All the people, people are standing back. All the people from their booths are all coming in to watch. He was pumped. Let's take a look at this awesome highlight video from yesterday isolation. >>Dave, that was a highlight reel yesterday. Um, VR has got some action on stage, great messaging, um, revolution, digital revolution. >>You know your comment about how you think like Elon Musk, that's an inspiration from it. I mean, what a lot of people don't know is when you look at autonomous vehicles, remember you're driving down Palo Alto, you see one of those LIDAR things he's doing away with lidars too expensive. It's $7,000. He's taken it with cheap cameras and software down to a couple hundred bucks per vehicle. That's the way he thinks. And you're doing the same thing to telco. >>I am, I am. I'm trying to change ELCA, right? I mean, he's changing the world. He might be one of the most important humans on earth right now. I don't think I'm exactly that level, but I'm trying to become a really important person to taco. We had this great message. I think it's going to help tacos get better businesses. And I think it's a great idea. >>The folks out there watching, what is that big change you're going to drive down this cloud city street, main street of cloud city and just all about cloud. Because public clouds here, it's going to become hybrid dynamics, operating models, and changing. What is the key message that you'd like to send me? >>I think all of the software in telco needs to be written. And that's how many millions of lines of code is that. And it's going to shrunk down and put out on the public cloud and rewritten using the software Legos of the public cloud. That is a big undertaking. No, one's working on it. I'm working on it. I'm doing it. >>Let's go do it. Let's do it. If you look out a couple of years, what would be a successful? What does checkmate look like in this >>I'm winning hashtag. I mean, I think it takes, again, it takes singular focus like Elon Musk on Mars. So when these to singularly focused on getting to the public cloud and you can't sit there and protect your old business models, your, you know, uh, CR revenue, if you're Amdocs, right? Give that up. When they start to give up their CR revenue to focus on public cloud, then they'll be okay. There's there's a worthy adversary out there really. >>I mean the late clay Christianson had all the same things. Innovator's dilemma. You just get stuck here. What do you do? You kill your own debris. You eat your own to bring in the new, I mean, all these things are going on and this is, this is a huge test. >>Have to be willing to burn some boats. >>I think it's transparency, simplicity, and the consumer saying, Hey, this is a great experience. That's the thing. Yeah. Right. And that's what we're going to see. Consumers >>Love their telco. I can't wait for that. I want to love my Netflix. Yes, exactly. >>We'd love you because you've got a bold vision. You putting it out there and you're driving it. You're walking the talk. Congratulations. And again, cloud cities, a home run. Great success. Thanks for >>Having me. It's always super fun. >>Okay. Cubes coverage here. And remember we're here getting all the action and it's all going to go online after the synchronous consumption. But right now it's all about mobile world Congress and cloud city. This is the action. And of course, Adam in cloud city studio was waiting for us and you're going to take it from here.

hey welcome to analytics unleashed i'm robert christensen your host today thank you for joining us today we have three quick wins that drive big gains in the enterprise workloads and today we have olaf with erickson we have john with orok and we have dragon with dxc welcome thank you for joining me gentlemen yeah good to be here thank you thank you good to have you hey olaf let's start off with you what big problems are you trying to solve today that are doing for those quick wins what are you trying to do today top top of mind yeah when we started looking into this microservices for our financial platform we immediately saw the challenges that we have and we wanted to have a strong partner and we have a good relationship with hp before so we turned to hp because we know that they have the technical support that we need the possibilities that we need in our platform to fulfill our requirements and also the reliability that we would need so tell me i think this is really important you guys are starting into a digital wallet space that correct yeah that's correct so we are in a financial platform so we are spanning across the world and delivering our financial services to our end customers well that's not classically what you hear about ericsson diving into what's really started you guys down that path and specifically these big wins around this digitization no what what we could see earlier was that we have a mobile networks right so we have a lot of a strong user base within them uh both kind of networks and in the where we started in the emerging markets uh you normally they have a lot of unbanked people and that people also were the ones that you want to target so be able to instead of going down and use your cash for example to buy your fruits or your electricity bill etc you could use your mobile wallet and and that's how it all started and now we're also turning into the emerged markets also like the western side part of worlds etc that's fantastic and i hey i want to talk to john here john's with o'rock and he's the one of those early adopters of those container platforms for the uh in the united states here the federal government tell us a little bit about that program and what's going on with that john yeah sure absolutely appreciate it yeah so with orock what we've done is we developed one of the first fedramp authorized container platforms that runs in our moderate and soon to be high cloud and what that does is building on the israel platform gave us the capability of offering customers both commercial as well as federal the capability and the flexibility of running their workloads in a you know as a service model where they can customize and typically what customers have to do is they have to either build it internally or if they go to the cloud they have to be able to take what resources are available then tweak to those designs to make what they need so in this architecture built on open source and with our own infrastructure we offer you know very low cost zero egress capability but the also the workload processing that they would need to run data analytics machine language and other types of high performance processing that typically they would need as we move forward in this computer age so john you you touched on a topic that's i think is really critical and you had mentioned open source why is open source a key aspect for this transformation that we're seeing coming up in like the next decade yeah sure yeah with open source we shifted early on to the company to move to open source only to offer the flexibility we didn't want to be set on one particular platform to operate within so we took and built the cloud infrastructure we went with open source as an open architecture that we can scale and grow within because of that we were one of the very first fedramp authorizations built on open source not on a specific platform and what we've seen from that is the increased performance capability that we would get as well as the flexibility to add additional components that typically you don't get on other platforms so it was a it was a good move we went with and one that the customer will definitely benefit from that that's that's huge actually because performance leads to better cost and better cost leads better performance around that i i'm just super super happy with all the advanced work that you always are doing there is fantastic and dragon so so you're in a space that i think is really interesting you're dealing with what everybody likes to talk about that's autonomous vehicles you're working with automobile manufacturers you're dealing with data at a scale that is unprecedented can you just open that door for us to talk to about these big big wins that you're trying to get over the line with these enterprises yeah absolutely and um thank you robert we approach uh leveraging esmeral from the data fabric angle we practically have a fully integrated the esmeral data fabric into our robotic drive solution rewarding drive solution is actually a game changer as you've mentioned in accelerating the development of autonomous driving vehicles it's a an end-to-end hyper-scale machine learning and ai platform as i mentioned based on the esmeralda data fabric which is used by the some of the largest manufacturers in the world for development of their autonomous driving algorithms and i think we all in technology i think and following up at the same type of news and research right across the globe in in this area so we're pretty proud that we're one of the leaders in actually providing uh hyperscale machine learning platforms for uh kind manufacturers some of them i cannot talk about but bmw is one of uh one of the current manufacturers that we provide uh these type of solutions and they have publicly spoken about their uh d3 platform uh data driven development platform uh just to give you an idea um of the scale as robert mentioned uh daily we collect over 1.5 petabytes of data of raw data did you say daily data daily the storage capacity is over 250 petabytes and growing uh there's over 100 000 cores and over 200 gpus in the in in the compute area um over 50 50 petabytes of data is delivered every two weeks into a hardware in loop right for testing and we have daily uh thousands of engineers and data scientists accessing the relevant data and developing machine learning models on the daily basis right part of it is the simulation right simulation cuts the cost as well as the uh time right for developing of the autonomous uh driving algorithms and uh the the simulations are taking probably 75 percent of the research uh that's being done on this platform that's amazing dragon i i i i the more i get involved with that and i've been part of these conversations with a number of the folks that are involved with it i i computer science me my geekiness my little propeller head starts coming out i might just blows my mind and i think so i'm going to pivot back over to olaf oh left so you're talking about something that is a global network of financial services okay correct and the flow of transactional typically non-relational transactional data flows to actual transactions going through you have issues of potential fraud you have issues a safety and you have multi-geographic regional problems with data and data privacy how are you guys addressing that today so so to answer that question today we have managed to solve that using the container platform to together with the data fabric but as you say we need to span across different regions we need to have the data as secure as possible because we have a lot of legal aspects to look into because if our data disappears but your money is also disappearing so it's a really important area for us with the security and the reliability of the platforms so so that's why we also went this way to make sure that we have this strong partner that could help us with this because just looking at where we are deployed in in more than 23 countries today and and we it's processing more than 900 million us dollars per day in our systems currently so it is a lot of money passing through and you need to take security in a it's as it's a very important point right it really is it really is and so uh john i mean you you uh obviously are dealing with you know a lot of folks that have three letters as acronyms around the government agencies and uh they range in various degrees of certa of security when you say fedramp i mean what could you just uh articulate why the esmerald platform was something that you selected to go to that fedrak compliant container platform because i think that's that that kind of speaks to the to the industrial strength of what we're talking about yeah it all comes down to being able to offer a product that's secure that the customers can trust and when we went with fedramp fedramp has very stringent security requirements that have monthly poems which are performance reviews and and updates that need to be done if not on a daily basis on a monthly basis so the customers there's a lot that goes on behind the scenes that they don't are able to articulate and what by selecting the hp esmerald platform for containers um one of the key strengths that we looked at was the esmo fabric and it's all about the data it's all about securing the data moving the data transferring the data and from a customer's perspective they want to be able to operate in an environment that they can trust no different than being able to turn on their lights or making sure there's water in their utilities you know containers with the israel platform built on orok's infrastructure gives that capability fedramp enables the security tied to the platform that we're able to follow so it's government uh guided which includes this and many and over hundreds of controls that typically you know the customers don't have time or the capability to address so our commercial customers benefit our federal customers you know that you discuss they're able to follow and check the box to meet those requirements and the container platform gives us a capability where now we're able to move files which we'll hear about through the optimal fabric and then we're able to run the workloads in the containers themselves and give isolation and the security element of fed wrapping esmeral gave us that capability in order to paint that environment fedramp authorized that the customers benefit from from security so they have confidence in running their workloads using their data and able to focus on their core job at hand and not worry about their infrastructure the fundamental requirement isn't it that that isolation between that compute and storage and going up a layer there in in a way that provides them a set of services that they can i wouldn't say set it and forget it but really had the confidence that what they're getting is the best performance for the dollars that they're spending uh john my hat's off to what the work that you all do in there thank you we appreciate it yeah yeah and dragon i want to i wanted to pivot a little bit here because you are primarily the the operator what i consider one of the largest data fabrics on the on the planet for that matter um and i just want to talk a little bit about the openness of our architecture right of all the multiple protocols that we support that allow for you know you know some people may have selected a different set of application deployment models and virtualization models that allow to plug into the data fabric you know it did can you talk a little bit about that yeah and i i think um in my mind right um to operate uh such a uh data fabric at scale right um there were three key elements that we were looking for right uh that we found in uh esmeralda fabric ring the first one was a speed cost and scalability right the second one was the globally distributed data lake or ability to distribute data globally and third was certainly the strength of our partnership with with hpe in this case right so if you look at the uh as well data fabric it's it's fast it's cost effective and it's certainly highly scalable because we as you just mentioned stretch the uh sort of the capabilities of the data fabric to hundreds of petabytes and over a million the data points if you will and it important what was important for us was that the esmeralda fabric actually eliminates the need for multiple vendor solutions which would be otherwise required right because it provides integrated file system database or or a data lake right and the data management on top of it right usually you would probably need to incorporate multiple tools right from different vendors and the file system itself it's it's so important right when you're working at scale like this right and honestly in our research maybe there are three file systems in the world that can support uh this kind of size of the auto data fabric the distributed data lake was also important to us and the reason for that is you can imagine that these large car manufacturers are testing and have testing vehicles all around the world right they're not just doing it locally around the uh their data their id centers right so uh collecting the data and this 1.5 petabytes example right uh for for bmw on a daily basis it's it's it's really challenging unless you have the ability to actually leverage the data in a distributed data like fashion right so data can basically reside in different data centers globally or even on-premise and in cloud environments which became uh very important later because a lot of this car manufacturers actually have oems right that would like to get either portions of the data or get access to the data in a in different environments not necessarily in their data center um and truly i think uh to build something at this scale right uh you you need a strong partner and we certainly had that in hpe and uh we got the comprehensive support right for uh for the software um but but more importantly i think uh partner that clearly understood uh criticality of the data fabric trend and the need for the vice fast response right to our clients and you know jointly i think we met all the challenges and it's so doing i think we made the esmo data fabric a much better and stronger product over the over the last few years that's fantastic thank you dragon appreciate it uh hey so if we're going to wrap up here any last words olaf do you want to share with us no looking forward now in from our perspective on helping out with the kobe 19 situation that we have uh enabling people to still be in the market without actually touching each other and and and leaving maybe for action market and being at home etc doing those transactions that's great thank you john in last comment yeah thanks yeah uh look for uh a joint offering announcement coming up between hpe and orok where we're going to be offering sandbox as a service where the data analytics and machine language where people can actually test drive the actual environment as a service and if they like it then they can move into a production-wise environment so stay tuned for that that's great john thank you for that and hey dragon last words yeah last words um we're pretty happy what we have done already for car manufacturers we're taking this solution right in terms of the uh distributed data-like capabilities as well as the uh hyperscale machine learning and ai platform to other industries and we hope to do it jointly with you well we hope that you do it with us as well so thank you very much everybody gentlemen thank you so much for joining us i appreciate it thank you very much thank you very much hey this is robert christensen with analytics unleashed i want to thank all of our guests here today and we'll catch you next time thank you for joining us bye [Music] [Music] [Music] easy [Music] you

>>Hello, and welcome to today's event, dealing government emergency responses beyond the pandemic. This is Bob Wooley, senior fellow for the center for digital government and formerly the chief tech clerk for the state of Utah. I'm excited to serve as moderator for today's event. And just want to say, thank you for joining us. I know we're in for an informative session over the next 60 minutes before we begin a couple of brief housekeeping notes or recording of this presentation will be emailed to all registrants within 48 hours. You can use the recording for your reference or feel free to pass it along to colleagues. This webcast is designed to be interactive and you can participate in Q and a with us by asking questions at any time during the presentation, you should see a Q and a box on the bottom left of the presentation panel. >>Please send in your questions as they come out throughout the presentation, our speakers will address as many of these questions as we can during the Q and a portion of the close of our webinar today, if you would like to download the PDF of the slides for this presentation, you can do so by clicking the webinar resources widget at the bottom of the console. Also during today's webinar, you'll be able to connect with your peers by LinkedIn, Twitter and Facebook. Please use the hashtag gov tech live to connect with your peers across the government technology platform, via Twitter. At the close of the webinar, we encourage you to complete a brief survey about the presentation. We would like to hear what you think if you're unable to see with us for the entire webinar, but we're just like to complete the survey. As much as you're able, please click the survey widget at the bottom of the screen to launch the survey. Otherwise it will pop up once the webinar concludes at this time, we recommend that you disable your pop-up blockers, and if you experiencing any media player issues or have any other problems, please visit our webcast help guide by clicking on the help button at the bottom of the console. >>Joining me today to discuss this very timely topic are Karen revolt and Tim Burch, Kim Berge currently serves as the administrator of human services for Clark County Nevada. He's invested over 20 years in improving health and human service systems of care or working in the private public and nonprofit sectors. 18 of those years have been in local government in Clark County, Las Vegas, where you served in a variety of capacities, including executive leadership roles as the director of department of social services, as well as the director for the department of family services. He has also served as CEO for provider of innovative hosted software solutions, as well as chief strategy officer for a boutique public sector consulting firm. Karen real-world is the social program management offering lead for government health and human services with IBM Watson health. Karen focuses delivering exciting new offerings by focusing on market opportunities, determining unmet needs and identifying innovative solutions. >>Much of her career has been in health and human services focused on snap, TANIF, Medicaid, affordable care act, and child welfare prior to joining IBM. Karen was the senior director of product management for a systems integrator. She naturally fell in love with being a project manager. She can take her user requirements and deliver offerings. Professionals would use to make their job easier and more productive. Karen has also found fulfillment in working in health and human services on challenges that could possibly impact the outcome of people's lives. Now, before we begin our discussion of the presentation, I want to one, we'd like to learn a little more about you as an audience. So I'm going to ask you a polling question. Please take a look at this. Give us an idea of what is your organization size. I won't bother to read all these to you, but there are other a range of sizes zero to 250 up to 50,000. Please select the one that is most appropriate and then submit. >>It looks like the vast majority are zero to two 50. Don't have too many over 250,000. So this is a very, very interesting piece of information. Now, just to set up our discussion today, what I want to do is just spend just a moment and talk about the issue that we're dealing with. So when you look the COVID-19 pandemic, it's put immense pressure on States. I've been a digital state judge and had been judging a lot of the responses from States around the country. It's been very interesting to me because they bifurcate really into two principle kinds of reactions to the stress providing services that COVID environment present. One is we're in a world of hurt. We don't have enough money. I think I'm going to go home and engage as little as I have to. Those are relatively uncommon. Thankfully, most of them have taken the COVID-19 pandemic has immense opportunity for them to really do a lot more with telework, to do more with getting people, employees, and citizens involved with government services. >>And I've done some really, really creative things along the way. I find that to be a really good thing, but in many States systems have been overloaded as individuals and families throughout the country submitted just an unprecedented number of benefit applications for social services. At the same time, government agencies have had to contend with social distance and the need for a wholly different approach to engage with citizens. Um, overall most public agencies, regardless of how well they've done with technology have certainly felt some strain. Now, today we have the opportunity to go into a discussion with our speakers, have some wonderful experience in these areas, and I'm going to be directing questions to them. And again, we encourage you as you hear what they have to say. Be sure and submit questions that we can pick up later at the time. So Tim, let's start with you. Given that Las Vegas is a hub for hospitality. An industry hit severely as a result of this pandemic. How's the County doing right now and how are you prioritizing the growing needs of the County? >>Thanks Bob. Thanks for having me. Let me start off by giving just a little, maybe context for Clark County too, to our audience today. So, uh, Clark County is, you know, 85% of the state of Nevada if we serve not just as a regional County by way of service provision, but also direct municipal services. Well, if, uh, the famous Las Vegas strip is actually in unincorporated Clark County, and if we were incorporated, we would be the largest city in the state. So I say all of that to kind of help folks understand that we provide a mix of services, not just regional services, like health and human services, the direct and, and missable, uh, services as well as we work with our other five jurisdiction partners, uh, throughout the area. Uh, we are very much, um, I think during the last recession we were called the Detroit of the West. >>And, uh, that was because we're very much seen as a one industry town. Uh, so most like when the car plants, the coal plants closed back East and in the communities fuel that very rapidly, the same thing happens to us when tourism, uh, it's cut. Uh, so of course, when we went into complete shutdown and March, uh, we felt it very rapidly, not just on, uh, uh, tax receipts and collectibles, but the way in which we could deliver services. So of course our first priority was to, uh, like I think you mentioned mobilized staff. We, we mobilized hundreds of staff overnight with laptops and phones and cars and the things they needed to do to get mobile and still provide the priority services that we're mandated to provide from a safety standpoint. Um, and then we got busy working for our clients and that's really where our partnership with IBM and Watson, uh, came in and began planning that in July. And we're able to open that portal up in October to, to really speed up the way in which we're giving assistance to, to our residents. Um, re focus has been on making sure that people stay housed. We have, uh, an estimated, uh, 2.5 million residents and over 150,000 of those households are anticipated to be facing eviction, uh, as of January one. So we, we've got a, a big task ahead of us. >>All of this sounds kind of expensive. Uh, one of the common threads as you know, runs throughout government is, ah, I don't really have the money for that. I think I'd be able to afford that a diaper too, as well. So what types of funding has been made available for counties, a result of a pandemic, >>Primarily our funding stream that we're utilizing to get these services out the door has been the federal cares act. Uh, now we had some jurisdictions regionally around us and even locally that prioritize those funds in a different way. Um, our board of County commissioners, uh, took, um, a sum total of about $85 million of our 240 million that said, this will go directly to residents in the form of rental assistance and basic needs support. No one should lose their home or go hungry during this pandemic. Uh, so we've really been again working through our community partners and through our IBM tools to make sure that happens. >>So how does, how does, how does the cares act funding then support Clark County? Cause it seems to me that the needs would be complex, diverse >>Pretty much so. So as you, as folks may know him a call there's several tronches of the cares act, the original cares act funding that has come down to us again, our board, uh, identified basic needs or rental assistance and, and gave that the department of social service to go to the tunicate, uh, through the community. We then have the cares act, uh, uh, coronavirus relief funds that have, uh, impacted our CDBG and our emergency solutions grants. We've taken those. And that's what we was going to keep a lot of the programs and services, uh, like our IBM Watson portal open past January one when the cares act dollars expire. Uh, our initial response was a very manual one, uh, because even though we have a great home grown homeless management information system, it does not do financials. Uh, so we had 14 local nonprofits adjudicating, uh, this rental assistance program. >>And so we could get our social service visitor portal up, uh, to allow us to take applications digitally and run that through our program. Uh, and, uh, so those partners were obviously very quickly overwhelmed and were able to stand up our portal, uh, which for the reason we were driving so hard, even from, uh, beginning of the conversations where after going into lockdown into contracting in July and getting the portal open in October, which was an amazing turnaround. Uh, so the kudos that IBM team, uh, for getting us up and out the door so quickly, uh, was a tie in, uh, to our, uh, Curam IBM, uh, case management system that we utilize to adjudicate benefits on daily basis in Clark County for all our local indigent population, uh, and high needs folks. Uh, and then that ties into our SAP IBM platform, which gets the checks out the door. >>So what, what we've been able to do with these dollars is created in Lucian, uh, that has allowed us in the last 60 days to get as much money out the door, as our nonprofits were able go out the door in the first six months pandemic. So it really has helped us. Uh, so I'm really grateful to our board of County commissioners for recognizing the investment in technology to, to not only get our teams mobile, but to create ease of access for our constituents and our local residents to give them the help they need quickly and the way that they need it. >>Just to follow up question to that, Tim, that I'm curious about having done a lot of work like this in government, sometimes getting procurement through in a timely way is a bit challenging. How were you able to work through those issues and getting this up and provision so quickly? >>Uh, yeah, so we, we put together a, what we call a pandemic playbook, which is kind of lessons learned. And what we've seen is the folks who were essential workers in the first 60 days of the, uh, pandemic. We were able to get a lot done quickly because we were taking full advantage of the emergency. Uh, it may sound a little crass to folks not inside the service world, but it was, uh, you know, don't want you to crisis. It was things we've been planning or trying to do for years. We need them yesterday. We should have had them yesterday, but let's get them tomorrow and get it moving very quickly. Uh, this IBM procurement was something we were able to step through very quickly because of our longstanding relationship. Our countywide, uh, system of record for our financials is SAP. Uh, we've worked with Curam, uh, solution, uh, for years. >>So we've got this long standing relationship and trust in the product and the teams, which helped us build the business case of why we did it, no need to go out for competitive procurement that we didn't have time. And we needed something that would integrate very quickly into our existing systems. Uh, so that part was there. Now when the folks who were non essential came back in June and the reopening, it was whiplash, uh, the speed at which we were moving, went back to the pace of normal business, uh, which feels like hitting a wall, doing a hundred miles an hour when you're used to having that, uh, mode of doing business. Uh, so that's certainly been a struggle, uh, for all of those involved, uh, in trying to continue to get things up. Um, but, uh, once again, the teams have been great because we've probably tripled our licensure on this portal since we opened it, uh, because of working with outside vendors, uh, to, uh, literally triple the size of our staff that are processing these applications by bringing on temporary staff, uh, and short-term professionals. Uh, and so we've been able to get those things through, uh, because we'd already built the purchasing vehicle during the early onset of the crisis. >>That's very helpful. Karen, IBM has played a really pivotal role in all of this. Uh, IBM Watson health works with a number of global government agencies, raging from counties like Clark County to federal governments. What are some of the major challenges you've seen with your clients as a result of the pandemic and how is technology supporting them in a time of need and give us some background Watson health too. So we kind of know a little more about it because this is really a fascinating area. >>Yeah. Thank you, Bob. And thanks Tim for the background on Clark County, because I think Clark County is definitely also an example of what federal governments and global governments are doing worldwide today. So, um, Watson health is our division within IBM where we really focus on health and human services. And our goal is to really focus in on, um, the outcomes that we're providing to individuals and families and looking at how we use data and insights to really make that impact and that change. And within that division, we have our government health and human services area, which is the focus of where we are with our clients around social program. But it also allows us to work with, um, different agencies and really look at how we can really move the ball in terms of, um, effecting change and outcomes for, um, really moving the needle of how we can, uh, make an impact on individuals and families. >>So as we look at the globe globally as well, you know, everything that Tim had mentioned about how the pandemic has really changed the way that government agencies operate and how they do services, I think it's amazing that you have that pandemic playbook because a lot of agencies in the same way also had these set of activities that they always wanted to go and take part on, but there was no impetus to really allow for that to happen. And with the pandemic, it allowed that to kind of open and say, okay, we can try this. And unfortunately I'm in a very partial house way to do that. And, um, what Tim has mentioned about the new program that they set up for the housing, some of those programs could take a number of years to really get a program online and get through and allowing, uh, the agencies to be able to do that in a matter of weeks is amazing. >>And I think that's really gonna set a precedent as we go forward and how you can bring on programs such as the housing and capability in Canada with the economic, uh, social, um, uh, development and, and Canada need that the same thing. They actually had a multi benefit delivery system that was designed to deliver benefits for three programs. And as part of the department of fisheries and oceans Canada, the, um, the state had an emergency and they really need to set up on how they could provide benefits to the fishermen who had been at that impacted, um, from that. And they also did set up a digital front-end using IBM citizen engagement to start to allow the applications that benefits, um, and they set it up in a matter of weeks. And as I mentioned, we, uh, Clark County had a backend legacy system where they could connect to and process those applications. And this case, this is a brand new program and the case management system that they brought up was on cloud. And they had to set up a new one, but allow them to set up a, what we used to call straight through processing, I think has been now turned, turned or coined contact less, uh, processing and allowing us to really start to move those benefits and get those capabilities out to the citizens in even a faster way than has been imagined. Uh, pre pandemic. >>Karen, I have one follow-up question. I want to ask you, having had a lot of experience with large projects in government. Sometimes there's a real gap between getting to identified real requirements and then actions. How do you, how do you work with clients to make sure that process time to benefit is shortened? >>So we really focus on the user themselves and we take a human centered design focus and really prioritizing what those needs are. Um, so working with the clients, uh, effectively, and then going through agile iterations of brain, that capability out as, um, in, in a phased approach to, so the idea of getting what we can bring out that provides quality and capability to the users, and then over time starting to really roll out additional functions and, um, other, uh, things that citizens or individuals and families would need >>Very helpful. Tim, this is an interesting partnership. It's always good to see partnerships between private sector and government. Tell us a little bit about how the partnership with IBM Watson health was established and what challenges or they were brought into assist, where they brought into assist with back to requirements. Again, within the requirements definitely shifted on us. You know, we had the con looking at, uh, Watson on our child welfare, uh, side of the house that I'm responsible for and how that we could, uh, increase access to everything from tele-health to, to, uh, foster parent benefit, uh, kinship, placement benefits, all those types of things that, that right now are very manual, uh, on the child welfare side. Uh, and then the pandemic kid. And we very quickly realized that we needed, uh, to stand up a, um, a new program because, uh, a little bit for context, uh, the park County, we don't administer TANIF or Medicaid at the County level. >>It is done at the state level. So we don't have, uh, unemployment systems or Medicaid, 10 of snap benefits systems to be able to augment and enroll out. We provide, uh, the indigent supports the, the homelessness prevention, referee housing continuum of care, long-term care, really deep emergency safety net services for our County, which is a little bit different and how those are done. So that was really our focus, which took a lot of in-person investigation. We're helping people qualify for disability benefits so they can get into permanent supportive housing, uh, things that are very intensive. And yet now we have a pandemic where we need things to happen quickly because the cares act money expires at the end of December. And people were facing eviction and eviction can help spread exposure to, to COVID. Uh, so, uh, be able to get in and very rapidly, think about what is the minimal pelvis to MVP. >>What's the minimum viable product that we can get out the door that will help people, uh, entrance to a system as contactless as possible, which again was a complete one 80 from how we had been doing business. Um, and, uh, so the idea that you could get on and you have this intelligent chat bot that can walk you through questions, help you figure out if you look like you might be eligible, roll you right into an application where you can upload the few documents that we're going to require to help verify your coat would impact and do that from a smartphone and under, you know, 20 minutes. Um, it, it, it is amazing. And the fact that we've stood that up and got it out the door in 90 days, it's just amazing to me, uh, when it shows the, uh, strength of partnership. Um, I think we can, we have some shared language because we had that ongoing partnership, but we were able to actually leverage some system architects that we had that were familiar with our community and our other products. So it really helped expedite, uh, getting this, uh, getting this out to the citizens. >>So, uh, I assume that there are some complexities in doing this. So overall, how has this deployment of citizen engagement with Watson gone and how do you measure success other than you got it out quick? How do you know if it's working? >>Yeah. Right. So it's the adage of, you know, quick, fast and good, right. Um, or fast, good and cheap. So, uh, we measure success in this way. Um, how are we getting access as our number one quality measurement here? So we were able to collect, uh, about 13,000 applications, uh, manual NRC, manually folks had to go onto our website, download a PDF, fill it out, email it, or physically drop it off along with their backup. One of their choice of 14 non-profits in town, whichever is closest to them. Um, and, uh, and then wait for that process. And they were able to get 13,000 of those, uh, process for the last six months. Uh, we have, I think we had about 8,000 applications the first month come into the portal and about an equal amount of folks who could not provide the same documentation that it was needed. >>And self-selected out. If we had not had the, the tool in place, we would have had 16,000 applications, half of which would have been non-eligible would have been jamming up the system, uh, when we don't have the bandwidth to deal to deal with that, we, we need to be able to focus in on, uh, Judy Kenny applications that we believe are like a 95% success rate from the moment our staff gets them, but because we have the complex and he was on already being dependent upon the landlord, having to verify the rent amount and be willing to work with us, um, which is a major hurdle. Um, but, uh, so w we knew we could not do is go, just reinvent the manual process digitally that that would have been an abject failure on our behalf. So, uh, the ideas that, uh, folks had can go on a very, had this very intuitive conversation to the chat bot, answer some questions and find out if they're eligible. >>And then self-select out was critical for us to not only make sure that the citizens got the help they needed, but not so burnt out and overload our workforce, which is already feeling the strain of the COVID pandemic on their own personal lives and in their homes and in the workplace. Um, so that was really critical for us. So it's not just about speed, ease of access was important. Uh, the ability to quickly automate things on the fly, uh, we have since changed, uh, the area median income, a qualifier for the rental assistance, because we were able to reallocate more money, uh, to the program. So we were able to open it up to more people. We were able to make that, uh, change to the system very quickly. Uh, the idea that we can go on the home page and put updates, uh, we recognized that, uh, some of our monolingual Hispanic residents were having difficulty even with some guidance getting through the system. >>So we're able to record a, a Spanish language walkthrough and get done on the home page the next day, right into the fordable, there'll be a fine, so they could literally run the YouTube video while they're walking through their application. Side-by-side so things like that, that those are how we are able to, for us measured success, not just in the raw dollars out the door, not just in the number of applications that have come in, but our ability to be responsive when we hear from our constituents and our elected officials that, Hey, I want, I appreciate the 15,000 applications as you all, a process and record time, I've got three, four, five, six, 10 constituents that having this type of problem and be able to go back and retool our systems to make them more intuitive, to do, be able to keep them responsive for us is definitely a measure of success and all of this, probably more qualitative than here we're looking >>For, but, uh, that's for us, that's important. Actually the qualitative side is what usually gets ignored. Uh, Karen, I've got a question that's a follow up for you on the same topic. How does IBM facilitate reporting within this kind of an environment given the different needs of stakeholders, online managers and citizens? What kinds of things do you, are you able to do >>So with, um, the influx of digitalization? I think it allows us to really take a more data-driven approach to start looking at that. So, as, as Tim was mentioning, you can see where potentially users are spending more time on certain questions, or if they're stuck on a question, you can see where the abandoned rate is. So using a more data-driven approach to go in to identify, you know, how do we actually go and, um, continue to drive that user experience that may not be something that we drive directly from the users. So I would say that analytics is really, uh, I think going to continue to be a driving force as government agencies go forward, because now they are capturing the data. But one thing that they have to be careful of is making sure that the data that they're getting is the right data to give them the information, to make the right next steps and decisions. >>And Tim, you know, use a really good example with, um, the chatbot in terms of, you know, with the influx of everything going on with COVID, the citizens are completely flooded with information and how do they get the right information to actually help them decide, can I apply for this chap program? Or should I, you know, not even try and what Tim mentioned just saved the citizens, you know, the people that may not be eligible a lot of time and going through and applying, and then getting denied by having that upfront, I have questions and I need answers. Um, so again, more data-driven of how do we provide that information? And, you know, we've seen traditionally citizens having to go on multiple website, web pages to get an answer to the question, because they're like, I think I have a question in this area, but I'm not exactly sure. And they, then they're starting to hunt and hunt and hunt and not even potentially get an answer. So the chocolate really like technology-wise helps to drive, you know, more data-driven answers to what, um, whether it's a citizen, whether it's, um, Tim who needs to understand how and where my citizens getting stuck, are they able to complete the application where they are? Can we really get the benefits to, um, this individual family for the housing needs >>Too many comments on the same thing. I know you have to communicate measures of success to County executives and others. How do you do that? I mean, are you, do you have enough information to do it? Yeah, we're able to, we actually have a standup meeting every morning where the first thing I learn is how many new applications came in overnight. How many of those were completed with full documentation? How many will be ported over into our system, assigned the staff to work, where they're waiting >>On landlord verification. So I can see the entire pipeline of applications, which helps us then determine, um, Oh, it's, it's not, you know, maybe urban legend is that folks are having difficulty accessing the system. When I see really the bottleneck there, it got gotten the system fine, the bottlenecks laying with our landlord. So let's do a landlord, a town hall and iterate and reeducate them about what their responsibilities are and how easy it is for them to respond with the form they need to attest to. And so it lets us see in real time where we're having difficulties, uh, because, uh, there's a constant pressure on this system. Not just that, uh, we don't want anyone to lose their home, uh, but these dollars also go away within a December. So we've got this dual pressure of get it right and get it right now. >>Uh, and so th the ability to see these data and these metrics on, on a daily basis is critical for us to, to continue to, uh, ModuLite our response. Um, and, and not just get comfortable are baked into well, that's why we developed the flowchart during requirements, and that's just the way things are gonna stay. Uh, that's not how you respond to a pandemic. Uh, and so having a tool and a partner that helps us, uh, stay flexible, state agile, I guess, to, to, to leverage some terminology, uh, is important. And, and it's, it's paid dividends for our citizens. Karen, again, is another up to the same thing. I'm kind of curious about one of the problems of government from time to time. And Tim, I think attest to this is how do you know when Dunn has been reached? How did you go about defining what done would look like for the initial rollout with this kind of a customer? >>So I think Doug, I guess in this case, um, is, is this, isn't able to get the benefits that they're looking for and how do we, uh, you know, starting from, I think what we were talking about earlier, like in terms of requirements and what is the minimum viable, um, part of that, and then you start to add on the bells and whistles that we're really looking to do. So, um, you know, our team worked with him to really define what are those requirements. I know it's a new program. So some of those policy decisions were still also being worked out as the requirements were being defined as well. So making sure that you are staying on top of, okay, what are the key things and what do we really need to do from a compliance standpoint, from a functionality, and obviously, um, the usability of how, uh, an assistant can come on and apply and, um, have those, uh, requirements, make sure that you can meet that, that version before you start adding on additional scope. >>Very helpful. Jim, what's your comment on this since I know done matters to you? Yeah. And look, I I've lived through a, again, multiple, uh, county-wide it implementations and some department wide initiatives as well. So I think we know that our staff always want more so nothing's ever done, uh, which is a challenge and that's on our side of the customer. Um, but, uh, for this, it really was our, our experience of recognizing the, the time was an essence. We didn't have a chance. We didn't have, uh, the space to get into these endless, uh, conversations, uh, the agile approach, rather than doing the traditional waterfall, where we would have been doing requirements tracking for months before we ever started coding, it was what do we need minimally to get a check in the hands of a landlord on behalf of a client, so they don't get evicted. >>And we kept just re honing on that. That's nice. Let's put that in the parking lot. We'll come back to it because again, we want to leverage this investment long term, uh, because we've got a we, and we've got the emergency solutions and CDBG, and then our, uh, mainstream, uh, services we brought on daily basis, but we will come back to those things speed and time are of the essence. So what do we need, uh, to, to get this? So a chance to really, um, educate our staff about the concepts of agile iteration, um, and say, look, this is not just on the it side. We're gonna roll a policy out today around how you're doing things. And we may figure out through data and metrics that it's not working next week, and we'll have to have that. You want it. And you're going to get the same way. >>You're getting updated guidance from the CDC on what to do and what not to do. Uh, health wise, you're getting the same from us, uh, and really to helping the staff understand that process from the beginning was key. And, uh, so, and, and that's, again, partnering with, with our development team in that way was helpful. Um, because once we gave them that kind of charter as I am project champion, this is what we're saying. They did an equally good job of staying on task and getting to the point of is this necessary or nice. And if it wasn't necessary, we put it in the nice category and we'll come back to it. So I think that's really helpful. My experience having done several hundred sheet applications also suggest the need for MBP matters, future stages really matter and not getting caught. My flying squirrels really matters. So you don't get distracted. So let's move on to, let's do a polling question before we go on to some of our other questions. So for our audience, do you have a digital front ends for your benefit delivery? Yes, no. Or we're planning to a lot of response here yet. There we go. Looks like about half, have one and half note. So that's an interesting question. What's going to one more polling question, learn a little more here. Has COVID-19 >>Accelerated or moved cloud. Yes, no. We already run a majority of applications on cloud. Take a moment and respond if you would, please. So this is interesting. No real acceleration was taken place and in terms of moving to cloud is not what I was expecting, but that's interesting. So let's go onto another question then. And Karen, let me direct this one to you, given that feedback, how do you envision technologies such as citizen engagement and watching the system will be used, respond to emergency situations like the pandemic moving forward? I mean, what should government agencies consider given the challenges? This kind of a pandemic is brought upon government and try to tie this in, if you would, what, what is the role of cloud in all of this for making this happen in a timely way? Karen, take it away. >>Okay. Thanks Bob. So as we started the discussion around the digital expansion, you know, we definitely see additional programs and additional capabilities coming online as we continue on. Um, I think, uh, agencies have really seen a way to connect with their citizens and families and landlords, um, in this case an additional way. And he prepared them like there were, uh, presuppose assumptions that the, um, the citizens or landlords really wanted to interact with agency face-to-face and have that high touch part. And I think, um, through this, the governments have really learned that there is a way to still have an impact on the citizen without having a slow, do a face to face. And so I think that's a big realization for them to now really explore other ways to digitally explain, expand their programs and capabilities. Another area that we touched on was around the AI and chat bot piece. >>So as we start to see capabilities like this, the reason why Clark County was able to bring it up quickly and everything was because it was housed on cloud, we are seeing the push of starting to move some of the workloads. I know from a polling question perspective that it's been, um, lighter in terms of getting, uh, moving to the cloud. But we have seen the surge of really chatbots. I think we've been talking about chatbots for a while now. And, um, agencies hadn't really had the ability to start to implement that and really put it into effect. But with the pandemic, they were able to bring things up and, you know, very short amount of time to solve, um, a big challenge of not having the call center be flooded and have a different way to direct that engagement between the citizen and the government. >>So really building a different type of channel for them to engage rather than having to call or to come into an office, which wasn't really allowed in terms of, um, the pandemic. Um, the other thing I'll touch on is, um, 10 mentioned, you know, the backlog of applications that are coming in and we're starting to see the, um, the increase in automation. How do we automate areas where it's administratively highly burdened, but it's really a way that we can start to automate those processes, to give our workers the ability to focus on more of those complex situations that really need attention. So we're starting to see where the trends of trying to push there of can we automate some of those processes, um, uh, uploading documents and verification documents is another way of like, trying to look at, is there a way that we can make that easier? >>Not only for the applicant that's applying, but also for the caseworker. So there's not having to go through that. Um, does the name match, um, the applicant, uh, information and what we're looking on here, and Bob, you mentioned cloud. So behind the scenes of, you know, why, uh, government agencies are really pushing the cloud is, um, you heard about, I mean, with the pandemic, you see a surge of applicants coming in for those benefits and how do we scale for that kind of demand and how do you do that in an inappropriate way, without the huge pressures that you put on to your data center or your staff who's already trying to help our citizens and applicants, applicants, and families get the benefits they need. And so the cloud, um, you know, proposition of trying, being able to be scalable and elastic is really a key driver that we've seen in terms of, uh, uh, government agencies going to cloud. >>We haven't really seen during a pandemic, the core competencies, some of them moving those to cloud, it's really been around that digital front end, the chat bot area of how do we start to really start with that from a cloud perspective and cloud journey, and then start to work in the other processes and other areas. Um, security is also huge, uh, focus right now with the pandemic and everything going online. And with cloud allows you to be able to make sure that you're secure and be able to apply the right security so that you're always covered in terms of the type of demand and, um, impact, uh, that is coming through >>Very helpful. Tim, I'm going to ask to follow up on this of a practical nature. So you brought this up very quickly. Uh, there's a certain amount of suspicion around state government County government about chatbots. How did you get a chat much and be functional so quickly? And were you able to leverage the cloud in this process? Yeah, so on the trust is important. Uh, and I'll go back to my previous statement about individuals being able to see upfront whether they believe they're eligible or not, because nothing will erode trust more than having someone in hours applying and weeks waiting to find out they were denied because they weren't eligible to begin with, uh, that erodes trust. So being able to let folks know right up front, here's what it looks like to be eligible, actually help us build some of that, uh, cause they don't feel like, uh, someone in the bureaucracy is just putting them through the ringer for no reason. >>Um, now in regard to how do we get the chat bot out? I will say, uh, we have a, uh, dynamic it and leadership, uh, team at the highest level of County government who we have been already having conversations over the last year about what it meant to be smart government, uh, the department of social service and family services that I'm responsible for. We're already, uh, hands up first in line, you know, Guinea pigs volunteering to be on the front end of, uh, certain projects. So w we have primed ourselves for, for some of this readiness in that aspect. Um, but for citizen trust, um, the timeliness of application right now is the biggest element of trust. Uh, so I've applied I've I feel like I put my housing future in your hands. Are you going to deliver and having the ability for us to rapidly scale up? >>Uh, we typically have 120 staff in the department of social service that, that are adjudicating benefits for programs on daily basis. We've doubled that with temporary staff, uh, through some partnerships, uh, we're, we're gonna, as of next week, probably have more temporary per professional staff helping an adjudicator applications. No, do full-time County staff, because again, this rush to get the dollars out, out the door. So having a system where I can easily, uh, ramp on new users and manage them without having to be solely dependent upon an already, uh, overworked it staff who were trying to support 37 other departments in the County, um, around infrastructure needs has been greatly helpful. Sounds to me like a strong outcome focus and one that seems to work. Let's move on now to our audience questions. We're getting close to the end of our time. So let's jump into some questions from the audience. A number of you have been asking about getting copies of today's presentation within the next 48 hours. Government technology will provide all attendees with the link to the recording for your reference, or to share with colleagues. Well, let's go to our first question. So this is an interesting one. And Karen, this is for you did IBM work with other counties and States to provide digital engagement portals. >>We did Bob, uh, we've worked, um, so globally we've provided guidance on this. We work closely with New York city. They've been the integral part of the development also with our citizen engagement offering. Um, we work closely with the States. So we worked with New York city. Um, North Carolina was also another state who, um, improved their, uh, citizen engagement piece, bring up their Medicaid and snap, um, applications along with Medicaid. COVID testing along that. And I mentioned, um, the economic and social development in Canada as well. And we also work with the ministry of social development in Singapore. So a number of our customers had put up, uh, a global, uh, or sorry, a citizen engagement frontend. And during this timeframe, >>Very helpful. I don't know how much did you hear your mom provide you, but how much did it cost for initial deployment and what are the ongoing costs in other words, is this thing going to be sustainable over time? >>Yeah, absolutely. So total, uh, to date, we've spent about a $1.8 million on development implementations and licensure. A big chunk of that again has been the rapid extended of licensure, uh, for this program. Um, I think over a third of that is probably licensing because again, we need to get the dollars out and we need staff to do that and making the short term several hundred thousand dollar investment in a professional support staff and having them be able to work this portal is much cheaper than the long-term investment of bringing on a staff, printing a job, uh, during a financial difficulty that we're facing, uh, the single largest fiscal cliff let's get into that us history. Um, so it's not smart to create jobs that have a 30 year, one way to retirement, uh, inside our in unionized government environment here. So having this, the staff that would come on and do this and get out the door on these federal dollars was critical for us. Um, and there is a $800,000 a year, I believe so ongoing costs associated with licensure and, and the programming support. Uh, but once again, we're going to be moving, um, our traditional services into this digital front end. We'll be continuing this because we're, we're, we're facing, it took us, I think, six and a half, seven years to come back from the previous recession. Undoubtedly, take a little longer to get back >>From this one. Here's another interesting question, I guess really primarily Tim Tim was the solution on primarily on premise or in the cloud. >>So we'll, we've done a mix. Uh, the, and I'm starting a lot of feedbacks. I don't know if you all can hear that or not, but the, uh, I think we went on prem for, uh, some people because of the, uh, bridge into our service case manager system, which is on prem. So we did some management there. I do believe the chat bot piece of it though is in the cloud. So we're bringing it down to, from one system to the other. Uh, and, and part of that was a student negotiations and costs and worrying about what long-term is that we have a very stated goal of moving, uh, our Curam platform, which is on-prem, this is the backend. So how are we? We, we set our IBM Watson, uh, portal up, uh, and moving all of that on cloud, uh, because I mean, we've got, uh, a workforce who, uh, has the ability to retire at a very high rate over the next five years. >>And, uh, having 24 seven support in the cloud is, is as a, someone who would be called to respond to emergency situations like the is, is a much better Cod deal for, for myself and the citizen. So migrating, uh, and, um, our typical on-prem stuff up into the cloud, uh, as we continue on this, uh, evolution of what IBM Watson, uh, and the plug into our Curam, uh, system looks like Karen related question for another user is the portal provided with Clara County and others linked to other third-party backend office apps, or can it be, >>Yeah, the answer is it can be it's interoperable. So through APIs, uh, rest, uh, however, um, assistance that they need to be integrated with can definitely be integrated with, uh, like, uh, Tim mentioned, we, we went to the case management solution, but it can be integrated with other applications as well. >>Tim, did you use some other backend third party apps with yours? Uh, we did not. Uh, again, just for speed of getting, uh, this MVP solution out the door. Uh, now what we do with that on the go forward, it is going to look different and probably will include some, another practical question. Given the cares funding should be expended by December. Can this application even be employed at this late date? And you want to take a cut at that? Yeah, for us, uh, once again, we brought up earlier, um, the emergency solutions grants and the community development block grants, which have a Corona virus, uh, CV traunch, each one of those, and those have two to three year expenditure timeframes on them. Uh, so we were going to leverage those to keep this system and some of these programs going once again, that the housing needs, uh, will outstrip our capacity for years to come. >>I guess probably I should have said upfront Las Vegas has one of the worst affordable housing inventories in the nation. Uh, so we know we're going to be facing a housing issue, um, because of this for, for a long time. So we'll be using those two traunches of dollars, ESE, ESPs, uh, CV CDBG, CB funds, uh, in addition to dollars earmarked through some, uh, recreational marijuana license fees that have been dedicated to our homelessness. And when you consider this housing, uh, stability program was part of that homelessness prevention. That's our funding mix locally. Very helpful. So questions maybe for bolts for you on this one, you can probably also teach respond is the system has been set up helping the small business community. Um, this user's been canvassing and the general feeling is that small businesses have been left behind and they've been unable to access funds. What's your response on that? Karen, do you want to take that first? >>Um, yes. So in terms of, uh, the security and sorry. Um, but, uh, can you repeat the last part of that? I just missed the last part when you >>Behind it, but unable to access funds. >>Uh, yeah, so I think from a funding perspective, there's different types of, I think what Tim mentioned in terms of the cares funding, there was different types of funding that came out from a government perspective. Uh, I think there were also other grants and things that are coming out one, uh, that we're still looking at. And I think as we go into the new year, it'll be interesting to see, you know, what additional funding, um, hopefully is, is provided. Uh, but in terms of creativity, we've seen other creative ways that organizations come together to kind of, uh, help with the different agencies, to provide some, some guidance to the community, um, and helping to, uh, provide efforts and, uh, maybe looking at different ways of, um, providing, uh, some of the capabilities that the, either at the County or at the state level that they're able to leverage. But Tim happy to maybe have you chime in here too. >>Yeah. So I'll first start with my wheelhouse and I'll expand out to, to some of my partners. Uh, so the primary, small business, we knew the idea was a daily basis inside this realm is going to be landlords. Uh, so actually this afternoon, we're doing a town hall with folks to be able to roll out, uh, which they will go to our portal to find a corporate landlord program. Uh, so that I seem a landlord for Camille the application pack and on behalf of a hundred residents, rather than us having to adjudicate a hundred individual applications and melon a hundred checks. Uh, so that is because we were listening to that particular segment of the, uh, the business community. Now I know early on, we were, we were really hoping that the, the paycheck protection program federally would have, uh, been dispersed in a way that helped our local small businesses. >>Uh, more we did a, our economic development team did a round of small business supports through our cares act. Uh, our quarterly unfortunate was not open yet. It was just about 15, 20 days shy. So we use, uh, another traditional grant mechanism that we have in place to dedicate that. Uh, but on a go forward board, willing to Congress passes something over the next 30 days, um, that if there's a round two of cares or some other programs, we absolutely now have a tool that we know we can create a digital opening for individuals to come figure out if they're eligible or not for whatever program it is, the it housing, the it, uh, small business operations supports, uh, and it would apply through that process and in a very lightweight, so we're looking forward to how we can expand our footprint to help all of the needs that are present in our community. This leads to another question which may be our last one, but this is an interesting question. How can agencies use COVID-19 as a proof point providing a low cost configurable solutions that can scale across government. Karen, do you want to respond to that? And then Tim also, >>Thanks, Bob. So I believe like, you know, some of the things that we've said in terms of examples of how we were able to bring up the solution quicker, I definitely see that scaling as you go forward and trying to really, um, focus in on the needs and getting that MVP out the door. Uh, and then Tim alluded to this as well. A lot of the change management processes that went into re-imagining what these processes look like. I definitely see a additional, you know, growth mindset of how do we get better processes in place, or really focusing on the core processes so that we can really move the ball forward and continuing to go that path of delivering on a quicker path, uh, leveraging cloud, as we mentioned of, um, some, some of the capabilities around the chat bot and other things to really start to push, um, uh, the capabilities out to those citizens quicker and really reduce that timeline that we have to take on the backend side, um, that that would be our hope and goal, um, given, you know, sort of what we've been able to accomplish and hoping using that as a proof point of how we can do this for other types of, uh, either programs or other processes. >>Yeah, I think, um, the, you know, the tool has given us capability now there, whether we use local leaders leverage that to the fullest really becomes a coming upon us. So do we take a beat, uh, when we can catch our breath and then, you know, work through our executive leadership to say, look, here's all the ways you can use this tool. You've made an enterprise investment in. Um, and I know for us, uh, at Clark County, we've stood up, uh, enterprise, uh, kind of governance team where we can come and talk through all of our enterprise solutions, uh, encourage our other department head peers, uh, to, to examine how you might be able to use this. Is there a way that, um, you know, parks and rec might use this to better access their scholarship programs to make sure that children get into youth sports leagues and don't get left out, uh, because we know youth suicide on the rise and they need something positive to do when this pandemic is clear, I'm there for them to get out and do those things. >>So the possibilities really are out there. It really becomes, um, how do we mind those internally? And I know that being a part of listservs and, uh, you know, gov tech and all the magazines and things are out there to help us think about how do we better use our solutions, um, as well as our IBM partners who are always eager to say, Hey, have you seen how they're using this? Um, it is important for us to continue to keep our imaginations open, um, so that we continue to iterate through this process. Um, cause I, I would hate to see the culture of, um, iteration go away with this pandemic. >>Okay. We have time for one final question. We've already addressed this in part two, and this one is probably for you and that you've used the cares act to eliminate some of the procurement red tape that's shown up. Well, how do you somehow that's been very positive. How do you see that impacting you going forward? What happens when the red tape all comes back? >>Yeah, so I think I mentioned a little bit, uh, about that when some of the folks who are deemed non essential came back during our reopening phases and they're operating at the speed of prior business and red tape where we had all been on this, these green tape, fast tracks, uh, it, it was a bit of a organizational whiplash. Uh, but it, for us, we've had the conversation with executive management of like, we cannot let this get in the way of what our citizens need. So like keep that pressure on our folks to think differently. Don't and, uh, we've gone so far as to, uh, even, uh, maybe take it a step further and investigate what had been done in, in, in Canada. Some other places around, um, like, like going right from in a 48 hour period, going from a procurement statement through a proof of concept and doing purchasing on the backside, like how can we even get this even more streamlined so that we can get the things we need quickly, uh, because the citizens don't understand, wait, we're doing our best, uh, your number 3000 and queue on the phone line that that's not what they need to hear or want to hear during times of crisis. >>Very helpful. Well, I want to be respectful of our one hour commitment, so we'll have to wrap it up here in closing. I want to thank everyone for joining us for today's event and especially a big, thank you goes to Karen and Tim. You've done a really great job of answering a lot of questions and laying this out for us and a special thanks to our partners at IBM for enabling us to bring this worthwhile discussion to our audience. Thanks once again, and we look forward to seeing you at another government technology event,

>>Yeah. >>Welcome back for our last session of the day how to deliver career making business outcomes with Search and AI. So we're very lucky to be hearing from Canada. Canadian Tire, one of Canada's largest and most successful retailers, have been powered 4.5 1000 employees to maximize the value of data with self service insights. So today we're joining us. We have Yarrow Baturin, who is the manager of Merch analytics and planning to support at Canadian Tire and then also Andrea Frisk, who is the engagement manager manager for thoughts. What s O U R Andrea? Thanks so much for being here. And with >>that, >>I'll pass the mic to you guys. >>Thank you for having us. Um, already, I I think I'll start with an introduction off who I am, what I do. A Canadian entire on what Canadian pair is all about. So, as a manager of Merch analytics at Canadian Tire, I support merchant organization with reporting tools, and then be I platform to enable decision making on a day to day basis. What is? Canadian Tire's Canadian tire is one of the largest retailers in Canada. Um, serving Canadians with a number of lines of business spanning automotive fixing, living, playing and SNG departments. We have a number of banners, including sport check Marks Party City Phl that covers more than 1700 locations. So as an organization, we've got vast variety of different data, whether it's product or loyalty. Now, as the time goes on, the number of asks the number off data points. The complexity of the analysis has been increasing on banned traditional tools. Analytical tools such as Excel Microsoft Access do find job but start hitting their limitations. So we started on the journey of exploring what other B I platforms would be suitable for our needs. And the criteria that we thought about as we started on that journey is to make sure that we enable customization as well as the McCarthy ization of data. What does that mean? That means we wanted to ensure that each one of the end users have ability to create their own versions off the report while having consistency from the data standpoint, we also wanted Thio ensure that they're able to create there at hawks search queries and draw insights based on the desired business needs. As each one of our lines of business as each one of our departments is quite unique in their nature. And this is where thoughts about comes into play. Um, you checked off all the boxes? Um, as current customers, as potential customers, you will discover that this is the tool that allows that at hawks search ability within a matter of seconds and ability to visualize the information and create those curated pin boards for each one of the business units, depending on what the needs are. And now where? I guess well, Andrea will talk a little bit more about how we gained adoption, but the usage was like and how we, uh, implemented the tool successfully in the organization. >>Okay, so I actually used to work for Canadian tire on DSO. During that time, I helped Thio build training and engaging users to sort of really kick start our use cases. Andi, the ongoing process of adopting thought spot through Canadian Tire s 01 of the sort of reasons that we moved into using thought spot was there was a need Thio evolve, um, in order to see the wealth of data that we had coming in. So the existing reporting again. And this is this sort of standard thoughts bought fix is, um, it brings the data toe. Everyone on git makes it more accessible, so you get more out of your data. So we want to provide users with the ability to customize what they could see and personalized three information so that they could get their specific business requirements out of the data rather than relying on the weekly monthly quarterly reporting. That was all usually fairly generic eso without the ability to deep dive in. So this gave the users the agility thio optimize their campaigns, optimize product murder, urgency where products are or where there's maybe supply chain gaps. Andi just really bring this out for trillions of rose to become accessible. Thio the Canadian tire. That's what user base think. That's the slide. >>That's the slight, Um So as Andrea talked about the business use of the particular tool, let's talk a little bit about how we set it up and a wonderful journey of how it's evolved. So we first implemented 5.3 version of that spot on the Falcon server on we've been adding horsepower to it over time. Now mhm. What I want to stress is the importance off the very first, Data said. That goes into the tool toe. Actually engage the users and to gain the adoption and to make sure there is no argument whether the tool is accurate or not. So what we've started with is a key p I marked layer with all the major metrics that we have and all the available permutations and combinations off the dimensions, whether it's a calendar dimension, proud of dimension or, let's say, customer attribute now, as we started with that data set, we wanted to make sure that we're we have the ability to add and the dimensions right. So now, as we're implementing the tool, we're starting to add in more dimension tables to satisfy the needs off our clients if you want to call it that way as they want to evolve their analytics. So we started adding in some of the store attributes we started adding in some of the product attributes on when I refer to a product attributes, let's say, uh, it involves costs and involves prices involved in some of the strategic internal pieces that we're thinking about now as the comprehensive mark contains right now, in our instance, close to five billion records. This is where it becomes the one source of truth for people declaring information against right so as they go in, we also wanted to make sure when they Corey thought spot there, we're really Onley. According one source of data. One source of truth. It became apparent over time, obviously, that more metrics are needed. They might not be all set up in that particular mark. And that's when we went on the journey off implementing some of the new worksheets or some of the new data sets particularly focused on the four looking pieces. And uh, that's where it becomes important to say This is how you gain the interest and keep the interests of the public right. So you're not just implementing a number off data sets all at once and then letting the users be you're implementing pieces and stages. You're keeping the interest thio, the tool relevant. You're keeping, um, the needs of the public in mind. Now, as you can imagine on the Falcon server piece, um, adding in the horsepower capacity might become challenging the mawr. Billions of Rosie erratic eso were actually in the middle of transitioning our environment to azure in snowflake so that we can connect it. Thio embrace capability of thoughts cloud. And that's where I'm looking forward to that in 2021 I truly believe this will enable us Thio increase the speed off adoption Increase the speed of getting insights out of the tool and scale with regards Thio new data sets that we're thinking about implementing as we're continuing our thoughts about journey >>Okay, so how we drove adoption Thio 4500 plus users eso When we first started Thio approach our use case with the merchants within Canadian Tire We had meetings with these users with who are used place is gonna be with and sort of found out. What are they searching for, Where they typically looking at what existing reports are available for them. Andi kind of sought out to like, What are those things where you're pulling this on your own or someone else's pulling this data because it's not accessible yet And we really use that as our foundation to determine one what data we needed to initially bring into the system but also to sort of create those launchpad pin boards that had the base information that the users we're gonna need so that we could twofold, make it easy for them, toe adopt into the tool and also quickly start Thio, deactivate or discontinue those reports. And just like these air now only available in thought spot because with the sort of formatting within thought spot around dates, it's really easy to make this year's report last year report etcetera. Just have everything roll over every month or a recorder s. So that was kind of some of the pre work foundation when we originally did it. But really, it's been a lot of training, a lot of training. So we conducted ah, lot of in person training, obviously pre co vid eso. We've started to train the group that we targeted, which was the merchants and all of the like, surrounding support groups. Eso we had planners going in and training as well, so that everyone who was really closely connected to the merchants I had an idea of what thoughts about what was and how to use it and where the reports were, and so we just sort of rolled it out that way, and then it started to fly like wildfire. Eso the merchants start to engage with supply chain to have conversations, or the merchants were engaging with the vendors to sort of have negotiations about pricing. And they're creating these reports and getting the access to the information so quickly, and they're sharing it out that we had other groups just coming to us asking, How do I get into thoughts about how can I get in on DSO on top of those groups, we also sought out other heavy analytics groups such a supply chain where we felt like they could have the same benefits if they on boarded into thought spot with their data as well on Ben. Just continuing to evolve the training roll out. Um, you know, we continued to engage with the users, >>so >>we had a newsletter briefly Thio, sort of just keep informing users of the new data coming in or when we actually upgraded our system. So the here are the new features that you'll start seeing. We did virtual trainings and maintaining an F A Q document with the incoming questions from the users, and then eventually evolved into a self guided learning so that users that were coming to a group, or maybe we've already done a full rollout could come in and have the opportunity to learn how to use thought spot, have examples that were relevant to the business and really get started. Eso then each use case sort of after our initial started to build into a formula of the things that we needed to have. So you need to understand it. Having SMEs ready and having the database Onda worksheets built out sort of became the step by step path to drive adoption. Um, from an implementation timeline, I think they're saying, Took about two months and about half of that waas Kenny entire figuring out how figuring out our security, how to get the data in on, Do we need the time to set up the environment and get on Falcon? So then, after that initial two months, then each use case that we come through. Generally, we've got users trained and SMEs set up within about 2 to 3 weeks after the data is ingested. It's not obviously, once snowflakes set up on the data starts to get into that and the data feeds in, then you're really just looking at the 2 to 3 weeks because the data is easily connected in, >>um, no. All right, let's talk about some of the use cases. So we started with what data we've implemented. Andrea touched upon what Use a training look like what the back curate that piece wants. Now let's talk a little bit about use cases and how we actually leverage thoughts bought together the insights. So the very first one is ultimately the benefit of the tool to the entire organization. Israel Time insights. To reiterate what Andrea said, we first implemented the tool with our buyers. They're the nucleus of any retail organization as they work with everybody within the company and as the buyer's eyes, Their responsibility to ensure both the procurement and the sales channel, um, stays afloat at the end of the day, right? So they need information on a regular basis. They needed fast. They needed timely, and they needed in a fashion that they choose to digest it. It right? Not every business is the same. Not every individual is the same. They consume digest, analyze information differently. And that's what that's what allows you to dio whether it's the search, whether it's a customized onboard, please now supply chain unexpected things. As Andrea mentioned Irish work a lot of supply chain. What is the goal of supply chain to receive product and to be able to ship that product to the stores Now, as our organization has been growing and is doing extremely well, we've actually published Q three results recently. Um, the aspect off prioritization at D C level becomes very important, And what drives some of that prioritization is the analysis around what the upcoming sales would be for specific products for specific categories. And that's where again thoughts. But is one of the tools that we've utilized recently to set our prioritization logic from both inbound and outbound us. It's right because it gives you most recent results. It gives you most granular results, depending on the business problem that you're trying to tackle. Now let's chat a little bit about covert 19 response, because this one is an extremely interesting case as a pandemic hit back in March. Um, as you can imagine, the everyday life a Canadian entire became as business unusual is our executives referred to it under business unusual. This speed and the intensity of the insights and the analytics has grown exponentially. And the speed and the intensity of the insights is driven by the fact that we were trying Thio ensure that we have the right selection of products for our Canadian customers because that's ultimately bread and butter off all of the retailers is the customers, right? So thoughts bought allowed us to have early trends off both sales and inventory patterns, where, whether we were stalking out of some of the products in specific stories of provinces, whether we saw some of the upload off different lines of business, depending on the region, ality right as pandemic hit, for example, um, gym's closed restaurants closed. So as Canadian pack carries a wide variety of different lines of business, we actually offer a wide selection of exercise equipment and accessories, cycling products as well as the kitchen appliances and kitchen accessories pieces. Right? So all of those items started growing exponentially and in certain areas more than others. And this is where thoughts about comes into play. A typical analysis on what the region ality of the sales has been over the last couple of days, which is lifetime and pandemic terms, um, could have taken days weeks for analysts to ultimately cobbled together an Excel spreadsheet. Meanwhile, it can take a couple of seconds for 12 Korean tosspot set up a PIN board that can be shared through a wide variety of individuals rather than fording that one Excel spreadsheet that gets manipulated every single time. And then you don't get the right inside. So from again merch supply chain covert response aspect of things. That spot has been one of those blessings and one of those amazing tools to utilize and improve the speed off insights, improved the speed of analytics and improve the speed of decision making that's ultimately impacting, then consumer at the store level. So Andrea talked about 4500 users that we have that number of school. But what I owe the recently like to focus on, uh, Andrew and I laughing because I think the last time we've spoken at a larger forum with the fastball community, I think we had only 500 users. That was in the beginning >>of the year in in February, we were aiming to have like 1000 >>exactly. So mission accomplished. So we've got 4500 employees now. Everybody asked me, Yeah, that's a big number, but how many times do people actually log in on a weekly or daily basis? I'm or interested in that statistic? So lately, um, we've had more than 400 users on the weekly basis. What's what's been cool lately is, uh, the exponential growth off ad hoc ways. So throughout October, we've reached a 75,000 ad hoc ways in our system and about 13,000 PIN board views. So why is that's that's significant? We started off, I would say, in January of 2020 when Andrea refers to it, I think we started off with about 40 45,000 ad hoc worries a month. So again, that was cool. But at the end of the day, we were able to thio double that amount as more people migrate to act hawk searches from PIN board views, and that's that's a tremendous phenomena, because that's what that's about is all about. So I touched upon a little bit about exercise and cycling. So these are our quarterly results for Q two, um, that have showed tremendous growth that we did not plan for, that we were able to achieve with, ultimately the individuals who work throughout the organization, whether it's the merch organization or whether it's the supply chain side of the business. But coming together and utilizing a B I platform by tools such a hot spot, we can see triple digit growth results. Eso What's next for us users at Hawks searches? That's fantastic. I would still like to get to more than 1200 people on the weekly basis. The cool number to me is if all of our lifetime users were you were getting into the tool on a weekly basis. That would be cool. And what's proven to be true is ultimately the only way to achieve it is to keep surprising and delighting them and your surprising and delighting them with the functionality of the tool. With more of the relevant content and ultimately data adding in more data, um, is again possible through ET else, and it's possible through pulling that information manually. But it's expensive, expensive not from the sense of monetary value, but it's expensive from the size time, all of those aspects of things So what I'm looking forward to is migrating our platform to azure in snowflake and being able thio scale our insights accordingly. Toe adding more data to Adam or incites more, uh, more individual worksheets and data sets for people to Korea against helps the each one of the individuals learn. Get some of the insights. Helps my team in particular be, well, more well versed in the data that we have existing throughout the organization. Um, and then now Andrea, in touch upon how we scale it further and and how each one of the individuals can become better with this wonderful >>Yeah, soas used a zero mentioned theater hawk searches going up. It's sort of it's a little internal victory because our starting platform had really been thio build the pin boards to replicate what the users were already expecting. So that was sort of how we easily got people in. And then we just cut off the tap Thio, whatever the previous report waas. So it gave them away. Thio get into the tool and understand the information. So now that they're using ad hoc really means they understand the tool. Um, then they they have the data literacy Thio access the information and use it how they need. So that's it's a really cool piece. Um, that worked on for Canadian tire. A very report oriented and heavy organization. So it was a good starting platforms. So seeing those ad hoc searches go up is great. Um, one of the ways that we sort of scaled out of our initial group and I kind of mentioned this earlier I sort of stepped on my own toes here. Um is that once it was a proven success with the merchants and it started to spread through word of mouth and we sought out the analyst teams. Um, we really just kept sort of driving the insights, finding the data and learning more about the pieces of the business. As you would like to think he knows everything about everything. He only knows what he knows. Eso You have to continue to cultivate the internal champions. Um Thio really keep growing the adoption eso find this means that air excited about the possibility of using thought spot and what they can do with it. You need to find those people because they're the ones who are going to be excited to have this rapid access to the information and also to just be able to quickly spend less time telling a user had access it in thought spot. Then they would running the report because euro mentioned we basically hit a curiosity tax, right? You you didn't want to search for things or you didn't want to ask questions of the data because it was so conversed. Um, it was took too much time to get the data. And if you didn't know exactly what you were looking for, it was worse. So, you know, you wouldn't run a query and be like, Oh, that's interesting. Let me let me now run another query of all that information to get more data. Just not. It's not time effective or resource effective. Actually, at the point, eso scaling the adoption is really cultivating those people who are really into it as well. Um, from a personal development perspective, sort of as a user, I mean, one who doesn't like being smartest person in the room on bought spot sort of provides that possibility. Andi, it makes it easier for you to get recognized for delivering results on Dahlia ble insights and sort of driving the business forward. So you know, B b that all star be the Trailblazer with all the answers, and then you can just sort of find out what really like helping the organization realized the power of thought spot on, baby. Make it into a career. >>Amazing. I love love that you've joined us, Andrea. Such a such an amazing create trajectory. No bias that all of my s o heaps of great information there. Thank you both. So much for sharing your story on driving such amazing adoption and the impact that you've been able to make a T organization through. That we've got a couple of minutes remaining. So just enough time for questions. Eso Andrea. Our first questions for you from your experience. What is one thing you would recommend to new thoughts about users? >>Um, yeah, I would say Be curious and creative. Um, there's one phrase that we used a lot in training, which was just mess around in the tool. Um, it's sort of became a catchphrase. It is really true. Just just try and use it. You can't break. It s Oh, just just play around. Try it you're only limitation of what you're gonna find is your own creativity. Um, and the last thing I would say is don't get trapped by trying to replicate things. Is that exactly as they were? B, this is how we've always done it. Isin necessarily The the best move on day isn't necessarily gonna find new insights. Right. So the change forces you thio look at things from a different perspective on defined. Find new value in the data. >>Yeah, absolutely. Sage advice there. Andan another one here for Yaro. So I guess our theme for beyond this year is analytics meets Cloud Open for everyone. So, in your experience, what does What does that mean for you? >>Wonderful question. Yeah. Listen, Angela Okay, so to me, in short, uh, means scale and it means turning Yes. Sorry. No, into a yes. Uh, no, I'm gonna elaborate. Is interest is laughing at me a little bit. That's right. >>I can talk >>Fancy Two. Okay, So scale from the scale perspective Cloud a zai touched upon Throw our conversation on our presentation cloud enables your ability Thio store have more data, have access to more data without necessarily employing a number off PTL developers and going toe a number of security aspect of things in different data sources now turning a no into a yes. What does that mean with more data with more scalability? Um, the analytics possibilities become infinite throughout my career at Canadian Tire. Other organizations, if you don't necessarily have access thio data or you do not have the necessary granularity, you always tell individuals No, it's not possible. I'm not able to deliver that result. And quite often that becomes the norm, saying no becomes the norm. And I think what we're all striving towards here on this call Aziz part the conference is turning that no one say yes on then making a yes a new, uh, standard a new form. Um, as we have more access to the data, more access to the insights. So that would be my answer. >>Love it. Amazing. Well, that kind of brings in into this session. So thank you, everyone for joining us today on did wrap up this dream. Don't miss the upcoming product roadmap eso We'll be sticking around to speak thio some of the speakers you heard earlier today and I'll make the experts round table, and you can absolutely continue the conversation with this life. Q. On Q and A So you've got an opportunity here to ask questions that maybe keep you up at night. Perhaps, but yet stay tuned for the meat. The experts secrets to scaling analytics adoption after the product roadmap session. Thanks everyone. And thank you again for joining us. Guys. Appreciate it. >>Thank you. Thanks. Thanks.

>>Yeah. >>All right. Welcome back to our third session, which is all about administering analytics at Global Scale. We're gonna be discussing how you can implement security data compliance and governance across the globe at for large numbers of users to ensure thoughts. What is open for everyone across your organization? So coming right up is Cheryl Zang, who is a senior director of product management of Thought spot, and Kendrick. He threw the sports sports director of Systems Engineering. So, Cheryl and Kendrick, the floor is yours. >>Thank you, Tina, for the introduction. So let's talk about analytics scale on. Let's understand what that is. It's really three components. It's the access to not only data but its technology, and we start looking at the intersection of that is the value that you get as an organization. When you start thinking about analytics scale, a lot of times we think of analysts at scale and we look at the cloud as the A seven m for it, and that's a That's an accurate statement because people are moving towards the cloud for a variety of reasons. And if you think about what's been driving, it has been the applications like Salesforce, Forcados, Mongo, DB, among others. And it's actually part of where we're seeing our market go where 64% of the company's air planning to move their analytics to the cloud. And if you think of stock spotted specifically, we see that vast majority of our customers are already in the cloud with one of the Big Four Cloud Data warehouses, or they're evaluated. And what we found, though, is that even though companies are moving their analytics to the cloud, we have not solved. The problem of accessing the data is a matter of fact. Our customers. They're telling us that 10 to 25% of that data warehouse that they're leveraging, they've moved and I'm utilizing. And if you look at in General, Forrester says that 60 to 73% of data that you have is not being leveraged, and if we think about why you go through, you have this process of taking enterprise data, moving it into these cubes and aggregates and building these reports dashboards. And there's this bottleneck typically of that be I to and at the end of the day, the people that are getting that data on the right hand side or on Lee. Anywhere from 20 to 30% of the population when companies want to be data driven is 20 to 30% of the population. Really what you're looking for now it's something north of that. And if you think of Cloud data, warehouse is being the the process and you bring Cloud Data Warehouse and it's still within the same framework. You know? Why invest? Why invest and truly not fix the problem? And if you take that out and your leverage okay, you don't necessarily have the You could go directly against the warehouse, but you're still not solving the reports and dashboards. Why investing truly not scale? It's the three pillars. It's technology, it's data, and it's a accessibility. So if we look at analytics at scale, it truly is being able to get to that north of the 20 to 30% have that be I team become enablers, often organization. Have them be ableto work with the data in the Cloud Data warehouse and allow the cells marking finding supplies and then hr get direct access to that. Ask their own questions to be able to leverage that to be able to do that. You really have to look at your modern data architecture and figure out where you are in this maturity, and then they'll be able to build that out. So you look at this from the left to right and sources. It's ingestion transformation. It's the storage that the technology brains e. It's the data from a historical predictive perspective. And then it's the accessibility. So it's technology. It's data accessibility. And how do you build that? Well, if you look at for a thought to spot perspective, it truly is taking and driving and leveraging the cloud data warehouse architectures, interrogated, essay behind it. And then the accessibility is the search answers pen boards and embedded analytics. If you take that and extend it where you want to augment it, it's adding our partners from E T L R E L t. Perspective like al tricks talent Matile Ian Streaming data from data brings or if you wanna leverage your cloud, data warehouses of Data Lake and then leverage the Martin capability of your child data warehouse. The augmentation leveraging out through its data bricks and data robot. And that's where your data side of that pillar gets stronger, the technologies are enabling it. And then the accessibility from the output. This thought spot. Now, if you look at the hot spots, why and how do we make this technology accessible? What's the user experience we are? We allow an organization to go from 20 to 30% population, having access to data to what it means to be truly data driven by our users. That user experience is enabled by our ability to lead a person through the search process. There are search index and rankings. This is built for search for corporate data on top of the Cloud Data Warehouse. On top of the data that you need to be able to allow a person who doesn't understand analytics to get access to the data and the questions they need to answer, Arcuri Engine makes it simple for customers to take. Ask those questions and what you might think are not complex business questions. But they turn into complex queries in the back end that someone who typically needs to know that's that power user needs to know are very engine. Isolate that from an end user and allows them to ask that question and drive that query. And it's built on an architecture that allows us to change and adapt to the types of things. It's micro services architecture, that we've not only gone from a non grim system to our cloud offering, in a matter of of really true these 23 years. And it's amazing the reason why we can do that, do that and in a sense, future proof your investment. It's because of the way we've developed this. It's wild. First, it's Michael Services. It's able to drive. So what this architecture ER that we've talked about. We've seen different conversations of beyond its thought spot everywhere, which allows us to take that spot. One. Our ability to for search for search data for auto analyzed the Monitor with that govern security in the background and being able to leverage that not only internally but externally and then being able to take thought spot modeling language for that analysts and that person who just really good at creating and let them create these models that it could be deployed anywhere very, very quickly and then taking advantage off the Cloud Data warehouse or the technology that you have and really give you accessibility the technology that you need as well as the data that you need. That's what you need to be able to administer, uh, to take analytics at scale. So what I'm gonna do now is I'm gonna turn it over to Cheryl and she's gonna talk about administration in thought spot. Cheryl, >>thank you very much Can take. Today. I'm going to show you how you can administrator and manage South Spot for your organization >>covering >>streaming topics, the user management >>data management and >>also user adoption and performance monitoring. Let's jump into the demo. >>I think the Southport Application The Admin Council provides all the core functions needed for system level administration. Let's start with user management and authentication. With the user tab. You can add or delete a user, or you can modify the setting for an existing user. For example, user name, password email. Or you can add the user toe a different group with the group's tab. You can add or delete group, or you can manage the group setting. For example, Privileges associated with all the group members, for example, can administrate a soft spot can share data with all users or can manage data this can manage data privilege is very important. It grants a user the privileges to add data source added table and worksheet, manage data for different organizations or use cases without being an at me. There is also a field called Default Pin Board. You can select a set of PIN board that will be shown toe all of the users in that group on their homepage in terms off authentication. Currently, we support three different methods local active directory and samel By default. Local authentication is enabled and you can also choose to have several integration with an external identity provider. Currently, we support actor Ping Identity, Seaside Minor or a T. F. S. The third method is integration with active directory. You can configure integration with L DAP through active directory, allowing you to authenticate users against an elder up server. Once the users and groups are added to the system, we can share pin board wisdom or they can search to ask and answer their own questions. To create a searchable data, we first need to connect to our data warehouses with embraced. You can directly query the data as it exists in the data warehouse without having to move or transfer the data. In this page, you can add a connection to any off the six supported data warehouses. Today we will be focusing on the administrative aspect off the data management. So I will close the tap here and we will be using the connections that are already being set up. Under the Data Objects tab, we can see all of the tables from the connections. Sometimes there are a lot of tables, and it may be overwhelming for the administrator to manage the data as a best practice. We recommend using stickers toe organize your data sets here, we're going to select the Salesforce sticker. This will refined a list off tables coming from Salesforce only. This helps with data, lineage and the traceability because worksheets are curated data that's based on those tables. Let's take a look at this worksheet. Here we can see the joints between tables that created a schema. Once the data analyst created the table and worksheet, the data is searchable by end users. Let's go to search first, let's select the data source here. We can see all of the data that we have been granted access to see Let's choose the Salesforce sticker and we will see all of the tables and work ship that's available to us as a data source. Let's choose this worksheet as a data source. Now we're ready to search the search Insight can be saved either into a PIN board or an answer. Okay, it's important to know that the sticker actually persist with PIN board and answers. So when the user logging, they will be able to see all of the content that's available to them. Let's go to the Admin Council and check out the User Adoption Pin board. The User Adoption Pin board contains essential information about your soft spot users and their adoption off the platform. Here, you can see daily active user, weekly, active user and monthly active user. Count that in the last 30 days you can also see the total count off the pin board and answers that saved in the system. Here, you can see that unique count off users. Now. You can also find out the top 10 users in the last 30 days. The top 10 PIN board consumers and top 10 ad hoc searchers here, you can see that trending off weekly, active users, daily, active users and hourly active users over time. You can also get information about popular pin boards and user actions in the last one month. Now let's zoom in into this chart. With this chart, you can see weekly active users and how they're using soft spot. In this example, you can see 60% of the time people are doing at Hawk search. If you would like to see what people are searching, you can do a simple drill down on quarry tax. Here we can find out the most popular credit tax that's being used is number off the opportunities. At last, I would like to show you assistant performance Tracking PIN board that's available to the ad means this PIN board contains essential information about your soft spot. Instance performance You this pimple. To understand the query, Leighton see user traffic, how users are interacting with soft spot, most frequently loaded tables and so on. The last component toe scowling hundreds of users, is a great on boarding experience. A new feature we call Search Assist helps automate on boarding while ensuring new users have the foundation. They need to be successful on Day one, when new users logging for the first time, they're presented with personalized sample searches that are specific to their data set. In this example, someone in a sales organization would see questions like What were sales by product? Type in 2020. From there are guided step by step process helps introduce new users with search ensuring a successful on boarding experience. The search assist. The coach is a customized in product Walk through that uses your own data and your own business vocabulary to take your business users from unfamiliar to near fluent in minutes. Instead of showing the entire end user experience today, I will focus on the set up and administration side off the search assist. Search Assist is easy to set up at worksheet level with flexible options for multiple guided lessons. Using preview template, we help you create multiple learning path based on department or based on your business. Users needs to set up a learning path. You're simply feeling the template with relevant search examples while previewing what the end user will see and then increase the complexity with each additional question toe. Help your users progress >>in summary. It is easy to administrator user management, data management, management and the user adoption at scale Using soft spot Admin Council Back to you, Kendrick. >>Thank you, Cheryl. That was great. Appreciate the demo there. It's awesome. It's real life data, real life software. You know what? Enclosing here? I want to talk a little bit about what we've seen out in the marketplace and some of them when we're talking through prospects and customers, what they talk a little bit about. Well, I'm not quite area either. My data is not ready or I've got I don't have a file data warehouse. That's this process. In this thinking on, we have examples and three different examples. We have a company that actually had never I hadn't even thought about analytics at scale. We come in, we talked to them in less than a week. They're able to move their data thought spot and ask questions of the billion rose in less than a week now. We've also had customers that are early adoption. They're sticking their toes in the water around the technology, so they have a lot of data warehouse and they put some data at it, and with 11 minute within 11 minutes, we were able to search on a billion rows of their data. Now they're adding more data to combine to, to be able to work with. And then we have customers that are more mature in their process. Uh, they put large volumes of data within nine minutes. We're asking questions of their data, their business users air understanding. What's going on? A second question we get sometimes is my data is not clean. We'll talk Spot is very, very good at finding that type of data. If you take, you start moving and becomes an inner door process, and we can help with that again. Within a week, we could take data, get it into your system, start asking business questions of that and be ready to go. You know, I'm gonna turn it back to you and thank you for your time. >>Kendrick and Carol thank you for joining us today and bringing all of that amazing inside for our audience at home. Let's do a couple of stretches and then join us in a few minutes for our last session of the track. Insides for all about how Canadian Tire is delivering Korean making business outcomes would certainly not in a I. So you're there

>>From around the globe. It's the queue with digital coverage of AWS reinvent 2020 sponsored by Intel, AWS, and our community partners. >>Hello, and welcome back to the cube live coverage of reinvent 2020 virtual. We're not there this year. It's the cube virtual. We are the cube virtual. I'm your host, John fro with Dave Alante and analyzing our take on the partner day. Um, keynotes and leadership sessions today was AWS APN, which is Amazon partner network global partner network day, where all the content being featured today is all about the partners and what Amazon is doing to create an ecosystem, build the ecosystem, nurture the ecosystem and reinvent what it means to be a partner. Dave, thanks for joining me today on the analysis of Amazon's ecosystem and partner network and a great stuff today. Thanks for coming on. >>Yeah, you're welcome. I mean, watch the keynote this morning. I mean, partners are critical to AWS. Look, the fact is that when, when AWS was launched, it was the developers ate it up. You know, if you're a developer, you dive right in infrastructure is code beautiful. You know, if you're mainstream it, this thing's just got more complex with the cloud. And so there's, there's a big gap right between how I, where I am today and where I want to be. And partners are critical to help helping people get there. And we'll talk about the details of specifically what Amazon did, but I mean, especially when John, when you look at things like smaller outposts, you know, going hybrid, Andy Jassy redefining hybrid, you need partners to really help you plan design, implement, manage at scale. >>Yeah. You know, one of the things I'm always, um, you know, saying nice things about Amazon, but one of the things that they're vulnerable on in my opinion is how they balanced their own SAS offerings and with what they develop in the ecosystem. This has been a constant, um, challenge and, and they've balanced it very well. Um, so other vendors, they are very clear. They make their own software, right. And they have a channel and it's kind of the old playbook. Amazon's got to reinvent the playbook here. And I think that's, what's key today on stage Doug Yom. He's the, uh, the leader you had, um, also Dave McCann who heads up marketplace and Sandy Carter who heads up worldwide public sector partners. So Dave interesting combination of three different teams, you had the classic ISV partners in the ecosystem, the cohesiveness of the world, the EMCs and so on, you had the marketplace with Dave McCann. That's where the future of procurement is. That's where people are buying product and you had public sector, huge tsunami of innovation happening because of the pandemic and Sandy is highlighting their partners. So it's partner day it's partner ecosystem, but multiple elements. They're moving marketplace where you buy programs and competencies with public sector and then ISV, all of those three areas are changing. Um, I want to get your take because you've been following ecosystems years and you've been close to the enterprise and how they buy your, >>And I think, I think John, Oh, a couple of things. One is, you know, Dave McCann was talking a lot about how CIO is one of modernize applications and they have to rationalize, and it will save some of that talk for later on, you know, Tim prophet on. But there's no question that Amazon's out to reinvent, as you said, uh, the whole experience from procurement all the way through, and, you know, normally you had to, to acquire services outside of the marketplace. And now what they're doing is bundling the services and software together. You know, it's straightforward services, implementation services, but those are well understood. The processes are known. You can pretty much size them and price them. So I think that's a huge opportunity for partners and customers to reduce friction. I think the other thing I would say is ecosystems are, are critical. >>Uh, one of the themes that we've been talking about in the cube as we've gone from a product centric world in the old days of it to a platform centric world, which has really been the last decade has been about SAS platforms and cloud platforms. And I think ecosystems are going to be a really power, the new innovation in the coming decade. And what I mean by that is look, if you're just building a service and Amazon is going to do that same service, you know, you got to keep innovating. And one of the ways you can innovate is you can build on ecosystems. There's all this data within industries, across industries, and you can through the partner network and through customer networks within industry start building new innovation around ecosystems and partners or that glue, Amazon's not going to go in. And like Jandy Jesse even said in the, uh, in his fireside chat, you know, customers will ask us for our advice and we're happy to give it to them, but frankly partners are better at that nitty gritty hardcore stuff. They have closer relationships with the customers. And so that's a really important gap that Amazon has been closing for the last, you know, frankly 10 years. And I think that to your point, they've still got a long way to go, but that's a huge opportunity in that. >>A good call out on any Jess, I've got to mention that one of the highlights of today's keynote was on a scheduled, um, Andy Jassy fireside chat. Uh, normally Andy does his keynote and then he kind of talks to customers and does his thing normally at a normal re-invent this time he came out on stage. And I think what I found interesting was he was talking about this builder. You always use the word builder customer, um, solutions. And I think one of the things that's interesting about this partner network is, is that I think there's a huge opportunity for companies to be customer centric and build on top of Amazon. And what I mean by that is, is that Amazon is pretty cool with you doing things on top of their platform that does two things serves the customer's needs better than they do, and they can make more money on and other services look at snowflake as an example, um, that's a company built on AWS. I know they've got other clouds going on, but mainly Amazon Zoom's the same way. They're doing a great solution. They've got Redshift, Amazon, Amazon's got Redshift, Dave, but also they're a customer and a partner. So this is the dynamic. If you can be successful on Amazon serving customers better than Amazon does, that's the growth hack. That's the hack on Amazon's partner network. If you could. >>I think, I think Snowflake's a really good example. You snowflake you use new Relic as an example, I've heard Andy Jess in the past use cloud air as an example, I like snowflake better because they're, they're sort of thriving. And so, but, but I will say this there's a, they're a great example of that ecosystem that we just talked about because yes, not only are they building on AWS, they're connecting to other clouds and that is an ecosystem that they're building out. And Amazon's got a lot of snowflake, I guess, unless you're the Redshift team, but, but generally speaking, Snowflake's driving a lot of business for Amazon and Andy Jesse addressed that in that, uh, in that fireside chat, he's asked that question a lot. And he said, look, we, we, we have our primary services. And at the same time we want to enable our partners to be successful. And snowflake is a really good example of that. >>Yeah. I want to call out also, uh, yesterday. Um, I had our Monday, I should say Tuesday, December 1st, uh, Jesse's keynote. I did an interview with Jerry chin with gray lock. He's investing in startups and one of the things he observed and he pointed out Dave, is that with Amazon, if you're, if you're a full all-in in the cloud, you're going to take advantage of things that are just not available on say on premises that is data patterns, other integrations. And I think one of the things that Doug pointed out was with interoperability and integration with say things like the SAS factor that they put out there there's advantages for being in the cloud specifically with Amazon, that you can get on integrations. And I think Dave McCann teases that out with the marketplace when they talk about integrations. But the idea of being in the cloud with all these other partners makes integration and interoperability different and unique and better potentially a differentiator. This is going to become a huge deal. >>I didn't pick up on that because yesterday I thought I wasn't in the keynote. I think it was in the analyst one-on-one with, with Jesse, he talked about, you know, this notion that people, I think he was addressing multi-cloud he didn't use that term, but this notion of an abstraction layer and how it does simplify things in, in his basic, he basically said, look, our philosophy is we want to have, you know, the, the ability to go deep with the primitives and have that fine grain access, because that will give us control. A lot of times when you put in this abstraction layer, which people are trying to do across clouds, you know, it limits your ability to really move fast. And then of course it's big theme is, is this year, at the same time, if you look at a company who was called out today, like, like Octa, you know, when you do an identity management and single sign-on, you're, you're touching a lot of pieces, there's a lot of integration to your point. >>So you need partners to come in and be that glue that does a lot of that heavy lifting that needs to needs to be done. Amazon. What Jessie was essentially saying, I think to the partner network is, look, we're not going to put in that abstraction layer. You're going to you, you got to do that. We're going to do stuff maybe between our own own services like they did with the, you know, the glue between databases, but generally speaking, that's a giant white space for partner organizations. He mentioned Okta. He been talked about in for apt Aptio. This was Dave McCann, actually Cohesity came up a confluent doing fully managed Kafka. So that to me was a signal to the partners. Look, here's where you guys should be playing. This is what customers need. And this is where we're not going to, you know, eat your lunch. >>Yeah. And the other thing McCann pointed out was 200 new Dave McCann pointed out who leads these leader of the, of the marketplace. He pointed out 200 new ISP. ISV is out there, huge news, and they're going to turn already. He went, he talked with his manage entitlements, which got my attention. And this is kind of an, um, kind of one of those advantage points that it's kind of not sexy and mainstream to talk about, but it's really one of those details. That's the heavy lifting. That's a pain in the butt to deal with licensing and tracking all this compliance stuff that goes on under the covers and distribution of software. I think that's where the cloud could be really advantaged. And also the app service catalog registry that he talked about and the professional services. So these are areas that Amazon is going to kind of create automation around. >>And as Jassy always talks about that undifferentiated heavy lifting, they're going to take care of some of these plumbing issues. And I think you're right about this differentiation because if I'm a partner and I could build on top of Amazon and have my own cloud, I mean, let's face it. Snowflake is a born in the cloud, in the cloud only solution on Amazon. So they're essentially Amazon's cloud. So I think the thing that's not being talked about this year, that is probably my come up in future reinvents is that whoever can build their own cloud on top of Amazon's cloud will be a winner. And I, I talked about this years ago, data around this tier two, I call it tier two clouds. This new layer of cloud service provider is going to be kind of the, on the power law, the, the second wave of cloud. >>In other words, you're on top of Amazon differentiating with a modern application at scale inside the cloud with all the other people in there, a whole new ecosystem is going to emerge. And to me, I think this is something that is not yet baked out, but if I was a partner, I would be out there planning like hell right now to say, I'm going to build a cloud business on Amazon. I'm going to take advantage of the relationships and the heavy lifting and compete and win that way. I think that's a re redefining moment. And I think whoever does that will win >>And a big theme around reinventing everything, reinvent the industry. And one of the areas that's being reinvented as is the, you know, the VAR channel really well, consultancies, you know, smaller size for years, these companies made a ton of dough selling boxes, right? All the, all the Dell and the IBM and the EMC resellers, you know, they get big boats and big houses, but that business changed dramatically. They had to shift toward value, value, value add. So what did they do? They became VMware specialists. They came became SAP specialists. There's a couple of examples, maybe, you know, adding into security. The cloud was freaking them out, but the cloud is really an opportunity for them. And I'll give you an example. We've talked a lot about snowflake. The other is AWS glue elastic views. That's what the AWS announced to connect all their databases together. Think about a consultancy that is able to come in and totally rearchitect your big data life cycle and pipeline with the people, the processes, the skillsets, you know, Amazon's not going to do that work, but the upside value for the organizations is tremendous. So you're seeing consultancies becoming managed service providers and adding all kinds of value throughout the stack. That's really reinvention of the partnership. >>Yeah. I think it's a complete, um, channel strategy. That's different. It doesn't, it looks like other channels, but it's not, it's, it's, it's driven by value. And I think this idea of competing on value versus just being kind of a commodity play is shifting. I think the ISV and the VARs, those traditional markets, David, as you pointed out, are going to definitely go value oriented. And you can just own a specialty area because as data comes in and when, and this is interesting. And one of the key things that Andy Jassy said in his fireside chat want to ask directly, how do partners benefit when asked about his keynote, how that would translate to partners. He really kind of went in and he was kind of rambling, but he, he, he hit the chips. He said, well, we've got our own chips, which means compute. Then he went into purpose-built data store and data Lake data, elastic views SageMaker Q and QuickSight. He kind of went down the road of, we have the horsepower, we have the data Lake data, data, data. So he was kind of hinting at innovate on the data and you'll do okay. >>Well, and this is again, we kind of, I'm like a snowflake fan boy, you know, in the way you, you like AWS. But look, if you look at AWS glue elastic views, that to me is like snowflakes data cloud is different, a lot of pushing and moving a date, a lot of copying data. But, but this is a great example of where like, remember last year at reinvent, they said, Hey, we're separating compute from storage. Well, you know, of course, snowflake popularized that. So this is great example of two companies thriving that are both competitors and partners. >>Well, I've got to ask you, you know, you, you and I always say we kind of his stories, we've been around the block on the enterprise for years. Um, where do you Mark the, um, evolution of their partner? Because again, Amazon has been so explosive in their growth. The numbers have been off the charts and they've done it well with and pass. And now you have the pandemic which kind of puts on full display, digital transformation. And then Jassy telegraphing that the digital global it spend is their next kind of conquering ground, um, to take, and they got the edge exploding with 5g. So you have this kind of range and they doing all kinds of stuff with IOT, and they're doing stuff in you on earth and in space. So you have this huge growth and they still don't have their own fully oriented business model. They rely on people to build on top of Amazon. So how do you see that evolving in your opinion? Because they're trying to add their own Amazon only, we've got Redshift that competes with others. How do you see that playing out? >>So I think it's going to be specialized and, and something that, uh, that I've talked about is Amazon, you know, AWS in the old day, old days being last decade, they really weren't that solution focused. It was really, you know, serving the builders with tooling, with you, look at something like what they're doing in the call center and what they're doing at the edge and IOT there. I think they're, so I think their move up the stack is going to be very solution oriented, but not necessarily, you know, horizontal going after CRM or going after, you know, uh, supply chain management or ERP. I don't think that's going to be their play. I think their play is going to be to really focus on hard problems that they can automate through their tooling and bring special advantage. And that's what they'll SAS. And at the same time, they'll obviously allow SAS players. >>It's just reminds me of the early days when you and I first met, uh, VMware. Everybody had to work with VMware because they had a such big ecosystem. Well, the SAS players will run on top. Like Workday does like Salesforce does Infour et cetera. And then I think you and I, and Jerry Chen talked about this years ago, I think they're going to give tools to builders, to disrupt the service now is in the sales forces who are out buying companies like crazy to try to get a, you know, half, half a billion dollar, half a trillion dollar market caps. And that is a really interesting dynamic. And I think right now, they're, they're not even having to walk a fine line. I think the lines are reasonably clear. We're going up to database, we're going to do specialized solutions. We're going to enable SAS. We're going to compete where we compete, come on, partner ecosystem. And >>Yeah, I, I, I think that, you know, the Slack being bought by Salesforce is just going to be one of those. I think it's a web van moment, you know, um, you know, where it's like, okay, Slack is going to go die on Salesforce. Okay. I get that. Um, but it's, it's just, it's just, it's just, it's just old school thinking. And I think if you're an entrepreneur and if you're a developer or a partner, you could really reinvent the business model because if you're, dis-aggregating all these other services like you can compete with Salesforce, Slack has now taken out of the game with Salesforce, but what Amazon is doing with say connect, which they're promoting heavily at this conference. I mean, you hear it, you heard it on Andy Jessie's keynote, Sandy Carter. They've had huge success with AWS connect. It's a call center mindset, but it's not calling just on phones. >>It's contact that is descent, intermediating, the Salesforce model. And I think when you start getting into specialists and specialism in channels, you have customer opportunity to be valuable. And I think call center, these kinds of stories that you can stand up pretty quickly and then integrate into a business model is going to be game changing. And I think that's going to going to a lot of threat on these big incumbents, like Salesforce, like Slack, because let's face it. Bots is just the chat bot is just a call center front end. You can innovate on the audio, the transcriptions there's so much Amazon goodness there, that connect. Isn't just a call center that could level the playing field and every vertical >>Well, and SAS is getting disrupted, you know, to your, to your point. I mean, you think about what happened with, with Oracle and SAP. You had, you know, these new emerging players come up like, like Salesforce, like Workday, like service now, but their pricing model, it was all the same. We lock you in for a one-year two-year three-year term. A lot of times you have to pay up front. Now you look at guys like Datadog. Uh, you, you look at a snowflake, you look at elastic, they're disrupting the Splunks of the world. And that model, I think that SAS model is right for disruption with a consumption pricing, a true cloud pricing model. You combine that with new innovation that developers are going to attack. I mean, you know, people right now, they complain about service now pricing, they complain about Splunk pricing. They, you know, they talk about, Oh, elastic. We can get that for half the price Datadog. And so I'm not predicting that those companies service now Workday, the great companies, but they are going to have to respond much in the same way that Oracle and SAP had to respond to the disruption that they saw. >>Yeah. It's interesting. During the keynote, they'll talk about going out to the mainframes today, too. So you have Amazon going into Oracle and Microsoft, and now the mainframes. So you have Oracle database and SQL server and windows server all going to being old school technologies. And now mainframe very interesting. And I think the, this whole idea of this SAS factory, um, got my attention to Cohesity, which we've been covering Dave on the storage front, uh, Mo with the founder was on stage. I'm a data management as a service they're part of this new SAS factory thing that Amazon has. And what they talk about here is they're trying to turn ISV and VARs into full-on SAS providers. And I think if they get that right with the SAS factory, um, then that's going to be potentially game changing. And I'm gonna look at to see if what the successes are there, because if Amazon can create more SAS applications, then their Tam and the global it market is there is going to, it can be mopped up pretty quickly, but they got to enable it. They got to enable that quickly. Yeah. >>Enabling to me means not just, and I think, you know, when Jesse answered your question, I saw it in the article that you wrote about, you know, you asked them about multi-cloud and it, to me, it's not about running on AWS and being compatible with Azure and being compatible with Google. No, it's about that frankly abstraction layer that he talked about, and that's what Cohesity is trying to do. You see others trying to do it as well? Snowflake for sure. It's about abstracting that complexity away and adding value on top of the cloud. In other words, you're using the cloud for scale being really expert at taking advantage of the native cloud services, which requires is that Jessie was saying different API APIs, different control, plane, different data plane, but taking that complexity away and then adding new value on top that's white space for a lot of players there. And, and, and I'll tell you, it's not trivial. It takes a lot of R and D and it takes really smart people. And that's, what's going to be really interesting to see, shake out is, you know, can the Dell and HPE, can they go fast enough to compete with the, the Cohesity's you've got guys like CLU Mayo coming in that are, that are brand new. Obviously we talked about snowflake a lot and many others. >>I think there's going to be a huge change in expectations, experience, huge opportunity for people to come in with unique solutions. We're going to have specialty programming on the cube all day today. So if you're watching us here on the Amazon channel, you know that we're going to have an all of a sudden demand. There's a little link on our page. On the, on the, um, the Amazon reinvent virtual event platform, click here, the bottom, it's going to be a landing page, check out all the interviews as we roll them out all day. We got a great lineup, Dave, we got Nutanix pure storage, big ID, BMC, Amazon leaders, all coming in to talk today. Uh, chaos search ed Walsh, Rachel Rose, uh, Medicar Kumar, um, Mike Gill, flux, tons of great, great, uh, partners coming in and they're going to share their story and what's working for them and their new strategies. And all throughout the day, you're going to hear specific examples of how people are changing and reinventing their business development, their partnership strategies on the product, and go to market with Amazon. So really interesting learnings. We're going to have great conversations all throughout the day. So check it out. And again, everything's going to be on demand. And when in doubt, go to the cube.net, we have everything there and Silicon angle.com, uh, for all the great coverage. So >>I don't think John is, we're going to have a conversation with him. David McCann touched on this. You talked about the need for modernization and rationalization, Tim Crawford on, on later. And th this is, this is sort of the, the, uh, the call-out that Andy Jassy made in his keynote. He gave the story of that one. CIO is a good friend of his who said, Hey, I love what you're doing, but it's not going to happen on my watch. And, and so, you know, Jessie's sort of poking at that, that, uh, complacency saying, guys, you have to reinvent, you have to go fast, you have to keep moving. And so we're gonna talk a little bit about what, what does that mean to modernize applications, why the CIO is want to rationalize what is the role of AWS and its ecosystem and providing that, that, that level of innovation, and really try to understand what the next five to seven years are gonna look like in that regard. >>Funny, you mentioned, uh, Andy Jesuit that story. When I had my one-on-one conversation with them, uh, he was kind of talking about that anonymous CIO and I, if people don't know Andy, he's a big movie buff, too, right? He loves it goes to Sundance every year. Um, so I said to him, I said, this error of digital transformation, uh, is kind of like that scene in the godfather, Dave, where, um, Michael Corleone goes to Tom Hagen, Tom, you're not a wartime conciliary. And what he meant by that was is that, you know, they were going to war with the other five families. I think now I think this is what chassis pointed out is that, that this is such an interesting, important time in history. And he pointed this out. If you don't have the leadership chops to lean into this, you're going to get swept away. >>And that story about the CIO being complacent. Yeah. He didn't want to shift. And the new guy came in or gal and they, and they, and they lost three years, three years of innovation. And the time loss, you can't get that back. And during this time, I think you have to have the stomach for the digital transformation. You have to have the fortitude to go forward and face the truth. And the truth is you got to learn new stuff. So the old way of doing things, and he pointed that out very aggressively. And I think for the partners, that same thing is true. You got to look in the mirror and say, where are we? What's the opportunity. And you gotta gotta go there. If not, you can wait, be swept away, be driftwood as Pat Gelsinger would say, or lean in and pick up a, pick up a shovel and start digging the new solution. >>You know what the other interesting thing, I mean, every year when you listen to Jassy and his keynotes and you sort of experienced re-invent culture comes through and John you're live in Silicon Valley, you talked to leaders of Silicon Valley, you know, well, what's the secret of success though? Nine times out of 10, they'll talk about culture, maybe 10 times out of 10. And, and, and so that's, that comes through in Jesse's keynotes. But one of the things that was interesting this year, and it's been thematic, you know, Andy, you know, repetition is important, uh, to, to him because he wants to educate people and make sure it sticks. One of the things that's really been he's been focused on is you actually can change your culture. And there's a lot of inertia. People say, well, not on my watch. Well, it doesn't work that way around here. >>And then he'll share stories about how AWS encourages people to write papers. Anybody in the organization say we should do it differently. And, and you know, they have to follow their protocol and work backwards and all of those stuff. But I believe him when he says that they're open to what you have a great example today. He said, look, if somebody says, well, it's 10 feet and somebody else says, well, it's, it's five feet. He said, okay, let's compromise and say it's seven and a half feet. Well, we know it's not seven and a half feet. We don't want to compromise. We either want to be a 10, Oh, we want to be at five, which is the right answer. And they push that. And that that's, he gives examples like that for the AWS culture, the working backwards, the frequently asked questions, documents, and he's always pushing. And that to me is very, very important and fundamental to understanding AWS. >>It's no doubt that Andy Jassy is the best CEO in the business. These days. If you look at him compared to everyone else, he's hands down, more humble as keynote who does three hour keynotes, the way he does with no notes with no, he memorize it all. So he's competitive and he's open. And he's a good leader. I think he's a great CEO. And I think it will be written and then looked back at his story this time in history. The next, I think post COVID Dave is going to be an error. We're going to look back and say the digital transformation was accelerated. Yes, all that good stuff, people process technology. But I think we're gonna look at this time, this year and saying, this was the year that there was before COVID and after COVID and the people who change and modernize will build the winners and not, and the losers will, will be sitting still. So I think it's important. I think that was a great message by him. So great stuff. All right. We gotta leave it there. Dave, the analysis we're going to be back within the power panel. Two sessions from now, stay with us. We've got another great guest coming on next. And then we have a pair of lb talk about the marketplace pricing and how enterprises have CIO is going to be consuming the cloud in their ecosystem. This is the cube. Thanks for watching..

>>Thank you. Thank you so much for having me. I'm thrilled to be in conversation with you today. And I thought I would just kick things off with some opening reflections on this really important session theme, and then we can jump into discussion. So I'd like us to, as a starting point, um, wrestle with these buzz words, empowerment and inclusion so that we can, um, have them be more than kind of big platitudes and really have them reflected in our workplace cultures and the things that we design and the technologies that we put out into the world. And so to do that, I think we have to move beyond techno determinism and I'll explain what that means in just a minute. And techno determinism comes in two forms. The first on your left is the idea that technology automate. Um, all of these emerging trends are going to harm us are going to necessarily, um, harm humanity. >>They're going to take all the jobs they're going to remove human agency. This is what we might call the techno dystopian version of the story. And this is what Hollywood loves to sell us in the form of movies like the matrix or Terminator. The other version on your right is the techno utopian story that technologies automation, the robots, as a shorthand are going to save humanity. They're going to make everything more efficient, more equitable. And in this case, on the surface, they seem like opposing narratives, right? They're telling us different stories. At least they have different endpoints, but when you pull back the screen and look a little bit more closely, you see that they share an underlying logic, that technology is in the driver's seat and that human beings, that social society can just respond to what's happening. But we don't, I really have a say in what technologies are designed. >>And so to move beyond techno determinism, the notion that technology is in the driver's seat, we have to put the human agents and agencies back into the story protagonists and think carefully about what the human desires, worldviews values assumptions are that animate the production of technology. We have to put the humans behind the screen back into view. And so that's a very first step in when we do that. We see as was already mentioned that it's a very homogenous group right now in terms of who gets the power and the resources to produce the digital and physical infrastructure that everyone else has to live with. And so, as a first step, we need to think about how to, to create more participation of those who are working behind the scenes to design technology. Now, to dig a little more deeper into this, I want to offer a kind of low tech example before we get to the more high tech ones. >>So what you see in front of you here is a simple park bench public it's located in Berkeley, California, which is where I went to graduate school. And on this one particular visit, I was living in Boston. And so I was back in California, it was February, it was freezing where I was coming from. And so I wanted to take a few minutes in between meetings to just lay out in the sun and soak in some vitamin D. And I quickly realized actually I couldn't lay down on the bench because of the way it had been designed with these arm rests at intermittent intervals. And so here I thought, okay, th th the armrests have a functional reason why they're there. I mean, you could literally rest your elbows there, or, um, you know, it can create a little bit of privacy of someone sitting there that you don't know. >>Um, when I was nine months pregnant, it could help me get up and down or for the elderly the same thing. So it has a lot of functional reasons, but I also thought about the fact that it prevents people who are, are homeless from sleeping on the bench. And this is the Bay area that we're talking about, where in fact, the tech boom has gone hand in hand with a housing crisis. Those things have grown in tandem. So innovation has grown with inequity because we have, I haven't thought carefully about how to address the social context in which technology grows and blossoms. And so I thought, okay, this crisis is growing in this area. And so perhaps this is a deliberate attempt to make sure that people don't sleep on the benches by the way that they're designed and where the, where they're implemented. And so this is what we might call structural inequity, by the way something is designed. >>It has certain yeah. Affects that exclude or harm different people. And so it may not necessarily be the intent, but that's the effect. And I did a little digging and I found, in fact, it's a global phenomenon, this thing that architect next call, hostile architecture around single occupancy, benches and Helsinki. So only one booty at a time, no Nolan down there. I've found caged benches in France. Yeah. And in this particular town, what's interesting here is that the mayor put these benches out in this little shopping Plaza and within 24 hours, the people in the town rally together and have them removed. So we see here that just because we, we have a discriminatory design in our public space, doesn't mean we have to live with it. We can actually work together to ensure that our public space reflects our better values. But I think my favorite example of all is the metered bench. >>And then this case, this bench is designed with spikes in them and to get the spikes to retreat into the bench, you have to feed the meter. You have to put some coins in, and I think it buys you about 15, 20 minutes, then the spikes come back up. And so you will be happy to know that in this case, uh, this was designed by a German artist to get people to think critically about issues of design, not the design of physical space, but the design of all kinds of things, public policies. And so we can think about how our public life in general is metered, that it serves those that can pay the price and others are excluded or harmed. Whether we're talking about education or healthcare. And the meter bench also presents something interesting for those of us who care about technology, it creates a technical fix for a social problem. >>In fact, it started out as art, but some municipalities in different parts of the world have actually adopted this in their public spaces, in their parks in order to deter so-called loiters from using that space. And so by a technical fix, we mean something that creates a short-term effect, right? It gets people who may want to sleep on it out of sight. They're unable to use it, but it doesn't address the underlying problems that create that need to sleep outside of the first place. And so, in addition to techno determinism, we have to think critically about technical fixes, that don't address the underlying issues that the tech tech technology is meant to solve. And so this is part of a broader issue of discriminatory design, and we can apply the bench metaphor to all kinds of things that we work with, or that we create. >>And the question we really have to continuously ask ourselves is what values are we building in to the physical and digital infrastructures around us? What are the spikes that we may unwittingly put into place? Or perhaps we didn't create the spikes. Perhaps we started a new job or a new position, and someone hands us something, this is the way things have always been done. So we inherit the spiked bench. What is our responsibility? When we notice that it's creating these kinds of harms or exclusions or technical fixes that are bypassing the underlying problem, what is our responsibility? All of this came to a head in the context of financial technologies. I don't know how many of you remember these high profile cases of tech insiders and CEOs who applied for apples, >>The Apple card. And in one case, a husband and wife applied, and the husband, the husband received a much higher limit, almost 20 times the limit as his, >>His wife, even though they shared bank accounts, they lived in common law state. Yeah. >>And so the question there was not only the fact that >>The husband was receiving a much better rate and a high and a better >>The interest rate and the limit, but also that there was no mechanism for the individuals involved to dispute what was happening. They didn't even know how, what the factors were that they were being judged that was creating this form of discrimination. So >>In terms of financial technologies, it's not simply the outcome, that's the issue, or that can be discriminatory, >>But the process that black box is all of the decision-making that makes it so that consumers and the general public have no way to question it, no way to understand how they're being judged adversely. And so it's the process, not only the product that we have to care a lot about. And so the case of the Apple card is part of a much broader phenomenon >>Of, um, races >>And sexist robots. This is how the headlines framed it a few years ago. And I was so interested in this framing because there was a first wave of stories that seemed to be shocked at the prospect, that technology is not neutral. Then there was a second wave of stories that seemed less surprised. Well, of course, technology inherits its creators biases. And now I think we've entered a phase of attempts to override and address the default settings of so-called racist and sexist robots for better or worse than here. Robots is just a kind of shorthand that the way that people are talking about automation and emerging technologies more broadly. And so, as I was encountering these headlines, I was thinking about how these are not problems simply brought on by machine learning or AI. They're not all brand new. And so I wanted to contribute to the conversation, a kind of larger context and a longer history for us to think carefully about the social dimensions of technology. And so I developed a concept called the new Jim code, >>Which plays on the phrase, >>Jim Crow, which is the way that the regime of white supremacy and inequality in this country was defined in a previous era. And I wanted us to think about how that legacy continues to haunt the present, how we might be coding bias into emerging technologies and the danger being that we imagine those technologies to be objective. And so this gives us a language to be able to name this phenomenon so that we can address it and change it under this larger umbrella of the new Jim code are four distinct ways that this phenomenon takes shape from the more obvious engineered inequity. Those are the kinds of inequalities tech mediated in the qualities that we can generally see coming. They're kind of obvious, but then we go down the line and we see it becomes harder to detect it's happening in our own backyards, it's happening around us. And we don't really have a view into the black box. And so it becomes more insidious. And so in the remaining couple of minutes, I'm just, just going to give you a taste of the last three of these, and then a move towards conclusion. Then we can start chatting. So when it comes to default discrimination, this is the way that social inequalities >>Become embedded in emerging technologies because designers of these technologies, aren't thinking carefully about history and sociology. A great example of this, uh, came to, um, uh, the headlines last fall when it was found that widely used healthcare algorithm, effecting millions of patients, um, was discriminating against black patients. And so what's especially important to note here is that this algorithm, healthcare algorithm does not explicitly take note of race. That is to say it is race neutral by using cost to predict healthcare needs this digital triaging system unwittingly reproduces health disparities, because on average, black people have incurred fewer costs for a variety of reasons, including structural inequality. So in my review of this study, by Obermeyer and colleagues, I want to draw attention to how indifference to social reality can be even more harmful than malicious intent. It doesn't have to be the intent of the designers to create this effect. >>And so we have to look carefully at how indifference is operating and how race neutrality can be a deadly force. When we move on to the next iteration of the new Jim code, coded exposure, there's a tension because on the one hand, you see this image where the darker skin individual is not being detected by the facial recognition system, right on the camera, on the computer. And so coded exposure names, this tension between wanting to be seen and included and recognized whether it's in facial recognition or in recommendation systems or in tailored advertising. But the opposite of that, the tension is with when you're over, it >>Included when you're surveilled, when you're >>Too centered. And so we should note that it's not simply in being left out, that's the problem, but it's in being included in harmful ways. And so I want us to think carefully about the rhetoric of inclusion and understand that inclusion is not simply an end point, it's a process, and it is possible to include people in harmful processes. And so we want to ensure that the process is not harmful for it to really be effective. The last iteration of the new Jim code. That means the, the most insidious let's say is technologies that are touted as helping us address bias. So they're not simply including people, but they're actively working to address bias. And so in this case, there are a lot of different companies that are using AI to hire, uh, create hiring, um, software and hiring algorithms, including this one higher view. >>And the idea is that there there's a lot that, um, AI can keep track of that human beings might miss. And so, so the software can make data-driven talent decisions after all the problem of employment discrimination is widespread and well-documented, so the logic goes, wouldn't this be even more reason to outsource decisions to AI? Well, let's think about this carefully. And this is the idea of techno benevolence, trying to do good without fully reckoning with what, how technology can reproduce inequalities. So some colleagues of mine at Princeton, um, tested a natural learning processing algorithm and was looking to see whether it exhibited the same, um, tendencies that psychologists have documented among humans. And what they found was that in fact, the algorithm associated black names with negative words and white names with pleasant sounding words. And so this particular audit builds on a classic study done around 2003 before all of the emerging technologies were on the scene where two university of Chicago economists sent out thousands of resumes to employers in Boston and Chicago. >>And all they did was change the names on those resumes. All of the other work history education were the same. And then they waited to see who would get called back and the applicants, the fictional applicants with white sounding names received 50% more callbacks than the, the black applicants. So if you're presented with that study, you might be tempted to say, well, let's let technology handle it since humans are so biased. But my colleagues here in computer science found that this natural language processing algorithm actually reproduced those same associations with black and white names. So two with gender coded words and names as Amazon learned a couple years ago, when its own hiring algorithm was found discriminating against women, nevertheless, it should be clear by now why technical fixes that claim to bypass human biases are so desirable. If only there was a way to slay centuries of racist and sexist demons with a social justice bot beyond desirable, more like magical, magical for employers, perhaps looking to streamline the grueling work of recruitment, but a curse from any job seekers as this headline puts it. >>Your next interview could be with a racist bot, bringing us back to that problem space. We started with just a few minutes ago. So it's worth noting that job seekers are already developing ways to subvert the system by trading answers to employers tests and creating fake applications as informal audits of their own. In terms of a more collective response. There's a Federation of European trade unions call you and I global that's developed a charter of digital rights for workers that touches on automated and AI based decisions to be included in bargaining agreements. And so this is one of many efforts to change the ecosystem, to change the context in which technology is being deployed to ensure more protections and more rights for everyday people in the U S there's the algorithmic accountability bill that's been presented. And it's one effort to create some more protections around this ubiquity of automated decisions. >>And I think we should all be calling for more public accountability when it comes to the widespread use of automated decisions. Another development that keeps me somewhat hopeful is that tech workers themselves are increasingly speaking out against the most egregious forms of corporate collusion with state sanctioned racism. And to get a taste of that, I encourage you to check out the hashtag tech, won't build it among other statements that they've made and walking out and petitioning their companies. One group said as the, at Google at Microsoft wrote as the people who build the technologies that Microsoft profits from, we refuse to be complicit in terms of education, which is my own ground zero. Um, it's a place where we can, we can grow a more historically and socially literate approach to tech design. And this is just one resource that you all can download, um, by developed by some wonderful colleagues at the data and society research Institute in New York. >>And the, the goal of this intervention is threefold to develop an intellectual understanding of how structural racism operates and algorithms, social media platforms and technologies not yet developed and emotional intelligence concerning how to resolve racially stressful situations within organizations and a commitment to take action, to reduce harms to communities of color. And so as a final way to think about why these things are so important, I want to offer, uh, a couple last provocations. The first is pressed to think a new about what actually is deep learning when it comes to computation. I want to suggest that computational depth when it comes to AI systems without historical or social depth is actually superficial learning. And so we need to have a much more interdisciplinary, integrated approach to knowledge production and to observing and understanding patterns that don't simply rely on one discipline in order to map reality. >>The last provocation is this. If as I suggested at the start in the inequity is woven into the very fabric of our society. It's built into the design of our, our policies, our physical infrastructures, and now even our digital infrastructures. That means that each twist coil and code is a chance for us to weave new patterns, practices, and politics. The vastness of the problems that we're up against will be their undoing. Once we, that we are pattern makers. So what does that look like? It looks like refusing colorblindness as an anecdote to tech media discrimination, rather than refusing to see difference. Let's take stock of how the training data and the models that we're creating. Have these built in decisions from the past that have often been discriminatory. It means actually thinking about the underside of inclusion, which can be targeting and how do we create a more participatory rather than predatory form of inclusion. And ultimately it also means owning our own power in these systems so that we can change the patterns of the past. If we're, if we inherit a spiked bench, that doesn't mean that we need to continue using it. We can work together to design more, just an equitable technologies. So with that, I look forward to our conversation.

[Music] cube coverage of AWS re-invent 2019 continues in a moment

>>from Santa Clara, California In the heart of Silicon Valley, it's the queue covering altitude 2020. Brought to you by aviatrix. >>Our next customer panel got great. Another set of cloud Network architect Justin Smith, was Aura Justin broadly with Ellie Mae and omit Otri job with Cooper. But on the stage. >>Yeah, all right. Thank you. Thank you, Thank you. Yeah, >>he's got all the cliff notes from the last session. Welcome, Rinse and repeat. Yeah, we got to go under the hood a little bit and I think they nailed the what we've been reporting and we've been having a conversation around. Networking is where the action is, cause that's the end of the day. You gotta move impact from a to B. And yet workloads exchanging data. So it's really killer. So let's get started. I mean, what are you seeing as the journey of of multi cloud as you go under the hood and say, Okay, I got to implement this, have to engineer the network, make it enabling make it programmable, making interoperable across clouds. And that's like I mean, almost sounds impossible to me. What's your take? >>Yeah, I mean it. It seems impossible. But if you're running an organization, which is running infrastructures, according all right, it is easily doable. Like you can use tools out there that's available today. You can use third party products that can do a better job, but But what? Your architecture first, don't wait. Architecture may not be perfect, but the best architecture that's available today and be agile to idiot and make improvements over there. >>We got to Justin's over here, so I have to be careful when I pointed Question and Justin, they both have the answer, but okay, Journeys. What's the journey been like? I mean, is there phases? We heard that from Gardner. People come into multi cloud and cloud native networking from different perspective. What's your take on the journey? Justin? >>Yeah, I mean, from our first do, we started out very much focused on one cloud. Ah, and as we started acquisitions, we started doing new products to market. The need for multi cloud becomes very apparent very quickly for us. And so, you know, having architecture that we can plug in, play into and be able to add and change things as it changes is super important For what we're doing in this space >>just in your journey? >>Yes, for us. We were very ad hoc oriented. And the idea is that we were reinventing all the time trying to move into these new things and coming up with great new ideas. And so rather than it being some iterative approach with our deployments that became a number of different deployments. And so we shifted that tour in the network has been a real enabler of this is that there's one network and it touches whatever cloud we wanted to touch on. It touches the data centers that we needed to touch, and it touches the customers that we needed to touch. Our job is to make sure that the services that are available in one of those locations are available in all of the locations. So the idea is not that we need to come up with this new solution every time. It's that we're just iterating on what we've already decided to dio >>before we get the architecture section, I want to ask you guys a question. A big fan of you know, let the app developers have infrastructure as code, so check, but having the right cloud run that workload. I'm a big fan of that if it works great. But we just heard from the other panel. You can't change the network. So I want to get your thoughts. What is cloud native networking and is that the engine really got the enabler for this multi cloud trend. But you guys take it, we'll start with What do you think about that? >>Yeah, So you're gonna have workloads running in different clouds, and the workloads would have affinity to one cloud or another. But how you expose that? It's a matter of how you're gonna build your networks, how we're gonna run security, how we're gonna do egress, ingress out offered. So >>networking is the big problem. How do you suppose this? What's the solution? That's the key Pain points and problem statement. >>I mean, they get the key pain point for most companies is how do you take your traditional on premise network and then blow it out to the cloud in a way that makes sense. I p conflicts. You have space. You have public eyepiece on premise as well as in the cloud. And how do you kind of make a sense of all of that. And I think that's where tools like aviatrix make a lot of sense >>in that space from our side. It's it's really simple. It's latency and bandwidth and availability. These don't change whether we're talking about cloud or data center or even corporate I t. Networking. So our job when when these all of these things are simplified into like s three, for instance. And our developers want to use those, we have to be able to deliver that and for a particular group or another group that wants to use just just GC p resource is these aren't we have to support these requirements and these wants, as opposed to saying, Hey, that's not a good idea. Our job is to enable them not disable them. Do >>you think I do? You guys think Infrastructure's code, which I love that because that's the future it is. We saw that with Dev Ops, but I just start getting the networking. Is it getting down to the network portion where it's network as code storage and compute? Working really well is seeing all kubernetes and service meshes. Trend network is code reality is that there is still got work >>to do. It's absolutely there. I mean, you mentioned that develops and it's very real. I mean, in Cooper, we build our networks through terraform and on not only just data from building a p I so that we can consistently build we nuts and vpc all across in the same way >>you got to do it. >>Yeah, and even security groups and then on top and aviatrix comes in. We can peer the network's bridge bridge, all the different regions through court. >>Same with you guys about >>everything we deploy is done with automation. And then we also run things like Lambda on top to make changes in real time. We don't make manual changes on our network in the data center. Funny enough, it's still manual, but the cloud has enabled us to move into this automation mindset. And all my guys, that's what they focus on is bringing now what they're doing in the cloud into the data center, which is kind of opposite what it should be that's full. When it used to be. >>It's full dev ops, then >>yes, yeah, I mean, for us, it was similar on premise. Still somewhat very manual though we're moving more Norton Ninja and Terra Form concepts, but everything in the production environment, confirmation terraform code and now coming into the data center. So I just wanted to jump in on Justin Smith. One of the comment >>that you made because it's something that we always >>talk about a lot is that >>the center of gravity of architectures used to be an on Prem, and now it's shifted in the cloud it once you have your strategic architecture what what do you do? You push that everywhere. So what you used to see the beginning of cloud was pushing the architecture on Prem into cloud. Now I won't pick up on what you said The you others agree that the center >>of architect of gravity is here. I'm now pushing what I do in the cloud >>back into on Prem. And so first that and then also in the journey Where are you at? From 0 to 100 of, actually in the journey to cloud Are you 50% There are you 10%. So you evacuating data centers next year? I mean, where are you guys at >>s. So there's there's two types of gravity that you typically are dealing with the migration. First is data gravity in your data set and where that data lives. And the second is the network platform that interrupts all that together in our case, the data gravity still mostly on Prem. But our network is now extending out to the APP tier that's gonna be in cloud, right? Eventually that data gravity will also move to cloud as we start getting more sophisticated. But, you know, in our journey, we're about halfway there halfway through the process, we're taking a handle of lift and shift. And when did that start? We started about three years ago. >>Okay, Well, for us, it's a very different story. Started from a garage 100% in the cloud. It's a business spend management platform as a software as a service, 100% in the cloud. It was like 10 years ago, right? Yes. >>Yeah. Has a riding the wave of the architecture. Just I want to ask you is or you guys mentioned Dev ops. I mean, obviously we saw the huge observe Ability way which essentially network management for the cloud. In my opinion, it's more dynamic, but this is about visibility. We heard from last panel. You don't know what's being turned on or turned off from a services standpoint at any given time. How is all this playing out when you start getting into the Dev Ops down? >>Well, this this is the big challenge for all of us is visibility when you talk transport within a cloud. You know, we very interesting. We have moved from having a backbone that we bought that we own. That would be data center connectivity. We now I work as a subscription billing company, so we want to support the subscription mindset. So rather than going and buying circuits and having to wait three months to install and then coming up with some way to get things connected and resiliency and redundancy, I my backbone, is in the cloud. I use the cloud providers interconnections between regions to transport data across. And so if you do that with their native solutions, you do lose visibility. There are areas in that you don't get, which is why controlling you know, controllers and having some type of management plane is a requirement for us to do what we're supposed to and provide consistency while doing it. >>Great conversation. I love what you said earlier laden C band with I think availability with your SIM pop three things guys s l A and just do ping times between clouds. It's like you don't know what you're getting for round trip times. This becomes a huge kind of risk management. Black hole, whatever you wanna call Blind Spot. How are you guys looking at the interconnect between clouds? Because, you know, I can see that working from ground to cloud on cloud. But when you start dealing multi clouds, workloads, sl A's will be all over the map, won't they? Just inherently. But how do you guys view that? >>Yeah, I think we talked about workload, and we know that the workloads are going to be different in different clouds, but they're going to be calling each other. So it's very important to have that visibility that you can see how their data is flowing at war latency and what our ability is is there and over a Saturday meets to operate on. So it's >>so use the software dashboard, look at the Times and look at the latest in >>the old days strong So on open salon, you try to figure it out, and then your data, as you look at our >>Justin, what's your answer? That cause you're in the middle of it? >>Yeah. I mean, I think the key thing there is that we have to plan for that failure. We have a plan for that latency and our applications. That something is we're tracking ingress, Ally, something you start planning for. And you loosely couple these services in a much more micro services approach. So you actually can handle that kind of failure or that type of unknown latency. And unfortunately, the cloud has made us much better at handling exceptions in much better way. >>You guys are all great examples of cloud native from day one you guys had When did you have the tipping point moment or the epiphany of saying a multi clouds real? I can't ignore it. I got a factor into all my design, design principles and everything you're doing. What? It was there a moment or was it from day one? >>Now there are 22 reasons. One was the business. So in business, there was some affinity to not be in one cloud or to be in one cloud, and that drove from the business side. So as a cloud architect of a responsibility was to support the business, and the other is the technology. Some things are really running better in, like if you're running dot net workload or you're gonna run machine learning or yeah, so you have You would have that reference off one cloud over the other. So >>your thoughts on that >>that was the bill that we got from AWS. I mean, that's that's what drives a lot of these conversations is the financial viability of what you're building, on top of which is so we this failure domain idea, which is which is fairly interesting. How do I solve or guarantee against a failure domain You have methodologies with, you know, back in direct, connects or interconnect with DCP. All of these ideas are something that you have to take into account. But that transport layer should not matter to whoever we're building this for Our job is to deliver the frames in the packets, what that flows across, how you get there. We want to make that seamless, and so whether it's a public Internet, AP I call or it's a back end connectivity through direct connect. It doesn't matter. It just has to meet a contract that you signed with your application, folks, >>that's the availability piece just in your thoughts on that comment >>s actually multi clouds become something much more recent. In the last 6 to 8 months, I'd say we always kind of had a very dramatic like movie to Amazon from our private cloud is hard enough. Why complicated further but the realities of the business. And as we started seeing, you know, improvements in Google and Azure and different technology space is the need for multi cloud becomes much more important as well. As other acquisition strategies matured. We're seeing that companies that used to be on premise that we typically acquire are now very much already on the cloud. And if they're on a cloud, I need to plug them into our ecosystem. And so that's really changed our multi cloud story >>in a big way. I'd love to get your thoughts on the cloud versus the cloud because you know you compare them. Amazon's got more features. They're rich with features. I'll see the bills. Are people using them. But Google's got a great network. Google's networks pretty damn good. And then you've got azure. What's the difference between the clouds? Who? Where they involved with a peak in certain areas, better than others? What? What are the characteristics which makes one cloud better? Do they have a unique feature that makes Azure better, then Google and vice versa. What you guys think about the different clouds? >>Yeah, so my experience, I think there is a deep approach is different in many places. Google has a different approach, very developer friendly. And you can run your workload with the your network and spend regions. I mean, but our application ready to accept Amazon is evolving. I mean, I remember 10 years back Amazon's network was a flat network. We will be launching certain words and 10.0 dot zero slash right on, and the repeat came out >>with Life is not good. >>So so the VPC concept came on multi core came out, so they are evolving as you already let's start. But because they have lived start, they saw the patron and they have some mature set up on the ground, >>and I think they're all trying to say they're equal in their own ways. I think they all have very specific design philosophies that allow them to be successful in different ways. And you have to kind of keep that in mind as you architect your own solution, for example, Amazon has a very much a very regional affinity. They don't like to go cross region in their architecture, whereas Google is very much it's a global network. We're gonna think about a global solution. I think Google also has evangelists third to market, and so has seen what Azure did wrong. It seemed, with AWS did wrong and it's made those improvements. And I think that's one of their big >>advantage. Great scale to Justin. Thoughts on the cloud. >>So yeah, Amazon built from the system up and Google built from the network down, so their ideas and approaches are from a global versus original. I agree with you completely that that is the big number one thing. But if you look at it from the outset, interestingly, the inability or the ability for Amazon to limit layer to broadcasting and what that really means from a VPC perspective changed all the routing protocols you can use all the things that we have built inside of a data center, provide resiliency and make things seamless to users. All of that disappeared on DSO because we had to accept that at the VPC level. Now we have to accept it. At the one level, Google's done a better job of being able to overcome those things and provide those traditional network facilities to us. >>Just scrape and go all day. Here is awesome So I heard. But we will get to the cloud native Naive question. So I kind of think about what's not even what clients that next. But I got to ask. You had a conversation with a friend. He's like, When is the new land? So if you think about what the land was in a data center, when is the new language you're talking about? The cloud impact. So that means SD win. The old SD Wan is kind of changing the new land. How do you guys look at that? Because even think about it. What lands were for inside of premises was all about networking speed. But now, when you take the win and make essentially land, do you agree with that. And how do you view this trend? Is it good or bad or ugly? What's what's your guys take on this? >>Yeah, I think it's Ah, it's a thing that you have to work with your application architect. So if you're managing networks and if you're a sorry engineer, you need to work with them toe expose the unreliability that would bring in. So the application has to handle a lot off this the difference in the latency, ease and and the reliability it has to be worked with application there >>land way and same concept as a B S >>E. I think we've been talking about for a long time the erosion of the edge and so is this is just a continuation of that journey we've been on for the last several years. As we get more and more cloud native and we talk about a p, I is the ability to lock my data in place and not be able to access. It really goes away. And so I think this is just continuation that thing. I think it has challenges. We are talking about land scale versus land scale. The tooling doesn't work the same. The scale of that tooling is much larger on the need for automation is much, much higher in a way that it wasn't a land. That's where you're seeing so much. Infrastructure is code. >>Yeah, so for me, I'll go back again to this. It's bandwidth, and it's latency, right that that define those two land versus went. But the other thing that's comes up more and more with cloud deployments is where's our security boundary? And where can I extend this secure, aware appliance or set of rules Teoh to protect what's inside of it. So for us, we're able to deliver VM ref. So our route forwarding tables for different segments wherever we're out in the world and so they're trusted to talk to each other. But if they're going to go to some place that's outside of their network, then they have to cross a security boundary, and we'll reinforce policy very heavily. So for me there is. It's not just land when it's it's how does environment get to environment more importantly, >>as a great point and security we haven't talked about yet, but that's got to be baked in from the beginning of this architecture. Thoughts on security, how you guys are dealing with it. >>Yes, start from the base. Have app, Web security built in? Have TLS have encryption on the data in transit or at rest? But as you bring the application to the cloud and they're going to go multi cloud talking toe or the Internet in some places, well, have app, Web security >>I mean our principles. Day security is a day zero every day, and so we always build it into our design, build into our architecture into our applications. It's encrypt everything. It's TLS everywhere. It's make sure that that data is security at all times. >>Yeah, one of the cool trends that are, say, just as a side note was the data in use encryption piece, which is a home or fix stuff. Interesting. Alright, guys, final question. You know, we heard on the earlier panel was also trending at reinvent. We take the tea out of cloud native. It spells cloud naive. They got shirts now aviatrix kind of got this trend going. What does that mean to be naive? So if you're to your peers out there watching the live stream and also the suppliers that are trying to supply you guys with technology and services. What's naive look like And what's native look like when it's someone naive about implementing all this stuff. >>So for me, it's because we are 100% cloud for us. Its main thing is ready for the change, and you you will find new building blocks coming in and the network design will evolve and change. So don't be naive and things that are static you all with the change. >>I think the big naivety that people have is that, well, I've been doing it this way for 20 years and been successful. It's going to be successful in cloud. The reality is, that's not the case. You have to think some of the stuff differently, and you need to think about it early enough so that you can become cloud native and really enable your business on cloud. >>Yet for me, it's it's being open minded, right? The our industry, the network industry as a whole has been very much I'm smarter than everybody else, and we're gonna tell everybody how it's gonna be done on the way we fell into a law when it came to producing infrastructure and so embracing this idea that we can deploy a new solution or a new environment in minutes as opposed to hours or weeks or months in some cases is really important. And so, you know, it's >>being closed minded native, being open minded. >>Exactly. And it took for me. It was that was a transformative kind of, uh, where I was looking to solve problems in a cloud way, as opposed to looking to solve problems in this traditional old school way. >>All right, I know we're out of time, but I got one more question for you guys. So good. It could be a quick answer. Um, what's the B s language? When you bs? Meter goes up when people talk about solutions, what's the kind of jargon that you here that's the BS meter going off? What are people talking about that in your opinion, You. Here you go. That's total bs. But what triggers? >>So I have two lines out of movies that are really like it. I say them without actually thinking them. It's like 1.21 gigawatts of your of your mind. Back to the future, right? Somebody's going to the bank and then and then Martin Ball and and Michael Keaton and Mr Mom when it goes to 22. 21 whatever it takes those two right there, if those go off in my mind where somebody's talking to me, I know they're full of baloney. >>A lot of speeds and feeds a lot of speeds and feeds a >>lot of data instead of talking about what you're actually doing in solution ing for You're talking about What does this? This This is okay, 2020 A. >>Just take any time I start seeing the cloud vendors start benchmarking against each other. Your workload is your workload. You need a benchmark yourself. Don't Don't listen to the marketing on that. That's that's >>what triggered you in the BSP. >>I think if somebody explains you're not simple, they cannot explain you in simplicity. Then then it's all >>that's a good one. All right, guys, Thanks for the great insight. Great panel around >>Applause, right? >>Yeah. Yeah,

ladies and gentlemen this is your captain speaking we will soon be taking off on our way to altitude please keep your seatbelts fastened and remain in your seats we will be experiencing turbulence until we are above the clouds ladies and gentlemen we are now cruising at altitude sit back and enjoy the ride [Music] altitude is a community of thought leaders and pioneers cloud architects and enlightened network engineers who have individually and are now collectively leading their own IT teams and the industry on a path to lift cloud networking above the clouds empowering Enterprise IT to architect design and control their own cloud network regardless of the turbulent clouds beneath them it's time to gain altitude ladies and gentlemen Steve Mulaney president and CEO of aviatrix the leader of multi cloud networking [Music] [Applause] all right good morning everybody here in Santa Clara as well as to the what millions of people watching the livestream worldwide welcome to altitude 2020 all right so we've got a fantastic event today really excited about the speakers that we have today and the experts that we have and really excited to get started so one of the things I wanted to just share was this is not a one-time event it's not a one-time thing that we're gonna do sorry for the aviation analogy but you know sherry way aviatrix means female pilot so everything we do as an aviation theme this is a take-off for a movement this isn't an event this is a take-off of a movement a multi-cloud networking movement and community that we're inviting all of you to become part of and-and-and why we're doing that is we want to enable enterprises to rise above the clouds so to speak and build their network architecture regardless of which public cloud they're using whether it's one or more of these public clouds so the good news for today there's lots of good news but this is one good news is we don't have any powerpoint presentations no marketing speak we know that marketing people have their own language we're not using any of that in those sales pitches right so instead what are we doing we're going to have expert panels we've got Simone Rashard Gartner here we've got 10 different network architects cloud architects real practitioners they're going to share their best practices and there are real-world experiences on their journey to the multi cloud so before we start and everybody know what today is in the u.s. it's Super Tuesday I'm not gonna get political but Super Tuesday there was a bigger Super Tuesday that happened 18 months ago and maybe eight six employees know what I'm talking about 18 months ago on a Tuesday every enterprise said I'm gonna go to the cloud and so what that was was the Cambrian explosion for cloud for the price so Frank kibrit you know what a Cambrian explosion is he had to look it up on Google 500 million years ago what happened there was an explosion of life where it went from very simple single-cell organisms to very complex multi-celled organisms guess what happened 18 months ago on a Tuesday I don't really know why but every enterprise like I said all woke up that day and said now I'm really gonna go to cloud and that Cambrian explosion of cloud went meant that I'm moving from very simple single cloud single use case simple environment to a very complex multi cloud complex use case environment and what we're here today is we're gonna go and dress that and how do you handle those those those complexities and when you look at what's happening with customers right now this is a business transformation right people like to talk about transitions this is a transformation and it's actually not just the technology transformation it's a business transformation it started from the CEO and the boards of enterprise customers where they said I have an existential threat to the survival of my company if you look at every industry who they're worried about is not the other 30 year old enterprise what they're worried about is the three year old enterprise that's leveraging cloud that's leveraging AI and that's where they fear that they're going to actually get wiped out right and so because of this existential threat this is CEO lead this is board led this is not technology led it is mandated in the organization's we are going to digitally transform our enterprise because of this existential threat and the movement to cloud is going to enable us to go do that and so IT is now put back in charge if you think back just a few years ago in cloud it was led by DevOps it was led by the applications and it was like I said before their Cambrian explosion is very simple now with this Cambrian explosion and enterprises getting very serious and mission critical they care about visibility they care about control they care about compliance conformance everything governance IT is in charge and and and that's why we're here today to discuss that so what we're going to do today is much of things but we're gonna validate this journey with customers do they see the same thing we're gonna validate the requirements for multi-cloud because honestly I've never met an enterprise that is not going to be multi-cloud many are one cloud today but they all say I need to architect my network for multiple clouds because that's just what the network is there to support the applications and the applications will run and whatever cloud it runs best in and you have to be prepared for that the second thing is is is architecture again with the IT in charge you architecture matters whether it's your career whether it's how you build your house it doesn't matter horrible architecture your life is horrible forever good architecture your life is pretty good so we're gonna talk about architecture and how the most fundamental and critical part of that architecture and that basic infrastructure is the network if you don't get that right nothing works right way more important and compute way more important than storm dense storage network is the foundational element of your infrastructure then we're going to talk about day 2 operations what does that mean well day 1 is one day of your life that's who you wire things up they do and beyond I tell everyone in networking and IT it's every day of your life and if you don't get that right your life is bad forever and so things like operations visibility security things like that how do I get my operations team to be able to handle this in an automated way because it's not just about configuring it in the cloud it's actually about how do I operationalize it and that's a huge benefit that we bring as aviatrix and then the last thing we're going to talk and it's the last panel we have I always say you can't forget about the humans right so all this technology all these things that we're doing it's always enabled by the humans at the end of the day if the humans fight it it won't get deployed and we have a massive skills gap in cloud and we also have a massive skill shortage you have everyone in the world trying to hire cloud network architects right there's just not enough of them going around so at aviatrix as leaders knew we're gonna help address that issue and try to create more people we created a program and we call the ACE program again an aviation theme it stands for aviatrix certified engineer very similar to what Cisco did with CC IES where Cisco taught you about IP networking a little bit of Cisco we're doing the same thing we're gonna teach network architects about multi-cloud networking and architecture and yeah you'll get a little bit of aviatrix training in there but this is the missing element for people's careers and also within their organization so we're gonna we're gonna go talk about that so great great event great show when try to keep it moving I'd next want to introduce my my host he's the best in the business you guys have probably seen him multiple million times he's the co CEO and co-founder of Tube John Fourier okay awesome great great speech they're awesome I totally agree with everything you said about the explosion happening and I'm excited here at the heart of Silicon Valley to have this event it's a special digital event with the cube and aviatrix where we live streaming to millions of people as you said maybe not a million maybe not really take this program to the world this is a little special for me because multi-cloud is the hottest wave and cloud and cloud native networking is fast becoming the key engine of the innovation so we got an hour and a half of action-packed programming we have a customer panel two customer panels before that Gartner is going to come on talk about the industry we have a global system integrators we talk about how they're advising and building these networks and cloud native networking and then finally the Aces the aviatrix certified engineer is gonna talk more about their certifications and the expertise needed so let's jump right in and let's ask someone rashard to come on stage from Gartner we'll check it all up [Applause] [Music] okay so kicking things off certain started gartner the industry experts on cloud really kind of more to your background talk about your background before you got the gardener yeah before because gardener was a chief network architect of a fortune five companies with thousands of sites over the world and I've been doing everything and IT from a C programmer in the 90 to a security architect to a network engineer to finally becoming a network analyst so you rode the wave now you're covering at the marketplace with hybrid cloud and now moving quickly to multi cloud is really I was talking about cloud natives been discussed but the networking piece is super important how do you see that evolving well the way we see Enterprise adapt in cloud first thing you do about networking the initial phases they either go in a very ad hoc way is usually led by non non IT like a shadow whitey or application people or some kind of DevOps team and it's it just goes as it's completely unplanned decreed VP sees left and right with a different account and they create mesh to manage them and their direct connect or Express route to any of them so that's what that's a first approach and on the other side again it within our first approach you see what I call the lift and shift way we see like Enterprise IT trying to basically replicate what they have in a data center in the cloud so they spend a lot of time planning doing Direct Connect putting Cisco routers and f5 and Citrix and any checkpoint Palo Alto divides the data that are sent removing that to that cloud and I ask you the aha moments gonna come up a lot of our panels is where people realize that it's a multi cloud world I mean they either inherit clouds certainly they're using public cloud and on-premises is now more relevant than ever when's that aha moment that you're seeing where people go well I got to get my act together and get on this well the first but even before multi-cloud so these two approach the first one like the adduct way doesn't scale at some point idea has to save them because they don't think about the two they don't think about operations they have a bunch of VPC and multiple clouds the other way that if you do the left and shift wake they cannot take any advantages of the cloud they lose elasticity auto-scaling pay by the drink these feature of agility features so they both realize okay neither of these ways are good so I have to optimize that so I have to have a mix of what I call the cloud native services within each cloud so they start adapting like other AWS constructor is your construct or Google construct then that's I would I call the up optimal phase but even that they they realize after that they are very different all these approaches different the cloud are different identities is completely difficult to manage across clouds I mean for example AWS has accounts there's subscription and in adarand GCP their projects it's a real mess so they realize well I can't really like concentrate used the cloud the cloud product and every cloud that doesn't work so I have I'm doing multi cloud I like to abstract all of that I still wanna manage the cloud from an API to interview I don't necessarily want to bring my incumbent data center products but I have to do that in a more API driven cloud they're not they're not scaling piece and you were mentioning that's because there's too many different clouds yes that's the piece there so what are they doing whether they really building different development teams as its software what's the solution well this the solution is to start architecting the cloud that's the third phase I call that the multi cloud architect phase where they have to think about abstraction that works across cloud fact even across one cloud it might not scale as well if you start having like 10,000 security group in AWS that doesn't scale you have to manage that if you have multiple VPC it doesn't scale you need a third party identity provider so it barely scales within one cloud if you go multiple cloud it gets worse and worse see way in here what's your thoughts I thought we said this wasn't gonna be a sales pitch for aviatrix you just said exactly what we do so anyway I'm just a joke what do you see in terms of where people are in that multi cloud a lot of people you know everyone I talked to started in one cloud right but then they look and they say okay but I'm now gonna move to adjourn I'm gonna move do you see a similar thing well yes they are moving but they're not there's not a lot of application that use a tree cloud at once they move one app in deserve one app in individuals one get happened Google that's what we see so far okay yeah I mean one of the mistakes that people think is they think multi-cloud no one is ever gonna go multi-cloud for arbitrage they're not gonna go and say well today I might go into Azure because I got a better rate of my instance that's never do you agree with that's never going to happen what I've seen with enterprise is I'm gonna put the workload in the app the app decides where it runs best that may be a sure maybe Google and for different reasons and they're gonna stick there and they're not gonna move let me ask you infrastructure has to be able to support from a networking team be able to do that do you agree with that yes I agree and one thing is also very important is connecting to that cloud is kind of the easiest thing so though while I run Network part of the cloud connectivity to the cloud is kind of simple I agree IPSec VP and I reckon Express that's a simple part what's difficult and even a provisioning part is easy you can use terraform and create v pieces and v nets across which we cloud provider right what's difficult is the day-to-day operations so it's what to find a to operations what is that what does that actually mean this is the day-to-day operations after it you know the natural let's add an app let's add a server let's troubleshoot a problem so what so your life something changes how would he do so what's the big concerns I want to just get back to this cloud native networking because everyone kind of knows with cloud native apps are that's been a hot trend what is cloud native networking how do you how do you guys define that because that seems to be the oddest part of the multi-cloud wave that's coming as cloud native networking well there's no you know official garner definition but I can create one on another spot it's do it I just want to leverage the cloud construct and a cloud epi I don't want to have to install like like for example the first version was let's put a virtual router that doesn't even understand and then the cloud environment right if I have if I have to install a virtual machine it has to be cloud aware it has to understand the security group if it's a router it has to be programmable to the cloud API and and understand the cloud environment you know one things I hear a lot from either see Saussure CIOs or CXOs in general is this idea of I'm definitely on going API so it's been an API economy so API is key on that point but then they say okay I need to essentially have the right relationship with my suppliers aka clouds you call it above the clouds so the question is what do i do from an architecture standpoint do I just hire more developers and have different teams because you mentioned that's a scale point how do you solve this this problem of okay I got AWS I got GCP or Azure or whatever do I just have different teams or just expose api's where is that optimization where's the focus well I take what you need from an android point of view is a way a control plane across the three clouds and be able to use the api of the cloud to build networks but also to troubleshoot them and do they to operation so you need a view across a three cloud that takes care of routing connectivity that's you know that's the aviatrix plug of you right there so so how do you see so again your Gartner you you you you see the industry you've been a network architect how do you see this this plane out what are the what are the legacy incumbent client-server on-prem networking people gonna do well these versus people like aviatrix well how do you see that plane out well obviously all the incumbent like Arista cisco juniper NSX right they want to basically do the lift and ship or they want to bring and you know VM I want to bring in a section that cloud they call that NSX everywhere and cisco monks bring you star in the cloud recall that each guy anywhere right so everyone what and and then there's cloud vision for my red star and contrail is in the cloud so they just want to bring the management plain in the cloud but it's still based most of them it's still based on putting a VM them in controlling them right you you extend your management console to the cloud that's not truly cloud native right cloud native you almost have to build it from scratch we like to call that cloud naive clown that close one letter yeah so that was a big con surgeon i reinvent take the tea out of cloud native its cloud naive i went super viral you guys got t-shirts now i know you love it but yeah but that really ultimately is kind of a double-edged sword you got to be you can be naive on the on the architecture side and rolling out but also suppliers are can be naive so how would you define who's naive and who's not well in fact they're evolving as well so for example in cisco you it's a little bit more native than other ones because they're really ACI in the cloud you call you you really like configure api so the cloud and nsx is going that way and so is Arista but they're incumbent they have their own tools it's difficult for them they're moving slowly so it's much easier to start from scratch Avenue like and you know and network happiness started a few years ago there's only really two aviatrix was the first one they've been there for at least three or four years and there's other ones like Al Kyra for example that just started now that doing more connectivity but they want to create an overlay network across the cloud and start doing policies and trying abstracting all the clouds within one platform so I gotta ask you I interviewed an executive at VMware Sanjay Pune and he said to me at RSA last week oh the only b2 networking vendors left Cisco and VMware what's your respect what's your response to that obviously I mean when you have these waves as new brands that emerge like AV X and others though I think there'll be a lot of startups coming out of the woodwork how do you respond to that comment well there's still a data center there's still like a lot of action on campus and there's the one but from the cloud provisioning and clown networking in general I mean they're behind I think you know in fact you don't even need them to start to it you can if you're small enough you can just keep if you're in AWS you can user it with us construct they have to insert themselves I mean they're running behind they're all certainly incumbents I love the term Andy Jesse's that Amazon Web Services uses old guard new guard to talk about the industry what does the new guard have to do the new and new brands that emerge in is it be more DevOps oriented neck Nets a cops is that net ops is the programmability these are some of the key discussions we've been having what's your view on how you see this program their most important part is they have to make the network's simple for the dev teams and from you cannot have that you cannot make a phone call and get it via line in two weeks anymore so if you move to that cloud you have to make the cloud construct as simple enough so that for example a dev team could say okay I'm going to create this VP see but this VP see automatically being your associate to your account you cannot go out on the internet you have to go to the transit VP C so there's a lot of action in terms of the I am part and you have to put the control around them too so to make it as simple as possible you guys both I mean you're the COC aviatrix but also you guys a lot of experience going back to networking going back to I call the OSI mace which for us old folks know that means but you guys know this means I want to ask you the question as you look at the future of networking here a couple of objectives oh the cloud guys they got networking we're all set with them how do you respond to the fact that networking is changing and the cloud guys have their own networking what some of the pain points that's going on premises and these enterprises so are they good with the clouds what needs what are the key things that's going on in networking that makes it more than just the cloud networking what's your take on well I as I said earlier that once you you could easily provision in the cloud you can easily connect to that cloud is when you start troubleshooting application in the cloud and try to scale so this that's where the problem occurs see what you're taking on it and you'll hear from the from the customers that that we have on stage and I think what happens is all the cloud the clouds by definition designed to the 80/20 rule which means they'll design 80% of the basic functionality and they'll lead the 20% extra functionality that of course every enterprise needs they'll leave that to ISVs like aviatrix because why because they have to make money they have a service and they can't have huge instances for functionality that not everybody needs so they have to design to the common and that's they all do it right they have to and then the extra the problem is that can be an explosion that I talked about with enterprises that's holy that's what they need that they're the ones who need that extra 20% so that's that's what I see is is there's always gonna be that extra functionality the in in an automated and simple way that you talked about but yet powerful with up with the visibility and control that they expect of on prep that that's that kind of combination that yin and the yang that people like us are providing some I want to ask you were gonna ask some of the cloud architect customer panels it's the same question this pioneers doing some work here and there's also the laggards who come in behind the early adopters what's gonna be the tipping point what are some of those conversations that the cloud architects are having out there or what's the signs that they need to be on this multi cloud or cloud native networking trend what are some the signals that are going on in their environment what are some of the threshold or things that are going on that there can pay attention to well well once they have application and multiple cloud and they have they get wake up at 2:00 in the morning to troubleshoot them they don't know it's important so I think that's the that's where the robber will hit the road but as I said it's easier to prove it it's okay it's 80s it's easy user transit gateway put a few V pcs and you're done and use create some presents like equinox and do Direct Connect and Express route with Azure that looks simple is the operations that's when they'll realize okay now I need to understand our car networking works I also need a tool that give me visibility and control not button tell me that I need to understand the basic underneath it as well what are some of the day in the life scenarios that you envision happening with multi cloud because you think about what's happening it kind of has that same vibe of interoperability choice multi-vendor because you have multi clouds essentially multi vendor these are kind of old paradigms that we've lived through the client-server and internet working wave what are some of those scenarios of success and that might be possible it would be possible with multi cloud and cloud native networking well I think once you have good enough visibility to satisfy your customers you know not only like to keep the service running an application running but to be able to provision fast enough I think that's what you want to achieve small final question advice for folks watching on the live stream if they're sitting there as a cloud architect or a CXO what's your advice to them right now in this market because honestly public check hybrid cloud they're working on that that gets on-premise is done now multi-class right behind it what's your advice the first thing they should do is really try to understand cloud networking for each of their cloud providers and then understand the limitation and is what their cloud service provider offers enough or you need to look to a third party but you don't look at a third party to start to it especially an incumbent one so it's tempting to say on and I have a bunch of f5 experts nothing against f5 I'm going to bring my five in the cloud when you can use a needle be that automatically understand ease ease and auto-scaling and so on and you understand that's much simpler but sometimes you need you have five because you have requirements you have like AI rules and that kind of stuff that you use for years you cannot do it's okay I have requirement and that net I'm going to use legacy stuff and then you have to start thinking okay what about visibility control about the tree cloud but before you do that you have to understand the limitation of the existing cloud providers so first try to be as native as possible until things don't work after that you can start taking multi-cloud great insight somewhat thank you for coming someone in charge with Gardner thanks for sharing thank you appreciate it [Applause] informatica is known as the leading enterprise cloud data management company we are known for being the top in our industry in at least five different products over the last few years especially we've been transforming into a cloud model which allows us to work better with the trends of our customers in order to see agile and effective in a business you need to make sure that your products and your offerings are just as relevant in all these different clouds than what you're used to and what you're comfortable with one of the most difficult challenges we've always had is that because we're a data company we're talking about data that a customer owns some of that data may be in the cloud some of that data may be on Prem some of them data may be actually in their data center in another region or even another country and having that data connect back to our systems that are located in the cloud has always been a challenge when we first started our engagement with aviatrix we only had one plan that was Amazon it wasn't till later that a jerk came up and all of a sudden we found hey the solution we already had in place for aviatrix already working in Amazon and now works in Missouri as well before we knew it GCP came up but it really wasn't a big deal for us because we already had the same solution in Amazon and integer now just working in GCP by having a multi cloud approach we have access to all three of them but more commonly it's not just one it's actually integrations between multiple we have some data and ensure that we want to integrate with Amazon we have some data in GCP that we want to bring over to a data Lake assure one of the nice things about aviatrix is that it gives a very simple interface that my staff can understand and use and manage literally hundreds of VPNs around the world and while talking to and working with our customers who are literally around the world now that we've been using aviatrix for a couple years we're actually finding that even problems that we didn't realize we had were actually solved even before we came across the problem and it just worked cloud companies as a whole are based on reputation we need to be able to protect our reputation and part of that reputation is being able to protect our customers and being able to protect more importantly our customers data aviatrix has been helpful for us in that we only have one system that can manage this whole huge system in a simple easy direct model aviatrix is directly responsible for helping us secure and manage our customers not only across the world but across multiple clouds users don't have to be VPN or networking experts in order to be able to use the system all the members on my team can manage it all the members regardless of their experience can do different levels of it one of the unexpected two advantages of aviatrix is that I don't have to sell it to my management the fact that we're not in the news at three o'clock in the morning or that we don't have to get calls in the middle of the night no news is good news especially in networking things that used to take weeks to build are done in hours I think the most important thing about a matrix is it provides me consistency aviatrix gives me a consistent model that I can use across multiple regions multiple clouds multiple customers okay welcome back to altitude 2020 for the folks on the livestream I'm John for Steve Mulaney with CEO of aviatrix for our first of two customer panels on cloud with cloud network architects we got Bobby Willoughby they gone Luis Castillo of National Instruments and David should Nick with fact set guys welcome to the stage for this digital event come on up [Music] hey good to see you thank you okay okay customer panelist is my favorite part we get to hear the real scoop we got the gardener giving us the industry overview certainly multi clouds very relevant and cloud native networking is the hot trend with the live stream out there and the digital event so guys let's get into it the journey is you guys are pioneering this journey of multi cloud and cloud native networking and it's soon gonna be a lot more coming so I want to get into the journey what's it been like is it real you got a lot of scar tissue and what are some of the learnings yeah absolutely so multi cloud is whether or not we we accepted as a network engineers is a is a reality like Steve said about two years ago companies really decided to to just to just bite the bullet and and and move there whether or not whether or not we we accept that fact we need to now create a consistent architecture across across multiple clouds and that that is challenging without orchestration layers as you start managing different different tool sets in different languages across different clouds so that's it's really important that to start thinking about that guys on the other panelists here there's different phases of this journey some come at it from a networking perspective some come in from a problem troubleshooting what's what's your experiences yeah so from a networking perspective it's been incredibly exciting it's kind of a once-in-a-generation 'el opportunity to look at how you're building out your network you can start to embrace things like infrastructure as code that maybe your peers on the systems teams have been doing for years but it just never really worked on pram so it's really it's really exciting to look at all the opportunities that we have and then all the interesting challenges that come up that you that you get to tackle an effect said you guys are mostly AWS right yep right now though we are looking at multiple clouds we have production workloads running in multiple clouds today but a lot of the initial work has been with Amazon and you've seen it from a networking perspective that's where you guys are coming at it from yep yeah we evolved more from a customer requirement perspective started out primarily as AWS but as the customer needed more resources to measure like HPC you know as your ad things like that even recently Google at Google Analytics our journey has evolved into mortal multi-cloud environment Steve weigh in on the architecture because this has been the big conversation I want you to lead this second yeah so I mean I think you guys agree the journey you know it seems like the journey started a couple years ago got real serious the need for multi-cloud whether you're there today of course it's gonna be there in the future so that's really important I think the next thing is just architecture I'd love to hear what you you know had some comments about architecture matters it all starts I mean every Enterprise that I talk to maybe talk about architecture and the importance of architecture maybe Bobby it's a particular perspective we sorted a journey five years ago Wow okay and we're just now starting our fourth evolution of our network architect and we'll call it networking security net sec yep adverse adjusters network and that fourth generation or architectures be based primarily upon Palo Alto Networks an aviatrix a matrix doing the orchestration piece of it but that journey came because of the need for simplicity okay I need for multi-cloud orchestration without us having to go and do reprogramming efforts across every cloud as it comes along right I guess the other question I also had around architectures also Louis maybe just talk about I know we've talked a little bit about you know scripting right and some of your thoughts on that yeah absolutely so so for us we started we started creating the network constructs with cloud formation and we've we've stuck with that for the most part what's interesting about that is today on premise we have a lot of a lot of automation around around how we provision networks but cloud formation has become a little bit like the new manual for us so we're now having issues with having to to automate that component and making it consistent with our on-premise architecture making it consistent with Azure architecture and Google cloud so it's really interesting to see to see companies now bring that layer of abstraction that SD when brought to the to the wine side now it's going up into into the into the cloud networking architecture so on the fourth generation of you mentioned you're in the fourth gen architecture what do you guys what have you learned is there any lessons scar tissue what to avoid what worked what was some of the there was a path that's probably the biggest list and there is when you think you finally figured it out you have it right Amazon will change something as you change something you know transit gateways a game changer so in listening to the business requirements is probably the biggest thing we need to do up front but I think from a simplicity perspective like I said we don't want to do things four times we want to do things one time we won't be able to write to an API which aviatrix has and have them do the orchestration for us so that we don't have to do it four times how important is architecture in the progression is it you guys get thrown in the deep end to solve these problems or you guys zooming out and looking at it it's a I mean how are you guys looking at the architecture I mean you can't get off the ground if you don't have the network there so all of those there we've gone through similar evolutions we're on our fourth or fifth evolution I think about what we started off with Amazon without a direct connect gate without a transit Gateway without a lot of the things that are available today kind of the 80/20 that Steve was talking about just because it wasn't there doesn't mean we didn't need it so we needed to figure out a way to do it we couldn't say oh you need to come back to the network team in a year and maybe Amazon will have a solution for it right you need to do it now and in evolve later and maybe optimize or change the way you're doing things in the future but don't sit around and wait you can I'd love to have you guys each individually answer this question for the live stream because it comes up a lot a lot of cloud architects out in the community what should they be thinking about the folks that are coming into this proactively and/or realizing the business benefits are there what advice would you guys give them an architecture what should be they be thinking about and what are some guiding principles you could share so I would start with looking at an architecture model that that can that can spread and and give consistency they're different to different cloud vendors that you will absolutely have to support cloud vendors tend to want to pull you into using their native toolset and that's good if only it was realistic to talk about only one cloud but because it doesn't it's it's it's super important to talk about and have a conversation with the business and with your technology teams about a consistent model so that's the David yeah talking as earlier about day two operations so how do I design how do I do my day one work so that I'm not you know spending eighty percent of my time troubleshooting or managing my network because I'm doing that then I'm missing out on ways that I can make improvements or embrace new technologies so it's really important early on to figure out how do I make this as low maintenance as possible so that I can focus on the things that the team really should be focusing on Bobby your advice the architect I don't know what else I can do that simplicity of operations is key alright so the holistic view of day to operation you mentioned let's can jump in day one is your your your getting stuff set up day two is your life after all right this is kinda what you're getting at David so what does that look like what are you envisioning as you look at that 20 mile stair out post multi-cloud world what are some of the things that you want in a day to operations yeah infrastructure is code is really important to us so how do we how do we design it so that we can fit start making network changes and fitting them into like a release pipeline and start looking at it like that rather than somebody logging into a router CLI and troubleshooting things on in an ad hoc nature so moving more towards the DevOps model is anything on that day - yeah I would love to add something so in terms of day 2 operations you can you can either sort of ignore the day 2 operations for a little while where you get well you get your feet wet or you can start approaching it from the beginning the fact is that the the cloud native tools don't have a lot of maturity in that space and when you run into an issue you're gonna end up having a bad day going through millions and millions of logs just to try to understand what's going on so that's something that that the industry just now is beginning to realize it's it's such a such a big gap I think that's key because for us we're moving to more of an event-driven or operations in the past monitoring got the job done it's impossible to modern monitor something there's nothing there when the event happens all right so the event-driven application and then detect is important yeah I think garden was all about the cloud native wave coming into networking that's gonna be a serious thing I want to get you guys perspectives I know you have different views of how you come into the journey and how you're executing and I always say the beauties in the eye of the beholder and that kind of applies how the networks laid out so Bobby you guys do a lot of high-performance encryption both on AWS and Azure that's kind of a unique thing for you how are you seeing that impact with multi cloud yeah and that's a new requirement for us to where we we have an intern crypt and they they ever get the question should I encryption and I'll encrypt the answer is always yes you should encrypt when you can encrypt for our perspective we we need to migrate a bunch of data from our data centers we have some huge data centers and then getting that data to the cloud is the timely experiencing some cases so we have been mandated that we have to encrypt everything leaving the data center so we're looking at using the aviatrix insane mode appliances to be able to encrypt you know 10 20 gigabits of data as it moves to the cloud itself David you're using terraform you got fire Ned you've got a lot of complexity in your network what do you guys look at the future for yours environment yeah so something exciting that or yeah now is fire net so for our security team they obviously have a lot of a lot of knowledge base around Palo Alto and with our commitments to our clients you know it's it's it's not very easy to shift your security model to a specific cloud vendor right so there's a lot of stuck to compliance of things like that where being able to take some of what you've you know you've worked on for years on Bram and put it in the cloud and have the same type of assurance that things are gonna work and be secure in the same way that they are on prem helps make that journey into the cloud a lot easier and Louis you guys got scripting and get a lot of things going on what's your what's your unique angle on this yeah no absolutely so full disclosure I'm not a not not an aviatrix customer yet it's ok we want to hear the truth that's good Ellis what are you thinking about what's on your mind no really when you when you talk about implementing the tool like this it's really just really important to talk about automation and focus on on value so when you talk about things like and things like so yeah encrypting tunnels and encrypting the paths and those things are it should it should should be second nature really when you when you look at building those backends and managing them with your team it becomes really painful so tools like aviatrix that that add a lot of automation it's out of out of sight out of mind you can focus on the value and you don't have to focus on so I gotta ask you guys I see AV traces here they're they're a supplier to the sector but you guys are customers everyone's pitching you stuff people are not gonna buy my stuff how do you guys have that conversation with the suppliers like the cloud vendors and other folks what's the what's it like where API all the way you got to support this what are some of the what are some of your requirements how do you talk to and evaluate people that walk in and want to knock on your door and pitch you something what's the conversation like um it's definitely it's definitely API driven we we definitely look at the at that the API structure of the vendors provide before we select anything that that is always first in mind and also what a problem are we really trying to solve usually people try to sell or try to give us something that isn't really valuable like implementing a solution on the on the on the cloud isn't really it doesn't really add a lot of value that's where we go David what's your conversation like with suppliers you have a certain new way to do things as as becomes more agile and essentially the networking become more dynamic what are some of the conversation is with the either incumbents or new new vendors that you're having what it what do you require yeah so ease of use is definitely definitely high up there we've had some vendors come in and say you know hey you know when you go to set this up we're gonna want to send somebody on site and they're gonna sit with you for your day to configure it and that's kind of a red flag what wait a minute you know do we really if one of my really talented engineers can't figure it out on his own what's going on there and why is that so you know having having some ease-of-use and the team being comfortable with it and understanding it is really important Bobby how about you I mean the old days was do a bake-off and you know the winner takes all I mean is it like that anymore what's the Volvic bake-off last year first you win so but that's different now because now when you you get the product you can install the product in AWS energy or have it up and running a matter of minutes and so the key is is they can you be operational you know within hours or days instead of weeks but but do we also have the flexibility to customize it to meet your needs could you want to be you won't be put into a box with the other customers we have needs that surpass their cut their needs yeah I almost see the challenge that you guys are living where you've got the cloud immediate value to make an roll-up any solutions but then you have might have other needs so you've got to be careful not to buy into stuff that's not shipping so you're trying to be proactive at the same time deal with what you got I mean how do you guys see that evolving because multi-cloud to me is definitely relevant but it's not yet clear how to implement across how do you guys look at this baked versus you know future solutions coming how do you balance that so again so right now we we're we're taking the the ad hoc approach and and experimenting with the different concepts of cloud and really leveraging the the native constructs of each cloud but but there's a there's a breaking point for sure you don't you don't get to scale this I like like Simone said and you have to focus on being able to deliver a developer they're their sandbox or their play area for the for the things that they're trying to build quickly and the only way to do that is with the with with some sort of consistent orchestration layer that allows you to so you've got a lot more stuff to be coming pretty quickly IDEs area I do expect things to start to start maturing quite quite quickly this year and you guys see similar trend new stuff coming fast yeah part of the biggest challenge we've got now is being able to segment within the network being able to provide segmentation between production on production workloads even businesses because we support many businesses worldwide and and isolation between those is a key criteria there so the ability to identify and quickly isolate those workloads is key so the CIOs that are watching or that are saying hey take that he'll do multi cloud and then you know the bottoms up organization think pause you're kind of like off a little bit it's not how it works I mean what is the reality in terms of implementing you know and as fast as possible because the business benefits are clear but it's not always clear in the technology how to move that fast yeah what are some of the barriers one of the blockers what are the enabler I think the reality is is that you may not think you're multi-cloud but your business is right so I think the biggest barriers there is understanding what the requirements are and how best to meet those requirements in a secure manner because you need to make sure that things are working from a latency perspective that things work the way they did and get out of the mind shift that you know it was a cheery application in the data center it doesn't have to be a Tier three application in the cloud so lift and shift is is not the way to go scale is a big part of what I see is the competitive advantage to allow these clouds and used to be proprietary network stacks in the old days and then open systems came that was a good thing but as clouds become bigger there's kind of an inherent lock in there with the scale how do you guys keep the choice open how're you guys thinking about interoperability what are some of the conversations and you guys are having around those key concepts well when we look at when we look at the moment from a networking perspective it it's really key for you to just enable enable all the all the clouds to be to be able to communicate between them developers will will find a way to use the cloud that best suits their their business team and and like like you said it's whether whether you're in denial or not of the multi cloud fact that your company is in already that's it becomes really important for you to move quickly yeah and a lot of it also hinges on how well is the provider embracing what that specific cloud is doing so are they are they swimming with Amazon or sure and just helping facilitate things they're doing the you know the heavy lifting API work for you or they swimming upstream and they're trying to hack it all together in a messy way and so that helps you you know stay out of the lock-in because they're you know if they're doing if they're using Amazon native tools to help you get where you need to be it's not like Amazon's gonna release something in the future that completely you know makes you have designed yourself into a corner so the closer they're more cloud native they are the more the easier it is to to deploy but you also need to be aligned in such a way that you can take advantage of those cloud native technologies will it make sense tgw is a game-changer in terms of cost and performance right so to completely ignore that would be wrong but you know if you needed to have encryption you know teach Adobe's not encrypted so you need to have some type of a gateway to do the VPN encryption you know so the aviatrix tool give you the beauty of both worlds you can use tgw with a gateway Wow real quick in the last minute we have I want to just get a quick feedback from you guys I hear a lot of people say to me hey the I picked the best cloud for the workload you got and then figure out multi cloud behind the scenes so that seems to be do you guys agree with that I mean is it do I go Mull one cloud across the whole company or this workload works great on AWS that work was great on this from a cloud standpoint do you agree with that premise and then wit is multi clouds did you mall together yeah from from an application perspective it it can be per workload but it can also be an economical decision certain enterprise contracts will will pull you in one direction that add value but the the network problem is still the same doesn't go away yeah yeah I mean you don't want to be trying to fit a square into a round hall right so if it works better on that cloud provider then it's our job to make sure that that service is there and people can use it agree you just need to stay ahead of the game make sure that the network infrastructure is there secure is available and is multi cloud capable yeah I'm at the end of the day you guys just validating that it's the networking game now how cloud storage compute check networking is where the action is awesome thanks for your insights guys appreciate you coming on the panel appreciate thanks thank you [Applause] [Music] [Applause] okay welcome back on the live feed I'm John fritz T Blaney my co-host with aviatrix I'm with the cube for the special digital event our next customer panel got great another set of cloud network architects Justin Smith was aura Justin broadly with Ellie Mae and Amit Oh tree job with Cooper welcome to stage [Applause] all right thank you thank you oK you've got all the cliff notes from the last session welcome rinse and repeat yeah yeah we're going to go under the hood a little bit I think they nailed the what we've been reporting and we've been having this conversation around networking is where the action is because that's the end of the day you got a move a pack from A to B and you get workloads exchanging data so it's really killer so let's get started Amit what are you seeing as the journey of multi cloud as you go under the hood and say okay I got to implement this I have to engineer the network make it enabling make it programmable make it interoperable across clouds I mean that's like I mean almost sounds impossible to me what's your take yeah I mean it's it seems impossible but if you are running an organization which is running infrastructure as a cordon all right it is easily doable like you can use tools out there that's available today you can use third-party products that can do a better job but but put your architecture first don't wait architecture may not be perfect put the best architecture that's available today and be agile to ET rate and make improvements over the time we got to Justin's over here so I have to be careful when I point a question adjusting they both have to answer okay journeys what's the journey been like I mean is there phases we heard that from Gardner people come into multi cloud and cloud native networking from different perspectives what's your take on the journey Justin yeah I mean from Mars like - we started out very much focused on one cloud and as we started doing errands we started doing new products the market the need for multi cloud comes very apparent very quickly for us and so you know having an architecture that we can plug in play into and be able to add and change things as it changes is super important for what we're doing in the space just in your journey yes for us we were very ad hoc oriented and the idea is that we were reinventing all the time trying to move into these new things and coming up with great new ideas and so rather than it being some iterative approach with our deployments that became a number of different deployments and so we shifted that tour and the network has been a real enabler of this is that it there's one network and it touches whatever cloud we want it to touch and it touches the data centers that we need it to touch and it touches the customers that we need it to touch our job is to make sure that the services that are of and one of those locations are available in all of the locations so the idea is not that we need to come up with this new solution every time it's that we're just iterating on what we've already decided to do before we get the architecture section I want to ask you guys a question I'm a big fan of you know let the app developers have infrastructure as code so check but having the right cloud run that workload I'm a big fan of that if it works great but we just heard from the other panel you can't change the network so I want to get your thoughts what is cloud native networking and is that the engine really that's the enabler for this multi cloud trend but you guys taken we'll start with Amit what do you think about that yeah so you are gonna have workloads running in different clouds and the workloads would have affinity to one cloud over other but how you expose that it's matter of how you are going to build your networks how we are going to run security how we are going to do egress ingress out of it so it's the big problem how do you split says what's the solution what's the end the key pain points and problem statement I mean the key pain point for most companies is how do you take your traditional on-premise network and then blow that out to the cloud in a way that makes sense you know IP conflicts you have IP space you pub public eye peas and premise as well as in the cloud and how do you kind of make them a sense of all of that and I think that's where tools like aviatrix make a lot of sense in that space from our site it's it's really simple it's latency and bandwidth and availability these don't change whether we're talking about cloud or data center or even corporate IT networking so our job when when these all of these things are simplified into like s3 for instance and our developers want to use those we have to be able to deliver that and for a particular group or another group that wants to use just just GCP resources these aren't we have to support these requirements and these wants as opposed to saying hey that's not a good idea now our job is to enable them not to disable them do you think you guys think infrastructure as code which I love that I think it's that's the future it is we saw that with DevOps but I just start getting the networking is it getting down to the network portion where it's network as code because storage and compute working really well is seeing all kubernetes on ServiceMaster and network is code reality is it there is it still got work to do it's absolutely there I mean you mentioned net DevOps and it's it's very real I mean in Cooper we build our networks through terraform and on not only just out of fun build an API so that we can consistently build V nets and VPC all across in the same way we get to do it yeah and even security groups and then on top and aviatrix comes in we can peer the networks bridge bridge all the different regions through code same with you guys but yeah about this everything we deploy is done with automation and then we also run things like lambda on top to make changes in real time we don't make manual changes on our network in the data center funny enough it's still manual but the cloud has enabled us to move into this automation mindset and and all my guys that's what they focus on is bringing what now what they're doing in the cloud into the data center which is kind of opposite of what it should be that's full or what it used to be it's full DevOps then yes yeah I mean for us it was similar on premise still somewhat very manual although we're moving more Norton ninja and terraform concepts but everything in the production environment is colored confirmation terraform code and now coming into the datacenter same I just wanted to jump in on a Justin Smith one of the comment that you made because it's something that we always talk about a lot is that the center of gravity of architecture used to be an on-prem and now it's shifted in the cloud and once you have your strategic architecture what you--what do you do you push that everywhere so what you used to see at the beginning of cloud was pushing the architecture on prem into cloud now i want to pick up on what you said to you others agree that the center of architect of gravity is here i'm now pushing what i do in the cloud back into on Prem and wait and then so first that and then also in the journey where are you at from zero to a hundred of actually in the journey to cloud do you 50% there are you 10% yes I mean are you evacuating data centers next year I mean were you guys at yeah so there's there's two types of gravity that you typically are dealing with no migration first is data gravity and your data set and where that data lives and then the second is the network platform that interrupts all that together right in our case the data gravity sold mostly on Prem but our network is now extend out to the app tier that's going to be in cloud right eventually that data gravity will also move to cloud as we start getting more sophisticated but you know in our journey we're about halfway there about halfway through the process we're taking a handle of you know lift and shift and when did that start and we started about three years ago okay okay go by it's a very different story it started from a garage and one hundred percent on the clock it's a business spend management platform as a software-as-a-service one hundred percent on the cloud it was like ten years ago right yes yeah you guys are riding the wave love that architecture Justin I want to ask you Sora you guys mentioned DevOps I mean obviously we saw the huge observability wave which is essentially network management for the cloud in my opinion right yeah it's more dynamic but this is about visibility we heard from the last panel you don't know what's being turned on or turned off from a services standpoint at any given time how is all this playing out when you start getting into the DevOps down well this layer this is the big challenge for all of us as visibility when you talk transport within a cloud you know we very interestingly we have moved from having a backbone that we bought that we owned that would be data center connectivity we now I work for soar as a subscription billing company so we want to support the subscription mindset so rather than going and buying circuits and having to wait three months to install and then coming up with some way to get things connected and resiliency and redundancy I my backbone is in the cloud I use the cloud providers interconnections between regions to transport data across and and so if you do that with their native solutions you you do lose visibility there there are areas in that that you don't get which is why controlling you know controllers and having some type of management plane is a requirement for us to do what we're supposed to do and provide consistency while doing it a great conversation I loved when you said earlier latency bandwidth availability with your sim pop3 things guys SLA I mean you just do ping times are between clouds it's like you don't know what you're getting for round-trip times this becomes a huge kind of risk management black hole whatever you want to call blind spot how are you guys looking at the interconnects between clouds because you know I can see that working from you know ground to cloud I'm per cloud but when you start doing with multi clouds workloads I mean s LA's will be all over the map won't they just inherently but how do you guys view that yeah I think we talked about workload and we know that the workloads are going to be different in different clouds but they are going to be calling each other so it's very important to have that visibility that you can see how data is flowing at what latency and whatever ability is our is there and our authority needs to operate on that so it's so you use the software dashboard look at the times and look at the latency in the old days strong so on open so on you try to figure it out and then your days you have to figure out just what she reinsert that because you're in the middle of it yeah I mean I think the the key thing there is that we have to plan for that failure we have to plan for that latency in our applications that start thinking start tracking in your SLI something you start planning for and you loosely couple these services and a much more micro services approach so you actually can handle that kind of failure or that type of unknown latency and unfortunately the cloud has made us much better at handling exceptions a much better way you guys are all great examples of cloud native from day one and you guys had when did you have the tipping point moment or the Epiphany of saying a multi clouds real I can't ignore it I got to factor it into all my design design principles and and everything you're doing what's it was there a moment was it was it from day one no there were two reasons one was the business so in business there was some affinity to not be in one cloud or to be in one cloud and that drove from the business side so as a cloud architect our responsibility was to support that business and other is the technology some things are really running better in like if you are running dot Network load or you are going to run machine learning or AI so that you have you would have that reference of one cloud over other so it was the bill that we got from AWS I mean that's that's what drives a lot of these conversations is the financial viability of what you're building on top of it which is so we this failure domain idea which is which is fairly interesting is how do I solve or guarantee against a failure domain you have methodologies with you know back-end direct connects or interconnect with GCP all of these ideas are something that you have to take into account but that transport layer should not matter to whoever we're building this for our job is to deliver the frames in the packets what that flows across how you get there we want to make that seamless and so whether it's a public internet API call or it's a back-end connectivity through Direct Connect it doesn't matter it just has to meet a contract that you signed with your application folks yeah that's the availability piece just in your thoughts on anything any common uh so actually a multi clouds become something much more recent in the last six to eight months I'd say we always kind of had a very much an attitude of like moving to Amazon from our private cloud is hard enough why complicate it further but the realities of the business and as we start seeing you know improvements in Google and Asia and different technology spaces the need for multi cloud becomes much more important as well as our acquisition strategies I matured we're seeing that companies that used to be on premise that we typically acquire are now very much already on a cloud and if they're on a cloud I need to plug them into our ecosystem and so that's really change our multi cloud story in a big way I'd love to get your thoughts on the clouds versus the clouds because you know you compare them Amazon's got more features they're rich with features I see the bills are how could people using them but Google's got a great network Google's networks pretty damn good and then you got a sure what's the difference between the clouds who with they've evolved something whether they peak in certain areas better than others what what are the characteristics which makes one cloud better do they have a unique feature that makes as you're better than Google and vice versa what do you guys think about the different clouds yeah to my experience I think there is approaches different in many places Google has a different approach very DevOps friendly and you can run your workload like the your network and spend regions time I mean but our application ready to accept that MS one is evolving I mean I remember 10 years back Amazon's Network was a flat network we will be launching servers and 10.0.0.0 so the VP sees concept came out multi-account came out so they are evolving as you are at a late start but because they have a late start they saw the pattern and they they have some mature set up on the yeah I think they're all trying to say they're equal in their own ways I think they all have very specific design philosophies that allow them to be successful in different ways and you have to kind of keep that in mind as you architectural solution for example amazon has a very much a very regional affinity they don't like to go cross region in their architecture whereas Google is very much it's a global network we're gonna think about as a global solution I think Google also has advantages its third to market and so has seen what Asia did wrong it seemed with AWS did wrong and it's made those improvements and I think that's one of their big advantage at great scale to Justin thoughts on the cloud so yeah Amazon built from the system up and Google built from the network down so their ideas and approaches are from a global versus or regional I agree with you completely that that is the big number one thing but the if you look at it from the outset interestingly the inability or the ability for Amazon to limit layer 2 broadcasting and and what that really means from a VPC perspective changed all the routing protocols you can use all the things that we have built inside of a data center to provide resiliency and and and make things seamless to users all of that disappeared and so because we had to accept that at the VPC level now we have to accept it at the LAN level Google's done a better job of being able to overcome those things and provide those traditional Network facilities to us just great panel can go all day here's awesome so I heard we could we'll get to the cloud native naive questions so kind of think about what's not even what's cloud is that next but I got to ask you had a conversation with a friend he's like Wayne is the new land so if you think about what the land was at a datacenter when is the new link you could talking about the cloud impact so that means st when the old st way is kind of changing into the new land how do you guys look at that because if you think about it what lands were for inside a premises was all about networking high-speed but now when you take the win and make it essentially a land do you agree with that and how do you view this trend and is it good or bad or is it ugly and what's what you guys take on this yeah I think it's a it's a thing that you have to work with your application architect so if you are managing networks and if you are a sorry engineer you need to work with them to expose the unreliability that would bring in so the application has to hand a lot of this the difference in the latencies and and the reliability has to be worked through the application there Lanois same concept is that BS I think we've been talking about for a long time the erosion of the edge and so is this is just a continuation of that journey we've been on for the last several years as we get more and more cloud native and we start about API is the ability to lock my data in place and not be able to access it really goes away and so I think this is just continuation that thing I think it has challenges we start talking about weighing scale versus land scale the tooling doesn't work the same the scale of that tooling is much larger and the need to automation is much much higher in a way and than it was in a land that's where is what you're seeing so much infrastructure as code yeah yes so for me I'll go back again to this its bandwidth and its latency right that bet define those two land versus win but the other thing that's comes up more and more with cloud deployments is where is our security boundary and where can I extend this secure aware appliance or set of rules to to protect what's inside of it so for us we're able to deliver vr af-s or route forwarding tables for different segments wherever we're at in the world and so they're they're trusted to talk to each other but if they're gonna go to someplace that's outside of their their network then they have to cross a security boundary and where we enforce policy very heavily so for me there's it's not just land when it's it's how does environment get to environment more importantly that's a great point and security we haven't talked to yet but that's got to be baked in from the beginning this architecture thoughts on security are you guys are dealing with it yeah start from the base have apt to have security built in have TLS have encryption on the data I transit data at rest but as you bring the application to the cloud and they are going to go multi-cloud talking to over the Internet in some places well have apt web security I mean I mean our principles day Security's day zero every day and so we we always build it into our design build into our architecture into our applications it's encrypt everything it's TLS everywhere it's make sure that that data is secured at all times yeah one of the cool trends at RSA just as a side note was the data in use encryption piece which is a homomorphic stuff is interesting all right guys final question you know we heard on the earlier panel was also trending at reinvent we take the tea out of cloud native it spells cloud naive okay they got shirts now aviatrix kind of got this trend going what does that mean to be naive so if you're to your peers out there watching a live stream and also the suppliers that are trying to supply you guys with technology and services what's naive look like and what's native look like when is someone naive about implementing all this stuff so for me it's because we are in hundred-percent cloud for us it's main thing is ready for the change and you will you will find new building blocks coming in and the network design will evolve and change so don't be naive and think that it's static you wall with the change I think the big naivety that people have is that well I've been doing it this way for 20 years and been successful it's going to be successful in cloud the reality is that's not the case you have to think some of the stuff a little bit differently and you need to think about it early enough so that you can become cloud native and really enable your business on cloud yeah for me it's it's being open minded right the the our industry the network industry as a whole has been very much I am smarter than everybody else and we're gonna tell everybody how it's going to be done and we had we fell into a lull when it came to producing infrastructure and and and so embracing this idea that we can deploy a new solution or a new environment in minutes as opposed to hours or weeks or four months in some cases is really important and and so you know it's are you being closed-minded native being open minded exactly and and it took a for me it was that was a transformative kind of where I was looking to solve problems in a cloud way as opposed to looking to solve problems in this traditional old-school way all right I know we're out of time but I ask one more question so you guys so good it could be a quick answer what's the BS language when you the BS meter goes off when people talk to you about solutions what's the kind of jargon that you hear that's the BS meter going off what are people talking about that in your opinion you here you go that's total BS but what triggers use it so that I have two lines out of movies that are really I can if I say them without actually thinking them it's like 1.21 jigowatts are you out of your mind from Back to the Future right somebody's getting a bang and then and then Martin Mull and and Michael Keaton and mr. mom when he goes to 22 21 whatever it takes yeah those two right there if those go off in my mind somebody's talking to me I know they're full of baloney so a lot of speech would be a lot of speeds and feeds a lot of data did it instead of talking about what you're actually doing and solutioning for you're talking about well I does this this this and any time I start seeing the cloud vendor start benchmarking against each other it's your workload is your workload you need to benchmark yourself don't don't listen to the marketing on that that's that's all what triggers you and the bsp I think if somebody explains you and not simple they cannot explain you in simplicity then that's good all right guys thanks for the great insight great time how about a round of applause DX easy solutions integrating company than we service customers from all industry verticals and we're helping them to move to the digital world so as a solutions integrator we interface with many many customers that have many different types of needs and they're on their IT journey to modernize their applications into the cloud so we encounter many different scenarios many different reasons for those migrations all of them seeking to optimize their IT solutions to better enable their business we have our CPS organization it's cloud platform services we support AWS does your Google Alibaba corkle will help move those workloads to wherever it's most appropriate no one buys the house for the plumbing equally no one buys the solution for the networking but if the plumbing doesn't work no one likes the house and if this network doesn't work no one likes a solution so network is ubiquitous it is a key component of every solution we do the network connectivity is the lifeblood of any architecture without network connectivity nothing works properly planning and building a scalable robust network that's gonna be able to adapt with the application needs critical when encountering some network design and talking about speed the deployment aviatrix came up in discussion and we then further pursued an area DHT products have incorporated aviatrix is part of a new offering that we are in the process of developing that really enhances our ability to provide cloud connectivity for the Lyons cloud connectivity is a new line of networking services so we're getting into as our clients moving the hybrid cloud networking it is much different than our traditional based services and aviatrix provides a key component in that service before we found aviatrix we were using just native peering connections but there wasn't a way to visualize all those peering connections and with multiple accounts multiple contacts for security with a VA Church were able to visualize those different peering connections of security groups it helped a lot especially in areas of early deployment scenarios were quickly able to then take those deployment scenarios and turn them into scripts that we can then deploy repeatedly their solutions were designed to work with the cloud native capabilities first and where those cloud native capabilities fall short they then have solution sets that augment those capabilities I was pleasantly surprised number one with the aviatrix team as a whole and their level of engagement with us you know we weren't only buying the product we were buying a team that came on board to help us implement and solution that was really good to work together to learn both what aviatrix had to offer as well as enhancements that we had to bring that aviatrix was able to put into their product and meet our needs even better aviatrix was a joy to find because they really provided us the technology that we needed in order to provide multi cloud connectivity that really added to the functionality that you can't get from the basically providing services we're taking our customers on a journey to simplify and optimize their IT maybe Atrix certainly has made my job much easier okay welcome back to altitude 2020 for the digital event for the live feed welcome back I'm John Ford with the cube with Steve Mulaney CEO aviatrix for the next panel from global system integrators the folks who are building and working with folks on their journey to multi cloud and cloud native networking we've got a great panel George Buckman with dxc and Derek Monahan with wwt welcome to the stage [Applause] [Music] okay you guys are the ones out there advising building and getting down and dirty with multi cloud and cloud native network and we just heard from the customer panel you can see the diversity of where people come in to the journey of cloud it kind of depends upon where you are but the trends are all clear cloud native networking DevOps up and down the stack this has been the main engine what's your guys take of the disk Jerry to multi cloud what do you guys seeing yeah it's it's critical I mean we're seeing all of our enterprise customers enter into this they've been through the migrations of the easy stuff you know now they're trying to optimize and get more improvement so now the tough stuffs coming on right and you know they need their data processing near where their data is so that's driving them to a multi cloud environment okay we heard some of the edge stuff I mean you guys are exactly you've seen this movie before but now it's a whole new ballgame what's your take yeah so I'll give you a hint so our practice it's not called the cloud practice it's the multi cloud practice and so if that gives you a hint of how we approach things it's very consultative and so when we look at what the trends are let's look a little year ago about a year ago we're having conversations with customers let's build a data center in the cloud let's put some VP C's let's throw some firewalls with some DNS and other infrastructure out there and let's hope it works this isn't a science project so what we're trying to see is customers are starting to have more of a vision and we're helping with that consultative nature but it's totally based on the business and you got to start understanding how the lines of business are using the and then we evolved into the next journey which is a foundational approach to what are some of the problem statement customers are solving when they come to you what are the top things that are on their my house or the ease of use of Julie all that stuff but what specifically they digging into yeah so complexity I think when you look at a multi cloud approach in my view is network requirements are complex you know I think they are but I think the approach can be let's simplify that so one thing that we try to do this is how we talk to customers is let's just like you simplify an aviatrix simplifies the automation orchestration of cloud networking we're trying to simplify the design the planning implementation of infrastructure across multiple workloads across multiple platforms and so the way we do it is we sit down we look at not just use cases and not just the questions in common we tis anticipate we actually build out based on the business and function requirements we build out a strategy and then create a set of documents and guess what we actually build in the lab and that lab that we platform we built proves out this reference architecture actually works absolutely we implement similar concepts I mean we they're proven practices they work great so well George you mentioned that the hard part's now upon us are you referring to networking what is specifically were you getting at Terrance's the easy parts done now so for the enterprises themselves migrating their more critical apps or more difficult apps into the environments you know they've just we've just scratched the surface I believe on what enterprises are doing to move into the cloud to optimize their environments to take advantage of the scale and speed to deployment and to be able to better enable their businesses so they're just now really starting the - so do you get you guys see what I talked about them in terms of their Cambrian explosion I mean you're both monster system integrators with you know top fortune enterprise customers you know really rely on you for for guidance and consulting and so forth and boy they're networks is that something that you you've seen I mean does that resonate did you notice a year and a half ago and all of a sudden the importance of cloud for enterprise shoot up yeah I mean we're seeing it not okay in our internal environment as you know we're a huge company or as customers so we're experiencing that internal okay and every one of our other customers so I have another question oh but I don't know the answer to this and the lawyer never asks a question that you don't know the answer to but I'm gonna ask it anyway DX c + w WT massive system integrators why aviatrix yep so great question Steve so I think the way we approach things I think we have a similar vision a similar strategy how you approach things how we approach things that world by technology number one we want to simplify the complexity and so that's your number one priorities let's take the networking let's simplify it and I think part of the other point I'm making is we have we see this automation piece as not just an afterthought anymore if you look at what customers care about visibility and automation is probably the top three maybe the third on the list and I think that's where we see the value and I think the partnership that we're building and what I would I get excited about is not just putting yours in our lab and showing customers how it works is Co developing a solution with you figuring out hey how can we make this better right visibility's a huge thing jump in security alone network everything's around visibility what automation do you see happening in terms of progression order of operations if you will it's a low-hanging fruit what are people working on now what are what are some of the aspirational goals around when you start thinking about multi cloud and automation yep so I wanted to get back to answer that question I want to answer your question you know what led us there and why aviatrix you know in working some large internal IT projects and and looking at how we were gonna integrate those solutions you know we like to build everything with recipes where network is probably playing catch-up in the DevOps world but with a DevOps mindset looking to speed to deploy support all those things so when you start building your recipes you take a little of this a little of that and you mix it all together well when you look around you say wow look there's this big bag of a VHS let me plop that in that solves a big part of my problems that I have to speed to integrate speed to deploy and the operational views that I need to run this so that was 11 years about reference architectures yeah absolutely so you know they came with a full slate of reference textures already the out there and ready to go that fit our needs so it's very very easy for us to integrate those into our recipes what do you guys think about all the multi vendor interoperability conversations that have been going on choice has been a big part of multi-cloud in terms of you know customers want choice they didn't you know they'll put a workload in the cloud that works but this notion of choice and interoperability is become a big conversation it is and I think our approach and that's why we talk to customers is let's let's speed and be risk of that decision making process and how do we do that because the interoperability is key you're not just putting it's not just a single vendor we're talking you know many many vendors I mean think about the average number of cloud application as a customer uses a business and enterprise business today you know it's it's above 30 it's it's skyrocketing and so what we do and we look at it from an interoperability approach is how do things interoperate we test it out we validate it we build a reference architecture it says these are the critical design elements now let's build one with aviatrix and show how this works with aviatrix and I think the the important part there though is the automation piece that we add to it in visibility so I think the visibility is what's what I see lacking across the industry today and the cloud needed that's been a big topic okay in terms of aviatrix as you guys see them coming in they're one of the ones that are emerging and the new brands emerging but multi-cloud you still got the old guard incumbents with huge footprints how our customers dealing with that that kind of component and dealing with both of them yeah I mean where we have customers that are ingrained with a particular vendor and you know we have partnerships with many vendors so our objective is to provide the solution that meets that client and you they all want multi vendor they all want interoperability correct all right so I got to ask you guys a question while we were defining day two operations what does that mean I mean you guys are looking at the big business and technical components of architecture what does day to Operations mean what's the definition of that yeah so I think from our perspective my experience we you know day to operations whether it's it's not just the you know the orchestration piece and setting up and let it a lot of automate and have some you know change control you're looking at this from a data perspective how do I support this ongoing and make it easy to make changes as we evolve the the the cloud is very dynamic the the nature of how the fast is expanding the number of features is astonish trying to keep up to date with a number of just networking capabilities and services that are added so I think day to operation starts with a fundable understanding of you know building out supporting a customer's environments and making it the automation piece easy from from you know a distance I think yeah and you know taking that to the next level of being able to enable customers to have catalog items that they can pick and choose hey I need this network connectivity from this cloud location back to this on pram and being able to have that automated and provisioned just simply by ordering it for the folks watching out there guys take a minute to explain as you guys are in the trenches doing a lot of good work what are some of the engagement that you guys get into how does that progress what is that what's what happens do they call you up and say hey I need some multi-cloud or you're already in there I mean take us through why how someone can engage to use a global si to come in and make this thing happen what's looks like typical engagement look like yeah so from our perspective we typically have a series of workshops in a methodology that we kind of go along the journey number one we have a foundational approach and I don't mean foundation meaning the network foundation that's a very critical element we got a factor in security we've got a factor in automation so we think about foundation we do a workshop that starts with education a lot of times we'll go in and we'll just educate the customer what is VP she's sharing you know what is a private Lincoln or how does that impact your business we have customers I want to share services out in an ecosystem with other customers and partners well there's many ways to accomplish that so our goal is to you know understand those requirements and then build that strategy with them thoughts Georgia yeah I mean I'm one of the guys that's down in the weeds making things happen so I'm not the guy on the front line interfacing with the customers every day but we have a similar approach you know we have a consulting practice that will go out and and apply their practices to see what those and when do you parachute in yeah and when I then is I'm on the back end working with our offering development leads for the networking so we understand or seeing what customers are asking for and we're on the back end developing the solutions that integrate with our own offerings as well as enable other customers to just deploy quickly to beep their connectivity needs it so the patterns are similar right final question for you guys I want to ask you to paint a picture of what success looks like and you know the name customers didn't forget in reveal kind of who they are but what does success look like in multi-cloud as you paint a picture for the folks here and watching on the live stream it's someone says hey I want to be multi-cloud I got to have my operations agile I want full DevOps I want programmability security built in from day zero what does success look like yeah I think success looks like this so when you're building out a network the network is a harder thing to change than some other aspects of cloud so what we think is even if you're thinking about that second cloud which we have most of our customers are on to public clouds today they might be dabbling in that as you build that network foundation that architecture that takes in consideration where you're going and so once we start building that reference architecture out that shows this is how to sit from a multi cloud perspective not a single cloud and let's not forget our branches let's not forget our data centers let's not forget how all this connects together because that's how we define multi-cloud it's not just in the cloud it's on Prem and it's off from and so collectively I think the key is also is that we provide them an hld you got to start with a high level design that can be tweaked as you go through the journey but you got to give a solid structural foundation and that that networking which we think most customers think as not not the network engineers but as an afterthought we want to make that the most critical element before you start the journey Jorge from your seed how do you success look for you so you know it starts out on these journeys often start out people not even thinking about what is gonna happen what what their network needs are when they start their migration journey to the cloud so I want this success to me looks like them being able to end up not worrying about what's happening in the network when they move to the cloud good point guys great insight thanks for coming on share and pen I've got a round of applause the global system integrators Hey [Applause] [Music] okay welcome back from the live feed I'm chef for with the cube Steve Eleni CEO of aviatrix my co-host our next panel is the aviatrix certified engineers also known as aces this is the folks that are certified their engineering they're building these new solutions please welcome Toby Foster min from Attica Stacy linear from Teradata and Jennifer Reid with Victor Davis to the stage I was just gonna I was just gonna rip you guys see where's your jackets and Jen's got the jacket on okay good love the aviatrix aces pile of gear they're above the clouds towards a new heights that's right so guys aviatrix aces love the name I think it's great certified this is all about getting things engineered so there's a level of certification I want to get into that but first take us through the day in the life of an ace and just to point out Stacey's a squad leader so he's like a Squadron Leader Roger and leader yeah Squadron Leader so he's got a bunch of aces underneath him but share your perspective day-in-the-life Jennifer will start with you sure so I have actually a whole team that works for me both in the in the North America both in the US and in Mexico and so I'm eagerly working to get them certified as well so I can become a squad leader myself but it's important because one of the the critical gaps that we've found is people having the networking background because they're you graduate from college and you have a lot of computer science background you can program you've got Python but now working in packets they just don't get and so just taking them through all the processes that it's really necessary to understand when you're troubleshooting is really critical mm-hmm and because you're gonna get an issue where you need to figure out where exactly is that happening on the network you know is my my issue just in the VP C's and on the instance side is a security group or is it going on print and this is something actually embedded within Amazon itself I mean I should troubleshot an issue for about six months going back and forth with Amazon and it was the vgw VPN because they were auto-scaling on two sides and we ended up having to pull out the Cisco's and put in aviatrix so I could just say okay it's fixed and I actually actually helped the application teams get to that and get it solved yeah but I'm taking a lot of junior people and getting them through that certification process so they can understand and see the network the way I see the network I mean look I've been doing this for 25 years when I got out when I went in the Marine Corps that's what I did and coming out the network is still the network but people don't get the same training they get they got in the 90s it's just so easy just write some software they work takes care of itself yes he'll be we'll come back to that I want to come back to that problem solve with Amazon but Toby I think the only thing I have to add to that is that it's always the network fault as long as I've been in network have always been the network's fault sure and I'm even to this day you know it's still the network's fault and part of being a network guy is that you need to prove when it is and when it's not your fault and that means you need to know a little bit about a hundred different things to make that and now you've got a full stack DevOps you got to know a lot more times another hundred and these times are changing they see your squadron leader I get that right what is what is a squadron leader first can you describe what it is I think it probably just leading all the network components of it but are they from my perspective when to think about what you asked them was it's about no issues and no escalation soft my day is like that's a good outcome that's a good day it's a good day Jennifer you mentioned the Amazon thing this brings up a good point you know when you have these new waves come in you have a lot of new things newly use cases a lot of the finger-pointing it's that guys problem that girls problem so what is how do you solve that and how do you get the young guns up to speed is there training is that this is where the certification comes in those where the certification is really going to come in I know when we we got together at reinvent one of the the questions that that we had with Stephen the team was what what should our certification look like you know she would just be teaching about what aviatrix troubleshooting brings to bear but what should that be like and I think Toby and I were like no no no that's going a little too high we need to get really low because the the better someone can get at actually understanding what actually happening in the network and and where to actually troubleshoot the problem how to step back each of those processes because without that it's just a big black box and they don't know you know because everything is abstracted in Amazon Internet and Azure and Google is substracted and they have these virtual gateways they have VPNs that you just don't have the logs on it's you just don't know and so then what tools can you put in front of them of where they can look because there are full logs well as long as they turned on the flow logs when they built it you know and there's like each one of those little things that well if they'd had decided to do that when they built it it's there but if you can come in later to really supplement that with training to actual troubleshoot and do a packet capture here as it's going through then teaching them how to read that even yeah Toby we were talking before he came on up on stage about your career you've been networking all your time and then you know you're now mentoring a lot of younger people how is that going because the people who come in fresh they don't have all the old war stories they don't know you talk about you know that's dimmer fault I walk in Mayr feet in the snow when I was your age I mean it's so easy now right they say what's your take on how you train the young P so I've noticed two things one is that they are up to speed a lot faster in generalities of networking they can tell you what a network is in high school level now where I didn't learn that too midway through my career and they're learning it faster but they don't necessarily understand why it's that way or you know everybody thinks that it's always slash 24 for a subnet and they don't understand why you can break it down smaller why it's really necessary so the the ramp up speed is much faster for these guys that are coming in but they don't understand why and they need some of that background knowledge to see where it's coming from and why is it important and that's old guys that's where we thrive Jennifer you mentioned you you got in from the Marines health spa when you got into networking how what was it like then and compare it now most like we've heard earlier static versus dynamic don't be static cuz back then you just said the network you got a perimeter yeah no there was no such thing ya know so back in the day I mean I mean we had banyan vines for email and you know we had token ring and I had to set up token ring networks and figure out why that didn't work because how many of things were actually sharing it but then actually just cutting fiber and running fiber cables and dropping them over you know shelters to plug them in and oh crap they swung it too hard and shattered it now I gotta be great polished this thing and actually shoot like to see if it works I mean that was the network current five cat 5 cables to run an Ethernet you know and then from that just said network switches dumb switches like those were the most common ones you had then actually configuring routers and you know logging into a Cisco router and actually knowing how to configure that and it was funny because I had gone all the way up and was a software product manager for a while so I've gone all the way up the stack and then two and a half three years ago I came across to to work with entity group that became Victor Davis but we went to help one of our customers Avis and it was like okay so we need to fix the network okay I haven't done this in 20 years but all right let's get to it you know because it really fundamentally does not change it's still the network I mean I've had people tell me well you know when we go to containers we will not have to worry about the network and I'm like yeah you don't I do and then with this within the program abilities it really interesting so I think this brings up the certification what are some of the new things that people should be aware of that come in with the aviatrix ace certification what are some of the highlights can you guys share some of the some of the highlights around the certifications I think some of the importance is that it's it doesn't need to be vendor specific for network generality or basic networking knowledge and instead of learning how Cisco does something or how Palo Alto does something we need to understand how and why it works as a basic model and then understand how each vendor has gone about that problem and solved it in a general that's true in multi cloud as well you can't learn how cloud networking works without understanding how AWS integer and GCP are all slightly the same but slightly different and some things work and some things don't I think that's probably the number one take I think having a certification across clouds is really valuable because we heard the global si you help the business issues what does it mean to do that is it code is that networking is it configuration is that aviatrix what is the amine oxy aviatrix is a certification but what is it about the multi cloud that makes it multi networking and multi vendor and easy answer is yes so you got to be a general let's go to your hands and all you have to be it takes experience because it's every every cloud vendor has their own certification whether that's hops and [Music] advanced networking and advanced security or whatever it might be yeah they can take the test but they have no idea how to figure out what's wrong with that system and the same thing with any certification but it's really getting your hands in there and actually having to troubleshoot the problems you know actually work the problem you know and calm down it's going to be okay I mean because I don't know how many calls I've been on or even had aviatrix join me on it's like okay so everyone calm down let's figure out what's happening it's like we've looked at that screen three times looking at it again it's not going to solve that problem right but at the same time you know remaining calm but knowing that it really is I'm getting a packet from here to go over here it's not working so what could be the problem you know and actually stepping them through those scenarios but that's like you only get that by having to do it you know and seeing it and going through it and then I have a question so we you know I just see it we started this program maybe six months ago we're seeing a huge amount of interest I mean we're oversubscribed on all the training sessions we've got people flying from around the country even with coronavirus flying to go to Seattle to go to these events were oversubscribed a good is that watching leader would put there yeah something that you see in your organizations are you recommending that to people do you see I mean I'm just I would guess I'm surprised I'm not surprised but I'm really surprised by the demand if you would of this multi-cloud network certification because it really isn't anything like that is that something you guys can comment on or do you see the same things in your organization's I say from my side because we operate in the multi cloud environment so it really helps an official for us I think I would add that networking guys have always needed to use certifications to prove that they know what they know it's not good enough to say yeah I know IP addresses or I know how a network works and a couple little check marks or a little letters buying helps give you validity so even in our team we can say hey you know we're using these certifications to know that you know enough of the basics enough of the understandings that you have the tools necessary right so okay I guess my final question for you guys is why an eighth certification is relevant and then second part is share what the livestream folks who aren't yet a certified or might want to jump in to be AVH or certified engineers why is it important so why is it relevant and why shouldn't someone want to be an ace-certified I'm used to right engineer I think my views a little different I think certification comes from proving that you have the knowledge not proving that you get a certification to get no I mean they're backwards so when you've got the training and the understanding and the you use that to prove and you can like grow your certification list with it versus studying for a test to get a certification and have no understanding it okay so that who is the right person that look at this is saying I'm qualified is it a network engineer is it a DevOps person what's your view you know is it a certain you know I think cloud is really the answer it's the as we talked like the edge is getting eroded so is the network definition getting eroded we're getting more and more of some network some DevOps some security lots and lots of security because network is so involved in so many of them that's just the next progression I don't say I expend that to more automation engineers because we have those nails probably well I think that the training classes themselves are helpful especially the entry-level ones for people who may be quote-unquote cloud architects but I've never done anything and networking for them to understand why we need those things to really work whether or not they go through to eventually get a certification is something different but I really think fundamentally understanding how these things work it makes them a better architect makes some better application developer but even more so as you deploy more of your applications into the cloud really getting an understanding even from our people who've tradition down on prime networking they can understand how that's going to work in the cloud too well I know we got just under 30 seconds left but I want to get one more question than just one more for the folks watching that are you may be younger that don't have that networking training from your experiences each of you can answer why is it should they know about networking what's the benefit what's in it for them motivate them share some insights and why they should go a little bit deeper in networking Stacey we'll start with you we'll go down let's say it's probably fundamental right if you want to deliver solutions no we're going use the very top I would say if you fundamental of an operating system running on a machine how those machines talk together as a fundamental change is something that starts from the base and work your way up right well I think it's a challenge because you've come from top-down now you're gonna start looking from bottom up and you want those different systems to cross communicate and say you've built something and you're overlapping IP space not that that doesn't happen but how can I actually make that still operate without having to reappear e-platform it's like those challenges like those younger developers or sis engineers can really start to get their hands around and understand those complexities and bring that forward in their career they got to know the how the pipes are working you guys know what's going some plumbing that's right and they gotta know how it works I had a code it it's right awesome thank you guys for great insights ace certain ABS your certified engineers also known as aces give a round of applause thank you okay all right that concludes my portion thank you Steve thanks for have Don thank you very much that was fantastic everybody round of applause for John for you yeah so great event great event I'm not gonna take long we got we've got lunch outside for that for the people here just a couple of things just call to action right so we saw the aces you know for those of you out on the stream here become a certified right it's great for your career it's great for not knowledge is is fantastic it's not just an aviatrix thing it's gonna teach you about cloud networking multi-cloud networking with a little bit of aviatrix exactly what the Cisco CCIE program was for IP network that type of the thing that's number one second thing is is is is learn right so so there's a there's a link up there for the four to join the community again like I started this this is a community this is the kickoff to this community and it's a movement so go to what a v8 community aviatrix comm starting a community a multi cloud so you know get get trained learn I'd say the next thing is we're doing over a hundred seminars in across the United States and also starting into Europe soon will come out and will actually spend a couple hours and talk about architecture and talk about those beginning things for those of you on the you know on the livestream in here as well you know we're coming to a city near you go to one of those events it's a great way to network with other people that are in the industry as well as to start to learn and get on that multi-cloud journey and then I'd say the last thing is you know we haven't talked a lot about what aviatrix does here and that's intentional we want you you know leaving with wanting to know more and schedule get with us in schedule a multi our architecture workshop session so we we sit out with customers and we talk about where they're at in that journey and more importantly where they're going and define that end state architecture from networking compute storage everything and everything you heard today every panel kept talking about architecture talking about operations those are the types of things that we solve we help you define that canonical architecture that system architecture that's yours so for so many of our customers they have three by five plotted lucid charts architecture drawings and it's the customer name slash aviatrix arc network architecture and they put it on their whiteboard that's what what we and that's the most valuable thing they get from us so this becomes their twenty-year network architecture drawing that they don't do anything without talking to us and look at that architecture that's what we do in these multi hour workshop sessions with customers and that's super super powerful so if you're interested definitely call us and let's schedule that with our team so anyway I just want to thank everybody on the livestream thank everybody here hopefully it was it was very useful I think it was and joined the movement and for those of you here join us for lunch and thank you very much [Applause] [Music] you