(upbeat music) >> Hello everyone. Welcome to theCube's presentation of the "AWS Startup Showcase." The topic this episode is AI and machine learning, top startups building foundational model infrastructure. This is season three, episode one of the ongoing series covering exciting startups from the AWS ecosystem. And this time we're talking about AI and machine learning. I'm your host, John Furrier. I'm excited I'm joined today by Robert Nishihara, who's the co-founder and CEO of a hot startup called Anyscale. He's here to talk about Ray, the open source project, Anyscale's infrastructure for foundation as well. Robert, thank you for joining us today. >> Yeah, thanks so much as well. >> I've been following your company since the founding pre pandemic and you guys really had a great vision scaled up and in a perfect position for this big wave that we all see with ChatGPT and OpenAI that's gone mainstream. Finally, AI has broken out through the ropes and now gone mainstream, so I think you guys are really well positioned. I'm looking forward to to talking with you today. But before we get into it, introduce the core mission for Anyscale. Why do you guys exist? What is the North Star for Anyscale? >> Yeah, like you mentioned, there's a tremendous amount of excitement about AI right now. You know, I think a lot of us believe that AI can transform just every different industry. So one of the things that was clear to us when we started this company was that the amount of compute needed to do AI was just exploding. Like to actually succeed with AI, companies like OpenAI or Google or you know, these companies getting a lot of value from AI, were not just running these machine learning models on their laptops or on a single machine. They were scaling these applications across hundreds or thousands or more machines and GPUs and other resources in the Cloud. And so to actually succeed with AI, and this has been one of the biggest trends in computing, maybe the biggest trend in computing in, you know, in recent history, the amount of compute has been exploding. And so to actually succeed with that AI, to actually build these scalable applications and scale the AI applications, there's a tremendous software engineering lift to build the infrastructure to actually run these scalable applications. And that's very hard to do. So one of the reasons many AI projects and initiatives fail is that, or don't make it to production, is the need for this scale, the infrastructure lift, to actually make it happen. So our goal here with Anyscale and Ray, is to make that easy, is to make scalable computing easy. So that as a developer or as a business, if you want to do AI, if you want to get value out of AI, all you need to know is how to program on your laptop. Like, all you need to know is how to program in Python. And if you can do that, then you're good to go. Then you can do what companies like OpenAI or Google do and get value out of machine learning. >> That programming example of how easy it is with Python reminds me of the early days of Cloud, when infrastructure as code was talked about was, it was just code the infrastructure programmable. That's super important. That's what AI people wanted, first program AI. That's the new trend. And I want to understand, if you don't mind explaining, the relationship that Anyscale has to these foundational models and particular the large language models, also called LLMs, was seen with like OpenAI and ChatGPT. Before you get into the relationship that you have with them, can you explain why the hype around foundational models? Why are people going crazy over foundational models? What is it and why is it so important? >> Yeah, so foundational models and foundation models are incredibly important because they enable businesses and developers to get value out of machine learning, to use machine learning off the shelf with these large models that have been trained on tons of data and that are useful out of the box. And then, of course, you know, as a business or as a developer, you can take those foundational models and repurpose them or fine tune them or adapt them to your specific use case and what you want to achieve. But it's much easier to do that than to train them from scratch. And I think there are three, for people to actually use foundation models, there are three main types of workloads or problems that need to be solved. One is training these foundation models in the first place, like actually creating them. The second is fine tuning them and adapting them to your use case. And the third is serving them and actually deploying them. Okay, so Ray and Anyscale are used for all of these three different workloads. Companies like OpenAI or Cohere that train large language models. Or open source versions like GPTJ are done on top of Ray. There are many startups and other businesses that fine tune, that, you know, don't want to train the large underlying foundation models, but that do want to fine tune them, do want to adapt them to their purposes, and build products around them and serve them, those are also using Ray and Anyscale for that fine tuning and that serving. And so the reason that Ray and Anyscale are important here is that, you know, building and using foundation models requires a huge scale. It requires a lot of data. It requires a lot of compute, GPUs, TPUs, other resources. And to actually take advantage of that and actually build these scalable applications, there's a lot of infrastructure that needs to happen under the hood. And so you can either use Ray and Anyscale to take care of that and manage the infrastructure and solve those infrastructure problems. Or you can build the infrastructure and manage the infrastructure yourself, which you can do, but it's going to slow your team down. It's going to, you know, many of the businesses we work with simply don't want to be in the business of managing infrastructure and building infrastructure. They want to focus on product development and move faster. >> I know you got a keynote presentation we're going to go to in a second, but I think you hit on something I think is the real tipping point, doing it yourself, hard to do. These are things where opportunities are and the Cloud did that with data centers. Turned a data center and made it an API. The heavy lifting went away and went to the Cloud so people could be more creative and build their product. In this case, build their creativity. Is that kind of what's the big deal? Is that kind of a big deal happening that you guys are taking the learnings and making that available so people don't have to do that? >> That's exactly right. So today, if you want to succeed with AI, if you want to use AI in your business, infrastructure work is on the critical path for doing that. To do AI, you have to build infrastructure. You have to figure out how to scale your applications. That's going to change. We're going to get to the point, and you know, with Ray and Anyscale, we're going to remove the infrastructure from the critical path so that as a developer or as a business, all you need to focus on is your application logic, what you want the the program to do, what you want your application to do, how you want the AI to actually interface with the rest of your product. Now the way that will happen is that Ray and Anyscale will still, the infrastructure work will still happen. It'll just be under the hood and taken care of by Ray in Anyscale. And so I think something like this is really necessary for AI to reach its potential, for AI to have the impact and the reach that we think it will, you have to make it easier to do. >> And just for clarification to point out, if you don't mind explaining the relationship of Ray and Anyscale real quick just before we get into the presentation. >> So Ray is an open source project. We created it. We were at Berkeley doing machine learning. We started Ray so that, in order to provide an easy, a simple open source tool for building and running scalable applications. And Anyscale is the managed version of Ray, basically we will run Ray for you in the Cloud, provide a lot of tools around the developer experience and managing the infrastructure and providing more performance and superior infrastructure. >> Awesome. I know you got a presentation on Ray and Anyscale and you guys are positioning as the infrastructure for foundational models. So I'll let you take it away and then when you're done presenting, we'll come back, I'll probably grill you with a few questions and then we'll close it out so take it away. >> Robert: Sounds great. So I'll say a little bit about how companies are using Ray and Anyscale for foundation models. The first thing I want to mention is just why we're doing this in the first place. And the underlying observation, the underlying trend here, and this is a plot from OpenAI, is that the amount of compute needed to do machine learning has been exploding. It's been growing at something like 35 times every 18 months. This is absolutely enormous. And other people have written papers measuring this trend and you get different numbers. But the point is, no matter how you slice and dice it, it' a astronomical rate. Now if you compare that to something we're all familiar with, like Moore's Law, which says that, you know, the processor performance doubles every roughly 18 months, you can see that there's just a tremendous gap between the needs, the compute needs of machine learning applications, and what you can do with a single chip, right. So even if Moore's Law were continuing strong and you know, doing what it used to be doing, even if that were the case, there would still be a tremendous gap between what you can do with the chip and what you need in order to do machine learning. And so given this graph, what we've seen, and what has been clear to us since we started this company, is that doing AI requires scaling. There's no way around it. It's not a nice to have, it's really a requirement. And so that led us to start Ray, which is the open source project that we started to make it easy to build these scalable Python applications and scalable machine learning applications. And since we started the project, it's been adopted by a tremendous number of companies. Companies like OpenAI, which use Ray to train their large models like ChatGPT, companies like Uber, which run all of their deep learning and classical machine learning on top of Ray, companies like Shopify or Spotify or Instacart or Lyft or Netflix, ByteDance, which use Ray for their machine learning infrastructure. Companies like Ant Group, which makes Alipay, you know, they use Ray across the board for fraud detection, for online learning, for detecting money laundering, you know, for graph processing, stream processing. Companies like Amazon, you know, run Ray at a tremendous scale and just petabytes of data every single day. And so the project has seen just enormous adoption since, over the past few years. And one of the most exciting use cases is really providing the infrastructure for building training, fine tuning, and serving foundation models. So I'll say a little bit about, you know, here are some examples of companies using Ray for foundation models. Cohere trains large language models. OpenAI also trains large language models. You can think about the workloads required there are things like supervised pre-training, also reinforcement learning from human feedback. So this is not only the regular supervised learning, but actually more complex reinforcement learning workloads that take human input about what response to a particular question, you know is better than a certain other response. And incorporating that into the learning. There's open source versions as well, like GPTJ also built on top of Ray as well as projects like Alpa coming out of UC Berkeley. So these are some of the examples of exciting projects in organizations, training and creating these large language models and serving them using Ray. Okay, so what actually is Ray? Well, there are two layers to Ray. At the lowest level, there's the core Ray system. This is essentially low level primitives for building scalable Python applications. Things like taking a Python function or a Python class and executing them in the cluster setting. So Ray core is extremely flexible and you can build arbitrary scalable applications on top of Ray. So on top of Ray, on top of the core system, what really gives Ray a lot of its power is this ecosystem of scalable libraries. So on top of the core system you have libraries, scalable libraries for ingesting and pre-processing data, for training your models, for fine tuning those models, for hyper parameter tuning, for doing batch processing and batch inference, for doing model serving and deployment, right. And a lot of the Ray users, the reason they like Ray is that they want to run multiple workloads. They want to train and serve their models, right. They want to load their data and feed that into training. And Ray provides common infrastructure for all of these different workloads. So this is a little overview of what Ray, the different components of Ray. So why do people choose to go with Ray? I think there are three main reasons. The first is the unified nature. The fact that it is common infrastructure for scaling arbitrary workloads, from data ingest to pre-processing to training to inference and serving, right. This also includes the fact that it's future proof. AI is incredibly fast moving. And so many people, many companies that have built their own machine learning infrastructure and standardized on particular workflows for doing machine learning have found that their workflows are too rigid to enable new capabilities. If they want to do reinforcement learning, if they want to use graph neural networks, they don't have a way of doing that with their standard tooling. And so Ray, being future proof and being flexible and general gives them that ability. Another reason people choose Ray in Anyscale is the scalability. This is really our bread and butter. This is the reason, the whole point of Ray, you know, making it easy to go from your laptop to running on thousands of GPUs, making it easy to scale your development workloads and run them in production, making it easy to scale, you know, training to scale data ingest, pre-processing and so on. So scalability and performance, you know, are critical for doing machine learning and that is something that Ray provides out of the box. And lastly, Ray is an open ecosystem. You can run it anywhere. You can run it on any Cloud provider. Google, you know, Google Cloud, AWS, Asure. You can run it on your Kubernetes cluster. You can run it on your laptop. It's extremely portable. And not only that, it's framework agnostic. You can use Ray to scale arbitrary Python workloads. You can use it to scale and it integrates with libraries like TensorFlow or PyTorch or JAX or XG Boost or Hugging Face or PyTorch Lightning, right, or Scikit-learn or just your own arbitrary Python code. It's open source. And in addition to integrating with the rest of the machine learning ecosystem and these machine learning frameworks, you can use Ray along with all of the other tooling in the machine learning ecosystem. That's things like weights and biases or ML flow, right. Or you know, different data platforms like Databricks, you know, Delta Lake or Snowflake or tools for model monitoring for feature stores, all of these integrate with Ray. And that's, you know, Ray provides that kind of flexibility so that you can integrate it into the rest of your workflow. And then Anyscale is the scalable compute platform that's built on top, you know, that provides Ray. So Anyscale is a managed Ray service that runs in the Cloud. And what Anyscale does is it offers the best way to run Ray. And if you think about what you get with Anyscale, there are fundamentally two things. One is about moving faster, accelerating the time to market. And you get that by having the managed service so that as a developer you don't have to worry about managing infrastructure, you don't have to worry about configuring infrastructure. You also, it provides, you know, optimized developer workflows. Things like easily moving from development to production, things like having the observability tooling, the debug ability to actually easily diagnose what's going wrong in a distributed application. So things like the dashboards and the other other kinds of tooling for collaboration, for monitoring and so on. And then on top of that, so that's the first bucket, developer productivity, moving faster, faster experimentation and iteration. The second reason that people choose Anyscale is superior infrastructure. So this is things like, you know, cost deficiency, being able to easily take advantage of spot instances, being able to get higher GPU utilization, things like faster cluster startup times and auto scaling. Things like just overall better performance and faster scheduling. And so these are the kinds of things that Anyscale provides on top of Ray. It's the managed infrastructure. It's fast, it's like the developer productivity and velocity as well as performance. So this is what I wanted to share about Ray in Anyscale. >> John: Awesome. >> Provide that context. But John, I'm curious what you think. >> I love it. I love the, so first of all, it's a platform because that's the platform architecture right there. So just to clarify, this is an Anyscale platform, not- >> That's right. >> Tools. So you got tools in the platform. Okay, that's key. Love that managed service. Just curious, you mentioned Python multiple times, is that because of PyTorch and TensorFlow or Python's the most friendly with machine learning or it's because it's very common amongst all developers? >> That's a great question. Python is the language that people are using to do machine learning. So it's the natural starting point. Now, of course, Ray is actually designed in a language agnostic way and there are companies out there that use Ray to build scalable Java applications. But for the most part right now we're focused on Python and being the best way to build these scalable Python and machine learning applications. But, of course, down the road there always is that potential. >> So if you're slinging Python code out there and you're watching that, you're watching this video, get on Anyscale bus quickly. Also, I just, while you were giving the presentation, I couldn't help, since you mentioned OpenAI, which by the way, congratulations 'cause they've had great scale, I've noticed in their rapid growth 'cause they were the fastest company to the number of users than anyone in the history of the computer industry, so major successor, OpenAI and ChatGPT, huge fan. I'm not a skeptic at all. I think it's just the beginning, so congratulations. But I actually typed into ChatGPT, what are the top three benefits of Anyscale and came up with scalability, flexibility, and ease of use. Obviously, scalability is what you guys are called. >> That's pretty good. >> So that's what they came up with. So they nailed it. Did you have an inside prompt training, buy it there? Only kidding. (Robert laughs) >> Yeah, we hard coded that one. >> But that's the kind of thing that came up really, really quickly if I asked it to write a sales document, it probably will, but this is the future interface. This is why people are getting excited about the foundational models and the large language models because it's allowing the interface with the user, the consumer, to be more human, more natural. And this is clearly will be in every application in the future. >> Absolutely. This is how people are going to interface with software, how they're going to interface with products in the future. It's not just something, you know, not just a chat bot that you talk to. This is going to be how you get things done, right. How you use your web browser or how you use, you know, how you use Photoshop or how you use other products. Like you're not going to spend hours learning all the APIs and how to use them. You're going to talk to it and tell it what you want it to do. And of course, you know, if it doesn't understand it, it's going to ask clarifying questions. You're going to have a conversation and then it'll figure it out. >> This is going to be one of those things, we're going to look back at this time Robert and saying, "Yeah, from that company, that was the beginning of that wave." And just like AWS and Cloud Computing, the folks who got in early really were in position when say the pandemic came. So getting in early is a good thing and that's what everyone's talking about is getting in early and playing around, maybe replatforming or even picking one or few apps to refactor with some staff and managed services. So people are definitely jumping in. So I have to ask you the ROI cost question. You mentioned some of those, Moore's Law versus what's going on in the industry. When you look at that kind of scale, the first thing that jumps out at people is, "Okay, I love it. Let's go play around." But what's it going to cost me? Am I going to be tied to certain GPUs? What's the landscape look like from an operational standpoint, from the customer? Are they locked in and the benefit was flexibility, are you flexible to handle any Cloud? What is the customers, what are they looking at? Basically, that's my question. What's the customer looking at? >> Cost is super important here and many of the companies, I mean, companies are spending a huge amount on their Cloud computing, on AWS, and on doing AI, right. And I think a lot of the advantage of Anyscale, what we can provide here is not only better performance, but cost efficiency. Because if we can run something faster and more efficiently, it can also use less resources and you can lower your Cloud spending, right. We've seen companies go from, you know, 20% GPU utilization with their current setup and the current tools they're using to running on Anyscale and getting more like 95, you know, 100% GPU utilization. That's something like a five x improvement right there. So depending on the kind of application you're running, you know, it's a significant cost savings. We've seen companies that have, you know, processing petabytes of data every single day with Ray going from, you know, getting order of magnitude cost savings by switching from what they were previously doing to running their application on Ray. And when you have applications that are spending, you know, potentially $100 million a year and getting a 10 X cost savings is just absolutely enormous. So these are some of the kinds of- >> Data infrastructure is super important. Again, if the customer, if you're a prospect to this and thinking about going in here, just like the Cloud, you got infrastructure, you got the platform, you got SaaS, same kind of thing's going to go on in AI. So I want to get into that, you know, ROI discussion and some of the impact with your customers that are leveraging the platform. But first I hear you got a demo. >> Robert: Yeah, so let me show you, let me give you a quick run through here. So what I have open here is the Anyscale UI. I've started a little Anyscale Workspace. So Workspaces are the Anyscale concept for interactive developments, right. So here, imagine I'm just, you want to have a familiar experience like you're developing on your laptop. And here I have a terminal. It's not on my laptop. It's actually in the cloud running on Anyscale. And I'm just going to kick this off. This is going to train a large language model, so OPT. And it's doing this on 32 GPUs. We've got a cluster here with a bunch of CPU cores, bunch of memory. And as that's running, and by the way, if I wanted to run this on instead of 32 GPUs, 64, 128, this is just a one line change when I launch the Workspace. And what I can do is I can pull up VS code, right. Remember this is the interactive development experience. I can look at the actual code. Here it's using Ray train to train the torch model. We've got the training loop and we're saying that each worker gets access to one GPU and four CPU cores. And, of course, as I make the model larger, this is using deep speed, as I make the model larger, I could increase the number of GPUs that each worker gets access to, right. And how that is distributed across the cluster. And if I wanted to run on CPUs instead of GPUs or a different, you know, accelerator type, again, this is just a one line change. And here we're using Ray train to train the models, just taking my vanilla PyTorch model using Hugging Face and then scaling that across a bunch of GPUs. And, of course, if I want to look at the dashboard, I can go to the Ray dashboard. There are a bunch of different visualizations I can look at. I can look at the GPU utilization. I can look at, you know, the CPU utilization here where I think we're currently loading the model and running that actual application to start the training. And some of the things that are really convenient here about Anyscale, both I can get that interactive development experience with VS code. You know, I can look at the dashboards. I can monitor what's going on. It feels, I have a terminal, it feels like my laptop, but it's actually running on a large cluster. And I can, with however many GPUs or other resources that I want. And so it's really trying to combine the best of having the familiar experience of programming on your laptop, but with the benefits, you know, being able to take advantage of all the resources in the Cloud to scale. And it's like when, you know, you're talking about cost efficiency. One of the biggest reasons that people waste money, one of the silly reasons for wasting money is just forgetting to turn off your GPUs. And what you can do here is, of course, things will auto terminate if they're idle. But imagine you go to sleep, I have this big cluster. You can turn it off, shut off the cluster, come back tomorrow, restart the Workspace, and you know, your big cluster is back up and all of your code changes are still there. All of your local file edits. It's like you just closed your laptop and came back and opened it up again. And so this is the kind of experience we want to provide for our users. So that's what I wanted to share with you. >> Well, I think that whole, couple of things, lines of code change, single line of code change, that's game changing. And then the cost thing, I mean human error is a big deal. People pass out at their computer. They've been coding all night or they just forget about it. I mean, and then it's just like leaving the lights on or your water running in your house. It's just, at the scale that it is, the numbers will add up. That's a huge deal. So I think, you know, compute back in the old days, there's no compute. Okay, it's just compute sitting there idle. But you know, data cranking the models is doing, that's a big point. >> Another thing I want to add there about cost efficiency is that we make it really easy to use, if you're running on Anyscale, to use spot instances and these preemptable instances that can just be significantly cheaper than the on-demand instances. And so when we see our customers go from what they're doing before to using Anyscale and they go from not using these spot instances 'cause they don't have the infrastructure around it, the fault tolerance to handle the preemption and things like that, to being able to just check a box and use spot instances and save a bunch of money. >> You know, this was my whole, my feature article at Reinvent last year when I met with Adam Selipsky, this next gen Cloud is here. I mean, it's not auto scale, it's infrastructure scale. It's agility. It's flexibility. I think this is where the world needs to go. Almost what DevOps did for Cloud and what you were showing me that demo had this whole SRE vibe. And remember Google had site reliability engines to manage all those servers. This is kind of like an SRE vibe for data at scale. I mean, a similar kind of order of magnitude. I mean, I might be a little bit off base there, but how would you explain it? >> It's a nice analogy. I mean, what we are trying to do here is get to the point where developers don't think about infrastructure. Where developers only think about their application logic. And where businesses can do AI, can succeed with AI, and build these scalable applications, but they don't have to build, you know, an infrastructure team. They don't have to develop that expertise. They don't have to invest years in building their internal machine learning infrastructure. They can just focus on the Python code, on their application logic, and run the stuff out of the box. >> Awesome. Well, I appreciate the time. Before we wrap up here, give a plug for the company. I know you got a couple websites. Again, go, Ray's got its own website. You got Anyscale. You got an event coming up. Give a plug for the company looking to hire. Put a plug in for the company. >> Yeah, absolutely. Thank you. So first of all, you know, we think AI is really going to transform every industry and the opportunity is there, right. We can be the infrastructure that enables all of that to happen, that makes it easy for companies to succeed with AI, and get value out of AI. Now we have, if you're interested in learning more about Ray, Ray has been emerging as the standard way to build scalable applications. Our adoption has been exploding. I mentioned companies like OpenAI using Ray to train their models. But really across the board companies like Netflix and Cruise and Instacart and Lyft and Uber, you know, just among tech companies. It's across every industry. You know, gaming companies, agriculture, you know, farming, robotics, drug discovery, you know, FinTech, we see it across the board. And all of these companies can get value out of AI, can really use AI to improve their businesses. So if you're interested in learning more about Ray and Anyscale, we have our Ray Summit coming up in September. This is going to highlight a lot of the most impressive use cases and stories across the industry. And if your business, if you want to use LLMs, you want to train these LLMs, these large language models, you want to fine tune them with your data, you want to deploy them, serve them, and build applications and products around them, give us a call, talk to us. You know, we can really take the infrastructure piece, you know, off the critical path and make that easy for you. So that's what I would say. And, you know, like you mentioned, we're hiring across the board, you know, engineering, product, go-to-market, and it's an exciting time. >> Robert Nishihara, co-founder and CEO of Anyscale, congratulations on a great company you've built and continuing to iterate on and you got growth ahead of you, you got a tailwind. I mean, the AI wave is here. I think OpenAI and ChatGPT, a customer of yours, have really opened up the mainstream visibility into this new generation of applications, user interface, roll of data, large scale, how to make that programmable so we're going to need that infrastructure. So thanks for coming on this season three, episode one of the ongoing series of the hot startups. In this case, this episode is the top startups building foundational model infrastructure for AI and ML. I'm John Furrier, your host. Thanks for watching. (upbeat music)

>> Narrator: "theCUBE's" live coverage is made possible by funding from Dell Technologies. Creating technologies that drive human progress. (intro music) >> Good afternoon, everyone. Welcome back to "theCUBE's" day three coverage of MWC23. Lisa Martin here in Spain, Barcelona, Spain with Dave Nicholson. We're going to have a really interesting conversation next. We're going to really dig into MWC, it's history, where it's going, some of the controversy here. Please welcome our guests. We have Jim Harris, International Best Selling Author of "Blindsided." And Carolina Milanese is here, President and Principle Analyst of creative strategies. Welcome to "theCUBE" guys. Thank you. >> Thanks. So great to be here. >> So this is day three. 80,000 people or so. You guys have a a lot of history up at this event. Caroline, I want to start with you. Talk a little bit about that. This obviously the biggest one in, in quite a few years. People are ready to be back, but there's been some, a lot of news here, but some controversy going on. Give us the history, and your perspective on some of the news that's coming out from this week's event. >> It feels like a very different show. I don't know if I would say growing up show, because we are still talking about networks and mobility, but there's so much more now around what the networks actually empower, versus the network themselves. And a little bit of maybe that's where some of the controversy is coming from, carriers still trying to find their identity, right, of, of what their role is in all there is to do with a connected world. I go back a long way. I go back to when Mobile World Congress was called, was actually called GSM, and it was in Khan. So, you know, we went from France to Spain. But just looking at the last full Mobile World Congress here in Barcelona, in pre-pandemic to now, very different show. We went from a show that was very much focused on mobility and smartphones, to a show that was all about cars. You know, we had cars everywhere, 'cause we were talking about smart cities and connected cars, to now a show this year that is very much focused on B2B. And so a lot of companies that are here to either work with the carriers, or also talk about sustainability for instance, or enable what is the next future evolution of computing with XR and VR. >> So Jim, talk to us a little bit about your background. You, I was doing a little sleuthing on you. You're really focusing on disruptive innovation. We talk about disruption a lot in different industries. We're seeing a lot of disruption in telco. We're seeing a lot of frenemies going on. Give us your thoughts about what you're seeing at this year's event. >> Well, there's some really exciting things. I listened to the keynote from Orange's CEO, and she was complaining that 55% of the traffic on her network is from five companies. And then the CEO of Deutsche Telecom got up, and he was complaining that 60% of the traffic on his network is from six entities. So do you think they coordinated pre, pre-show? But really what they're saying is, these OTT, you know, Netflix and YouTube, they should be paying us for access. Now, this is killer funny. The front page today of the show, "Daily," the CO-CEO of Netflix says, "Hey, we make less profit than the telcos, "so you should be paying us, "not the other way around." You know, we spend half of the money we make just on developing content. So, this is really interesting. The orange CEO said, "We're not challenging net neutrality. "We don't want more taxes." But boom. So this is disruptive. Huge pressure. 67% of all mobile traffic is video, right? So it's a big hog bandwidth wise. So how are they going to do this? Now, I look at it, and the business model for the, the telcos, is really selling sim cards and smartphones. But for every dollar of revenue there, there's five plus dollars in apps, and consulting and everything else. So really, but look at how they're structured. They can't, you know, take somebody who talks to the public and sells sim cards, and turn 'em in, turn 'em in to an app developer. So how are they going to square this circle? So I see some, they're being disrupted because they're sticking to what they've historically done. >> But it's interesting because at the end of the day, the conversation that we are having right now is the conversation that we had 10 years ago, where carriers don't want to just be a dumb pipe, right? And that's what they are now returning to. They tried to be media as well, but that didn't work out for most carriers, right? It is a little bit better in the US. We've seen, you know, some success there. But, but here has been more difficult. And I think that's the, the concern, that even for the next, you know, evolution, that's the, their role. >> So how do they, how do they balance this dumb pipe idea, with the fact that if you make the toll high enough, being a dumb pipe is actually a pretty good job. You know, sit back, collect check, go to the beach, right? So where, where, where, where does this end up? >> Well, I think what's going to happen is, if you see five to 15 X the revenue on top of a pipe, you know, the hyperscalers are going to start going after the business. The consulting companies like PWC, McKinsey, the app developers, they're... So how do you engage those communities as a telco to get more revenue? I think this is a question that they really need to look at. But we tend to stick within our existing business model. I'll just give you one stat that blows me away. Uber is worth more than every taxi cab company in North America added together. And so the taxi industry owns billions in assets in cars and limousines. Uber doesn't own a single vehicle. So having a widely distributed app, is a huge multiplier on valuation. And I look to a company like Safari in Kenya, which developed M-Pesa, which Pesa means mo, it's mobile money in Swahili. And 25% of the country's GDP is facilitated by M-Pesa. And that's not even on smartphones. They're feature phones, Nokia phones. I call them dumb phones, but Nokia would call them "feature phones." >> Yeah. >> So think about that. Like 25, now transactions are very small, and the cut is tiny. But when you're facilitating 25% of a country's GDP, >> Yeah. >> Tiny, over billions of transactions is huge. But that's not the way telcos have historically thought or worked. And so M-Pesa and Safari shows the way forward. What do you think on that? >> I, I think that the experience, and what they can layer on top from a services perspective, especially in the private sector, is also important. I don't, I never believe that a carrier, given how they operate, is the best media company in the world, right? It is a very different world. But I do think that there's opportunity, first of all, to, to actually tell their story in a different way. If you're thinking about everything that a network actually empowers, there's a, there's a lot there. There's a lot that is good for us as, as society. There's a lot that is good for business. What can they do to start talking about differently about their services, and then layer on top of what they offer? A better way to actually bring together private and public network. It's not all about cellular, wifi and cellular coming together. We're talking a lot about satellite here as well. So, there's definitely more there about quality of service. Is, is there though, almost a biological inevitability that prevents companies from being able to navigate that divide? >> Hmm. >> Look at, look at when, when, when we went from high definition 720P, very exciting, 1080P, 4K. Everybody ran out and got a 4K TV. Well where was the, where was the best 4K content coming from? It wasn't, it wasn't the networks, it wasn't your cable operator, it was YouTube. It was YouTube. If you had suggested that 10 years before, that that would happen, people would think that you were crazy. Is it possible for folks who are now leading their companies, getting up on stage, and daring to say, "This content's coming over, "and I want to charge you more "for using my pipes." It's like, "Really? Is that your vision? "That's the vision that you want to share with us here?" I hear the sound of dead people walking- (laughing) when I hear comments like that. And so, you know, my students at Wharton in the CTO program, who are constantly looking at this concept of disruption, would hear that and go, "Ooh, gee, did the board hear what that person said?" I, you know, am I being too critical of people who could crush me like a bug? (laughing) >> I mean, it's better that they ask the people with money than not consumers to pay, right? 'Cause we've been through a phase where the carriers were actually asking for more money depending on critical things. Like for instance, if you're doing business email, then were going to charge you more than if you were a consumer. Or if you were watching video, they would charge you more for that. Then they understood that a consumer would walk away and go somewhere else. So they stopped doing that. But to your point, I think, and, and very much to what you focus from a disruption perspective, look at what Chat GTP and what Microsoft has been doing. Not much talk about this here at the show, which is interesting, but the idea that now as a consumer, I can ask new Bing to get me the 10 best restaurants in Barcelona, and I no longer go to Yelp, or all the other businesses where I was going to before, to get their recommendation, what happens to them? You're, you're moving away, and you're taking eyeballs away from those websites. And, and I think that, that you know, your point is exactly right. That it's, it's about how, from a revenue perspective, you are spending a lot of money to facilitate somebody else, and what's in it for you? >> Yeah. And to be clear, consumers pay for everything. >> Always. Always. (laughs) >> Taxpayers and consumers always pay for everything. So there is no, "Well, we're going to make them pay, so you don't have to pay." >> And if you are not paying, you are the product. Exactly. >> Yes. (laughing) >> Carolina, talk a little bit about what you're seeing at the event from some of the infrastructure players, the hyperscalers, obviously a lot of enterprise focus here at this event. What are some of the things that you're seeing? Are you impressed with, with their focus in telco, their focus to partner, build an ecosystem? What are you seeing? >> I'm seeing also talk about sustainability, and enabling telco to be more sustainable. You know, there, there's a couple of things that are a little bit different from the US where I live, which is that telcos in Europe, have put money into sustainability through bonds. And so they use the money that they then get from the bonds that they create, to, to supply or to fuel their innovation in sustainability. And so there's a dollar amount on sustainability. There's also an opportunity obviously from a growth perspective. And there's a risk mitigation, right? Especially in Europe, more and more you're going to be evaluated based on how sustainable you are. So there are a lot of companies here, if you're thinking about the Ciscos of the world. Dell, IBM all talking about sustainability and how to help carriers measure, and then obviously be more sustainable with their consumption and, and power. >> Going to be interesting to see where that goes over the years, as we talk to, every company we talk to at whatever show, has an ESG sustainability initiative, and only, well, many of them only want to work with other companies who have the same types of initiative. So a lot of, great that there's focus on sustainability, but hopefully we'll see more action down the road. Wanted to ask you about your book, "Blind," the name is interesting, "Blindsided." >> Well, I just want to tag on to this. >> Sure. >> One of the most exciting things for me is fast charging technology. And Shalmie, cell phone, or a smartphone maker from China, just announced yesterday, a smartphone that charges from 0 to 100% in five minutes. Now this is using GAN FEST technology. And the leader in the market is a company called Navitas. And this has profound implications. You know, it starts with the smartphone, right? But then it moves to the laptops. And then it'll move to EV's. So, as we electrify the $10 trillion a year transportation industry, there's a huge opportunity. People want charging faster. There's also a sustainability story that, to Carolina's point, that it uses less electricity. So, if we electrify the grid in order to support transportation, like the Tesla Semi's coming out, there are huge demands over a period. We need energy efficiency technologies, like this GAN FEST technology. So to me, this is humongous. And it, we only see it here in the show, in Shalmie, saying, "Five minutes." And everybody, the consumers go, "Oh, that's cool." But let's look at the bigger story, which is electrifying transportation globally. And this is going to be big. >> Yeah. And, and to, and to double click on that a little bit, to be clear, when we talk about fast charging today, typically it's taking the battery from a, not a zero state of charge, but a relatively low state of charge to 80%. >> Yep. >> Then it tapers off dramatically. And that translates into less range in an EV, less usable time on any other device, and there's that whole linkage between the power in, and the battery's ability to be charged, and how much is usable. And from a sustainability perspective, we are going to have an avalanche of batteries going into secondary use cases over time. >> They don't get tossed into landfills contrary to what people might think. >> Yep. >> In fact, they are used in a variety of ways after their primary lifespan. But that, that is, that in and of itself is a revolutionary thing. I'm interested in each of your thoughts on the China factor. Glaringly absent here, from my perspective, as sort of an Apple fanboy, where are they? Why aren't they talking about their... They must, they must feel like, "Well we just don't need to." >> We don't need to. We just don't need to. >> Absolutely. >> And then you walk around and you see these, these company names that are often anglicized, and you don't necessarily immediately associate them with China, but it's like, "Wait a minute, "that looks better than what I have, "and I'm not allowed to have access to that thing." What happens in the future there geopolitically? >> It's a pretty big question for- >> Its is. >> For a short little tech show. (Caroline laughs) But what happens as we move forward? When is the entire world going to be able to leverage in a secure way, some of the stuff that's coming out of, if they're not the largest economy in the world yet, they shortly will be. >> What's the story there? >> Well, it's interesting that you mentioned First Apple that has never had a presence at Mobile World Congress. And fun enough, I'm part of the GSMA judges for the GLOMO Awards, and last night I gave out Best Mobile Phone for last year, and it was to the iPhone4 Team Pro. and best disruptive technology, which was for the satellite function feature on, on the new iPhone. So, Apple might not be here, but they are. >> Okay. >> And, and so that's the first thing. And they are as far as being top of mind to every competitor in the smartphone market still. So a lot of the things that, even from a design perspective that you see on some of the Chinese brands, really remind you of, of Apple. What is interesting for me, is how there wouldn't be, with the exception of Samsung and Motorola, there's no one else here that is non-Chinese from a smartphone point of view. So that's in itself, is something that changed dramatically over the years, especially for somebody like me that still remember Nokia being the number one in the market. >> Huh. >> So. >> Guys, we could continue this conversation. We are unfortunately out of time. But thank you so much for joining Dave and me, talking about your perspectives on the event, the industry, the disruptive forces. It's going to be really interesting to see where it goes. 'Cause at the end of the day, it's the consumers that just want to make sure I can connect wherever I am 24 by seven, and it just needs to work. Thank you so much for your insights. >> Thank you. >> Lisa, it's been great. Dave, great. It's a pleasure. >> Our pleasure. For our guests, and for Dave Nicholson, I'm Lisa Martin. You're watching, "theCUBE," the leader in live and emerging tech coverage coming to you day three of our coverage of MWC 23. Stick around. Our next guest joins us momentarily. (outro music)

>> TheCUBE's live coverage is made possible by funding from Dell Technologies. Creating technologies that drive human progress. (upbeat music) >> Welcome back to Spain, everybody. We're here at the Theater Live and MWC 23. You're watching theCUBE's Continuous Coverage. This is day two. I'm Dave Vellante with my co-host, Dave Nicholson. Lisa Martin is also in the house. John Furrier out of our Palo Alto studio covering all the news. Check out silicon angle.com. Okay, we're going to dig into the core infrastructure here. We're going to talk a little bit about servers. Manya Rastogi is here. She's in technical marketing at Dell Technologies. And Abdel Bagegni is technical program manager at the Telecom Infra Project. Folks, welcome to theCUBE. Good to see you. >> Thank you. >> Abdel, what is the Telecom Infras Project? Explain to our audience. >> Yeah. So the Telecom Infra Project is a US based non-profit organization community that brings together different participants, suppliers, vendors, operators SI's together to accelerate the adoption of open RAN and open interface solutions across the globe. >> Okay. So that's the mission is open RAN adoption. And then how, when was it formed? Give us the background and some of the, some of the milestones so far. >> Yeah. So the telecom infra project was established five years ago from different vendor leaders and operators across the globe. And then the mission was to bring different players in to work together to accelerate the adoption of, of open RAN. Now open RAN has a lot of potential and opportunities, but in the same time there's challenges that we work together as a community to facilitate those challenges and overcome those barriers. >> And we've been covering all week just the disaggregation of the network. And you know, we've seen this movie sort of before playing out now in, in telecom. And Manya, this is obviously a compute intensive environment. We were at the Dell booth earlier this morning poking around, beautiful booth, lots of servers. Tell us what your angle is here in this marketplace. >> Yeah, so I would just like to say that Dell is kind of leading or accelerating the innovation at the telecom edge with all these ruggedized servers that we are offering. So just continuing the mission, like Abdel just mentioned for the open RAN, that's where a lot of focus will be from these servers will be, so XR 8000, it's it's going to be one of the star servers for telecom with, you know, offering various workloads. So it can be rerun, open run, multi access, edge compute. And it has all these different features with itself and the, if we, we can talk more about the performance gains, how it is based on the Intel CPUs and just try to solve the purpose like along with various vendors, the whole ecosystem solve this challenge for the open RAN. >> So Manya mentioned some of those infrastructure parts. Does and do, do you say TIP or T-I-P for short? >> Abdel: We say TIP. >> TIP. >> Abdel: T-I-P is fine as well. >> Does, does, does TIP or T-I-P have a certification process or a, or a set of guidelines that someone like Dell would either adhere to or follow to be sort of TIP certified? What does that look like? >> Yeah, of course. So what TIP does is TIP accredits what solutions that actually work in a real commercial grade environment. So what we do is we bring the different players together to come up with the most efficient optimized solution. And then it goes through a process that the community sets the, the, the criteria for and accepts. And then once this is accredited it goes into TIP exchange for other operators and the participants and the industry to adopt. So it's a well structured process and it's everything about how we orchestrate the industry to come together and set those requirements and and guidelines. Everything starts with a use case from the beginning. It's based on operators requirements, use cases and then those use cases will be translated into a solution that the industry will approve. >> So when you say operator, I can think of that sort of traditionally as the customer side of things versus the vendor side of things. Typically when organizations get together like TIP, the operator customer side is seeking a couple of things. They want perfect substitutes in all categories so that they could grind vendors down from a price perspective but they also want amazing innovation. How do you, how do you deliver both? >> Yeah, I mean that's an excellent question. We be pragmatic and we bring all players in one table to discuss. MNO's want this, vendors can provide a certain level and we bring them together and they discuss and come up with something that can be deployed today and future proof for the future. >> So I've been an enterprise technology observer for a long time and, you know, I saw the, the attempt to take network function virtualization which never really made much of an impact, but it was a it was the beginning of the enterprise players really getting into this market. And then I would see companies, whether it was Dell or HPE or Cisco, they'd take an X 86 server, put a cool name on it, edge something, and throw it over the fence and that didn't work so well. Now it's like, Manya. We're starting to get serious. You're building relationships. >> Manya: Totally. >> I mentioned we were at the Dell booth you're actually building purpose built systems now for this, this segment. Tell us what's different about this market and the products that you're developing for this market than say the commercial enterprise. >> So you are absolutely right, like, you know, kind of thinking about the journey, there has been a lot of, it has been going for a long time for all these improvements and towards going more open disaggregated and overall that kind of environment and what Dell brings together with our various partners and particularly if you talk about Intel. So these servers are powered by the players four gen intel beyond processors. And so what Intel is doing right now is providing us with great accelerators like vRAN Boost. So it increases performance like doubles what it was able to do before. And power efficiency, it has been an issue for a long, long time and it still continues but there is some improvement. For example 20% reduction overall with the power savings. So that's a step forward in that direction. And then we have done some of our like own testing as well with these servers and continuing that, you know it's not just telecom but also going towards Edge or inferencing like all these comes together not just X 30,000 but for example XR 56 10, 70, 76 20. So these are three servers which combines together to like form telecom and Edge and covers altogether. So that's what it is. >> Great, thank you. So Abdel, I mean I think generally people agree that in the fullness of time all radio access networks are going to be open, right? It's just a matter of okay, how do we get there? How do we make sure that it has the same, you know, quality of service characteristics. So where are we on on that, that journey from your perspective? And, and maybe you could project what, what it's going to look like over this decade. 'Cause it's going to take, you know, years. >> It's going to take a bit of time to mature and be a kind of a plug and play different units together. I think there was a lot, there was a, was a bit of over-promising in a few, in the last few years on the acceleration of open RAN deployment. That, well, a TIP is trying to do is trying to realize the pragmatic approach of the open run deployment. Now we know the innovation cannot happen when you have a kind of closed interfaces when you allow small players to be within the market and bring the value to, to the RAN areas. This is where the innovation happens. I think what would happen on the RAN side of things is that it would be driven by use cases and the operators. And the minute that the operators are no longer can depend on the closed interface vendors because there's use cases that fulfill that are requires some open RAN functionality, be the, the rig or the SMO layers and the different configurations of the rUSE getting the servers to the due side of things. This kind of modular scalability on this layer is when the RAN will, the Open RAN, would boost. This would happen probably, yeah. >> Go ahead. >> Yeah, it would happen in, in the next few years. Not next year or the year after but definitely something within the four to five years from now. >> I think it does feel like it's a second half of the decade and you feel like the, the the RAN intelligent controller is going to be a catalyst to actually sort of force the world into this open environment. >> Let's say that the Rick and the promises that were given to, to the sun 10 years ago, the Rick is realizing it and the closed RAN vendors are developing a lot on the Rick side more than the other parts of the, of the open RAN. So it will be a catalyst that would drive the innovation of open RAN, but only time will tell. >> And there are some naysayers, I mean I've seen some you know, very, very few, but I've seen some works that, oh the economics aren't there. It'll, it'll never get there. What, what do you, what do you say to that? That, that it won't ever, open RAN won't ever be as cost effective as you know, closed networks. >> Open RAN will open innovations that small players would have the opportunity to contribute to the, to the RAN space. This opportunity is not given to small players today. Open RAN provides this kind of opportunity and given that it's a path for innovation, then I would say that, you know, different perspectives some people are making sure that, you know the status quo is the way forward. But it would certainly put barriers on on innovation and this is not the way forward. >> Yeah. You can't protect the past in the future. My own personal opinion is, is that it doesn't have to be comparable from a, from a TCO perspective it can be close enough. It's the innovative, same thing with like you watch the, the, the adoption of Cloud. >> Exactly. >> Like cloud was more expensive it's always more expensive to rent, but people seem to be doing public Cloud, you know, because of the the innovation capabilities and the developer capabilities. Is that a fair analogy in this space, do you think? >> I mean this is what all technologies happens. >> Yeah. >> Right? It starts with a quite costly and then the the cost will start dropping down. I mean the, the cost of, of a megabyte two decades ago is probably higher than what it costly terabyte. So this is how technology evolves and it's any kind of comparison, either copper or even the old generation, the legacy generations could be a, a valid comparison. However, they need to be at a market demand for something like that. And I think the use cases today with what the industry is is looking for have that kind of opportunity to pull this kind of demand. But, but again, it needs to go work close by the what happens in the technology space, be it, you know we always talk about when we, we used to talk about 5G, there was a lot of hypes going on there. But I think once it realized in, in a pragmatic, in a in a real life situation, the minutes that governments decide to go for autonomous vehicles, then you would have limitations on the current closed RAN infrastructures and you would definitely need something to to top it up on the- >> I mean, 5G needs open RAN, I mean that's, you know not going to happen without it. >> Exactly. >> Yeah, yeah. But, but what is, but what would you say the most significant friction is between here and the open RAN nirvana? What are, what are the real hurdles that need to be overcome? There's obviously just the, I don't want to change we've been doing this the same way forever, but what what are the, what are the real, the legitimate concerns that people have when we start talking about open RAN? >> So I think from a technology perspective it will be solved. All of the tech, I mean there's smart engineers in the world today that will fix, you know these kind of problems and all of the interability, interruptability issues and, and all of that. I think it's about the mindset, the, the interfaces between the legacy core and RAN has been became more fluid today. We don't have that kind of a hard line between these kind of different aspects. We have the, the MEC coming closer to the RAN, we have the RAN coming closer to the Core, and we have the service based architectures in the Core. So these kind of things make it needs a paradigm shift between how operators that would need to tackle the open RAN space. >> Are there specific deployment requirements for open RAN that you can speak to from your perspective? >> For sure and going in this direction, like, you know evolution with the technology and how different players are coming together. Like that's something I wanted to comment from the previous question. And that's where like, you know these servers that Dell is offering right now. Specific functionality requirements, for example, it's it's a small server, it's short depth just 430 millimeters of depth and it can fit anywhere. So things like small form factor, it's it's crucial because if you, it can replace like multiple servers 10 years ago with just one server and you can place it like near a base band unit or to a cell site on top of a roof wherever. Like, you know, if it's a small company and you need this kind of 5G connection it kind of solves that challenge with this server. And then there are various things like, you know increasing thermals for example temperatures. It is classified like, you know kind of compliant with the negative 5 to 55 degree Celsius. And then we are also moving towards, for example negative 20 to 65 degree Celsius. Which is, which is kind of great because in situations where, which are out of our hands and you need specific thermals for those situations that's where it can solve that problem. >> Are those, are those statistics in those measurements different than the old NEB's standards, network equipment building standards? Or are they, are they in line with that? >> It is, it is a next step. Like so most of our servers that we have right now are negative five to five degree Celsius, for especially the extremely rugged server series and this one XR 8,000 which is focused for the, it's telecom inspired so it's focused on those customers. So we are trying to come up like go a step ahead and also like offering this additional temperatures testing and yeah compliance. So, so it is. >> Awesome. So we, I said we were at the booth early today. Looks like some good traffic people poking around at different, you know, innovations you got going. Some of the private network stuff is kind of cool. I'm like how much does that cost? I think I might like one of those, you know, but- >> [Private 5G home network. >> Right? Why not? Guys, great to have you on the show. Thanks so much for sharing. Appreciate it. >> Thank you. >> Thank you so much. >> Okay. For Dave Nicholson and Lisa Martin this is Dave Vellante, theCUBE's coverage. MWC 23 live from the Fida in Barcelona. We'll be right back. (outro music)

(bright upbeat music) >> Hello, everyone. Welcome to SiliconANGLE News breaking story here. Amazon Web Services expanding their relationship with Hugging Face, breaking news here on SiliconANGLE. I'm John Furrier, SiliconANGLE reporter, founder, and also co-host of theCUBE. And I have with me, Swami, from Amazon Web Services, vice president of database, analytics, machine learning with AWS. Swami, great to have you on for this breaking news segment on AWS's big news. Thanks for coming on and taking the time. >> Hey, John, pleasure to be here. >> You know- >> Looking forward to it. >> We've had many conversations on theCUBE over the years, we've watched Amazon really move fast into the large data modeling, SageMaker became a very smashing success, obviously you've been on this for a while. Now with ChatGPT OpenAI, a lot of buzz going mainstream, takes it from behind the curtain inside the ropes, if you will, in the industry to a mainstream. And so this is a big moment, I think, in the industry, I want to get your perspective, because your news with Hugging Face, I think is another tell sign that we're about to tip over into a new accelerated growth around making AI now application aware, application centric, more programmable, more API access. What's the big news about, with AWS Hugging Face, you know, what's going on with this announcement? >> Yeah. First of all, they're very excited to announce our expanded collaboration with Hugging Face, because with this partnership, our goal, as you all know, I mean, Hugging Face, I consider them like the GitHub for machine learning. And with this partnership, Hugging Face and AWS, we'll be able to democratize AI for a broad range of developers, not just specific deep AI startups. And now with this, we can accelerate the training, fine tuning and deployment of these large language models, and vision models from Hugging Face in the cloud. And the broader context, when you step back and see what customer problem we are trying to solve with this announcement, essentially if you see these foundational models, are used to now create like a huge number of applications, suggest like tech summarization, question answering, or search image generation, creative, other things. And these are all stuff we are seeing in the likes of these ChatGPT style applications. But there is a broad range of enterprise use cases that we don't even talk about. And it's because these kind of transformative, generative AI capabilities and models are not available to, I mean, millions of developers. And because either training these elements from scratch can be very expensive or time consuming and need deep expertise, or more importantly, they don't need these generic models, they need them to be fine tuned for the specific use cases. And one of the biggest complaints we hear is that these models, when they try to use it for real production use cases, they are incredibly expensive to train and incredibly expensive to run inference on, to use it at a production scale. So, and unlike web search style applications, where the margins can be really huge, here in production use cases and enterprises, you want efficiency at scale. That's where Hugging Face and AWS share our mission. And by integrating with Trainium and Inferentia, we're able to handle the cost efficient training and inference at scale, I'll deep dive on it. And by teaming up on the SageMaker front, now the time it takes to build these models and fine tune them is also coming down. So that's what makes this partnership very unique as well. So I'm very excited. >> I want to get into the time savings and the cost savings as well on the training and inference, it's a huge issue, but before we get into that, just how long have you guys been working with Hugging Face? I know there's a previous relationship, this is an expansion of that relationship, can you comment on what's different about what's happened before and then now? >> Yeah. So, Hugging Face, we have had a great relationship in the past few years as well, where they have actually made their models available to run on AWS, you know, fashion. Even in fact, their Bloom Project was something many of our customers even used. Bloom Project, for context, is their open source project which builds a GPT-3 style model. And now with this expanded collaboration, now Hugging Face selected AWS for that next generation office generative AI model, building on their highly successful Bloom Project as well. And the nice thing is, now, by direct integration with Trainium and Inferentia, where you get cost savings in a really significant way, now, for instance, Trn1 can provide up to 50% cost to train savings, and Inferentia can deliver up to 60% better costs, and four x more higher throughput than (indistinct). Now, these models, especially as they train that next generation generative AI models, it is going to be, not only more accessible to all the developers, who use it in open, so it'll be a lot cheaper as well. And that's what makes this moment really exciting, because we can't democratize AI unless we make it broadly accessible and cost efficient and easy to program and use as well. >> Yeah. >> So very exciting. >> I'll get into the SageMaker and CodeWhisperer angle in a second, but you hit on some good points there. One, accessibility, which is, I call the democratization, which is getting this in the hands of developers, and/or AI to develop, we'll get into that in a second. So, access to coding and Git reasoning is a whole nother wave. But the three things I know you've been working on, I want to put in the buckets here and comment, one, I know you've, over the years, been working on saving time to train, that's a big point, you mentioned some of those stats, also cost, 'cause now cost is an equation on, you know, bundling whether you're uncoupling with hardware and software, that's a big issue. Where do I find the GPUs? Where's the horsepower cost? And then also sustainability. You've mentioned that in the past, is there a sustainability angle here? Can you talk about those three things, time, cost, and sustainability? >> Certainly. So if you look at it from the AWS perspective, we have been supporting customers doing machine learning for the past years. Just for broader context, Amazon has been doing ML the past two decades right from the early days of ML powered recommendation to actually also supporting all kinds of generative AI applications. If you look at even generative AI application within Amazon, Amazon search, when you go search for a product and so forth, we have a team called MFi within Amazon search that helps bring these large language models into creating highly accurate search results. And these are created with models, really large models with tens of billions of parameters, scales to thousands of training jobs every month and trained on large model of hardware. And this is an example of a really good large language foundation model application running at production scale, and also, of course, Alexa, which uses a large generator model as well. And they actually even had a research paper that showed that they are more, and do better in accuracy than other systems like GPT-3 and whatnot. So, and we also touched on things like CodeWhisperer, which uses generative AI to improve developer productivity, but in a responsible manner, because 40% of some of the studies show 40% of this generated code had serious security flaws in it. This is where we didn't just do generative AI, we combined with automated reasoning capabilities, which is a very, very useful technique to identify these issues and couple them so that it produces highly secure code as well. Now, all these learnings taught us few things, and which is what you put in these three buckets. And yeah, like more than 100,000 customers using ML and AI services, including leading startups in the generative AI space, like stability AI, AI21 Labs, or Hugging Face, or even Alexa, for that matter. They care about, I put them in three dimension, one is around cost, which we touched on with Trainium and Inferentia, where we actually, the Trainium, you provide to 50% better cost savings, but the other aspect is, Trainium is a lot more power efficient as well compared to traditional one. And Inferentia is also better in terms of throughput, when it comes to what it is capable of. Like it is able to deliver up to three x higher compute performance and four x higher throughput, compared to it's previous generation, and it is extremely cost efficient and power efficient as well. >> Well. >> Now, the second element that really is important is in a day, developers deeply value the time it takes to build these models, and they don't want to build models from scratch. And this is where SageMaker, which is, even going to Kaggle uses, this is what it is, number one, enterprise ML platform. What it did to traditional machine learning, where tens of thousands of customers use StageMaker today, including the ones I mentioned, is that what used to take like months to build these models have dropped down to now a matter of days, if not less. Now, a generative AI, the cost of building these models, if you look at the landscape, the model parameter size had jumped by more than thousand X in the past three years, thousand x. And that means the training is like a really big distributed systems problem. How do you actually scale these model training? How do you actually ensure that you utilize these efficiently? Because these machines are very expensive, let alone they consume a lot of power. So, this is where SageMaker capability to build, automatically train, tune, and deploy models really concern this, especially with this distributor training infrastructure, and those are some of the reasons why some of the leading generative AI startups are actually leveraging it, because they do not want a giant infrastructure team, which is constantly tuning and fine tuning, and keeping these clusters alive. >> It sounds like a lot like what startups are doing with the cloud early days, no data center, you move to the cloud. So, this is the trend we're seeing, right? You guys are making it easier for developers with Hugging Face, I get that. I love that GitHub for machine learning, large language models are complex and expensive to build, but not anymore, you got Trainium and Inferentia, developers can get faster time to value, but then you got the transformers data sets, token libraries, all that optimized for generator. This is a perfect storm for startups. Jon Turow, a former AWS person, who used to work, I think for you, is now a VC at Madrona Venture, he and I were talking about the generator AI landscape, it's exploding with startups. Every alpha entrepreneur out there is seeing this as the next frontier, that's the 20 mile stairs, next 10 years is going to be huge. What is the big thing that's happened? 'Cause some people were saying, the founder of Yquem said, "Oh, the start ups won't be real, because they don't all have AI experience." John Markoff, former New York Times writer told me that, AI, there's so much work done, this is going to explode, accelerate really fast, because it's almost like it's been waiting for this moment. What's your reaction? >> I actually think there is going to be an explosion of startups, not because they need to be AI startups, but now finally AI is really accessible or going to be accessible, so that they can create remarkable applications, either for enterprises or for disrupting actually how customer service is being done or how creative tools are being built. And I mean, this is going to change in many ways. When we think about generative AI, we always like to think of how it generates like school homework or arts or music or whatnot, but when you look at it on the practical side, generative AI is being actually used across various industries. I'll give an example of like Autodesk. Autodesk is a customer who runs an AWS and SageMaker. They already have an offering that enables generated design, where designers can generate many structural designs for products, whereby you give a specific set of constraints and they actually can generate a structure accordingly. And we see similar kind of trend across various industries, where it can be around creative media editing or various others. I have the strong sense that literally, in the next few years, just like now, conventional machine learning is embedded in every application, every mobile app that we see, it is pervasive, and we don't even think twice about it, same way, like almost all apps are built on cloud. Generative AI is going to be part of every startup, and they are going to create remarkable experiences without needing actually, these deep generative AI scientists. But you won't get that until you actually make these models accessible. And I also don't think one model is going to rule the world, then you want these developers to have access to broad range of models. Just like, go back to the early days of deep learning. Everybody thought it is going to be one framework that will rule the world, and it has been changing, from Caffe to TensorFlow to PyTorch to various other things. And I have a suspicion, we had to enable developers where they are, so. >> You know, Dave Vellante and I have been riffing on this concept called super cloud, and a lot of people have co-opted to be multicloud, but we really were getting at this whole next layer on top of say, AWS. You guys are the most comprehensive cloud, you guys are a super cloud, and even Adam and I are talking about ISVs evolving to ecosystem partners. I mean, your top customers have ecosystems building on top of it. This feels like a whole nother AWS. How are you guys leveraging the history of AWS, which by the way, had the same trajectory, startups came in, they didn't want to provision a data center, the heavy lifting, all the things that have made Amazon successful culturally. And day one thinking is, provide the heavy lifting, undifferentiated heavy lifting, and make it faster for developers to program code. AI's got the same thing. How are you guys taking this to the next level, because now, this is an opportunity for the competition to change the game and take it over? This is, I'm sure, a conversation, you guys have a lot of things going on in AWS that makes you unique. What's the internal and external positioning around how you take it to the next level? >> I mean, so I agree with you that generative AI has a very, very strong potential in terms of what it can enable in terms of next generation application. But this is where Amazon's experience and expertise in putting these foundation models to work internally really has helped us quite a bit. If you look at it, like amazon.com search is like a very, very important application in terms of what is the customer impact on number of customers who use that application openly, and the amount of dollar impact it does for an organization. And we have been doing it silently for a while now. And the same thing is true for like Alexa too, which actually not only uses it for natural language understanding other city, even national leverages is set for creating stories and various other examples. And now, our approach to it from AWS is we actually look at it as in terms of the same three tiers like we did in machine learning, because when you look at generative AI, we genuinely see three sets of customers. One is, like really deep technical expert practitioner startups. These are the startups that are creating the next generation models like the likes of stability AIs or Hugging Face with Bloom or AI21. And they generally want to build their own models, and they want the best price performance of their infrastructure for training and inference. That's where our investments in silicon and hardware and networking innovations, where Trainium and Inferentia really plays a big role. And we can nearly do that, and that is one. The second middle tier is where I do think developers don't want to spend time building their own models, let alone, they actually want the model to be useful to that data. They don't need their models to create like high school homeworks or various other things. What they generally want is, hey, I had this data from my enterprises that I want to fine tune and make it really work only for this, and make it work remarkable, can be for tech summarization, to generate a report, or it can be for better Q&A, and so forth. This is where we are. Our investments in the middle tier with SageMaker, and our partnership with Hugging Face and AI21 and co here are all going to very meaningful. And you'll see us investing, I mean, you already talked about CodeWhisperer, which is an open preview, but we are also partnering with a whole lot of top ISVs, and you'll see more on this front to enable the next wave of generated AI apps too, because this is an area where we do think lot of innovation is yet to be done. It's like day one for us in this space, and we want to enable that huge ecosystem to flourish. >> You know, one of the things Dave Vellante and I were talking about in our first podcast we just did on Friday, we're going to do weekly, is we highlighted the AI ChatGPT example as a horizontal use case, because everyone loves it, people are using it in all their different verticals, and horizontal scalable cloud plays perfectly into it. So I have to ask you, as you look at what AWS is going to bring to the table, a lot's changed over the past 13 years with AWS, a lot more services are available, how should someone rebuild or re-platform and refactor their application of business with AI, with AWS? What are some of the tools that you see and recommend? Is it Serverless, is it SageMaker, CodeWhisperer? What do you think's going to shine brightly within the AWS stack, if you will, or service list, that's going to be part of this? As you mentioned, CodeWhisperer and SageMaker, what else should people be looking at as they start tinkering and getting all these benefits, and scale up their ups? >> You know, if we were a startup, first, I would really work backwards from the customer problem I try to solve, and pick and choose, bar, I don't need to deal with the undifferentiated heavy lifting, so. And that's where the answer is going to change. If you look at it then, the answer is not going to be like a one size fits all, so you need a very strong, I mean, granted on the compute front, if you can actually completely accurate it, so unless, I will always recommend it, instead of running compute for running your ups, because it takes care of all the undifferentiated heavy lifting, but on the data, and that's where we provide a whole variety of databases, right from like relational data, or non-relational, or dynamo, and so forth. And of course, we also have a deep analytical stack, where data directly flows from our relational databases into data lakes and data virus. And you can get value along with partnership with various analytical providers. The area where I do think fundamentally things are changing on what people can do is like, with CodeWhisperer, I was literally trying to actually program a code on sending a message through Twilio, and I was going to pull up to read a documentation, and in my ID, I was actually saying like, let's try sending a message to Twilio, or let's actually update a Route 53 error code. All I had to do was type in just a comment, and it actually started generating the sub-routine. And it is going to be a huge time saver, if I were a developer. And the goal is for us not to actually do it just for AWS developers, and not to just generate the code, but make sure the code is actually highly secure and follows the best practices. So, it's not always about machine learning, it's augmenting with automated reasoning as well. And generative AI is going to be changing, and not just in how people write code, but also how it actually gets built and used as well. You'll see a lot more stuff coming on this front. >> Swami, thank you for your time. I know you're super busy. Thank you for sharing on the news and giving commentary. Again, I think this is a AWS moment and industry moment, heavy lifting, accelerated value, agility. AIOps is going to be probably redefined here. Thanks for sharing your commentary. And we'll see you next time, I'm looking forward to doing more follow up on this. It's going to be a big wave. Thanks. >> Okay. Thanks again, John, always a pleasure. >> Okay. This is SiliconANGLE's breaking news commentary. I'm John Furrier with SiliconANGLE News, as well as host of theCUBE. Swami, who's a leader in AWS, has been on theCUBE multiple times. We've been tracking the growth of how Amazon's journey has just been exploding past five years, in particular, past three. You heard the numbers, great performance, great reviews. This is a watershed moment, I think, for the industry, and it's going to be a lot of fun for the next 10 years. Thanks for watching. (bright music)

(lively music) >> Hello, everyone. Welcome back to the Supercloud presentation here. I'm theCUBE, I'm John Furrier, your host. What a great segment here. We're going to unpack the networking aspect of the cloud, how that translates into what Supercloud architecture and platform deployment scenarios look like. And demystify multi-cloud, hybridcloud. We've got two great experts. Amir Khan, the Co-Founder and CEO of Alkira, Atif Khan, Co-Founder and CTO of Alkira. These guys been around since 2018 with the startup, but before that story, history in the tech industry. I mean, routing early days, multiple waves, multiple cycles. >> Welcome three decades. >> Welcome to Supercloud. >> Thanks. >> Thanks for coming on. >> Thank you so much for having us. >> So, let's get your take on Supercloud because it's been one of those conversations that really galvanized the industry because it kind of highlights almost this next wave, this next side of the street that everyone's going to be on that's going to be successful. The laggards on the legacy seem to be stuck on the old model. SaaS is growing up, it's ISVs, it's ecosystems, hyperscale, full hybrid. And then multi-cloud around the corners cause all this confusion, everyone's hand waving. You know, this is a solution, that solution, where are we? What do you guys see as this supercloud dynamic? >> So where we start from is always focusing on the customer problem. And in 2018 when we identified the problem, we saw that there were multiple clouds with many diverse ways of doing things from the network perspective, and customers were struggling with that. So we delved deeper into that and looked at each one of the cloud architectures completely independent. And there was no common solution and customers were struggling with that from the perspective. They wanted to be in multiple clouds, either through mergers and acquisitions or running an application which may be more cost effective to run in something or maybe optimized for certain reasons to run in a different cloud. But from the networking perspective, everything needed to come together. So that's, we are starting to define it as a supercloud now, but basically, it's a common infrastructure across all clouds. And then integration of high lift services like, you know, security or IPAM services or many other types of services like inter-partner routing and stuff like that. So, Amir, you agree then that multi-cloud is simply a default result of having whatever outcomes, either M&A, some productivity software, maybe Azure. >> Yes. >> Amazon has this and then I've got on-premise application, so it's kinds mishmash. >> So, I would qualify it with hybrid multi-cloud because everything is going to be interconnected. >> John: Got it. >> Whether it's on-premise, remote users or clouds. >> But have CTO perspective, obviously, you got developers, multiple stacks, got AWS, Azure and GCP, other. Not everyone wants to kind of like go all in, but yet they don't want to hedge too much because it's a resource issue. And I got to learn this stack, I got to learn that stack. So then now, you have this default multi-cloud, hybrid multi-cloud, then it's like, okay, what do I do? How do you spread that around? Is it dangerous? What's the the approach technically? What's some of the challenges there? >> Yeah, certainly. John, first, thanks for having us here. So, before I get to that, I'll just add a little bit to what Amir was saying, like how we started, what we were seeing and how it, you know, correlates with the supercloud. So, as you know, before this company, Alkira, we were doing, we did the SD-WAN company, which was Viptela. So there, we started seeing when people started deploying SD-WAN at like a larger scale. We started like, you know, customers coming to us and saying they needed connectivity into the cloud from the SD-WAN. They wanted to extend the SD-WAN fabric to the cloud. So we came up with an architecture, which was like later we started calling them Cloud onRamps, where we built, you know, a transit VPC and put like the virtual instances of SD-WAN appliances extended from there to the cloud. But before we knew, like it started becoming very complicated for the customers because it wasn't just connectivity, it also required, you know, other use cases. You had to instantiate or bring in security appliances in there. You had to secure all of that stuff. There were requirements for, you know, different regions. So you had to bring up the same thing in different regions. Then multiple clouds, what did you do? You had to replicate the same thing in multiple clouds. And now if there was was requirement between clouds, how were you going to do it? You had to route traffic from somewhere, and come up with all those routing controls and stuff. So, it was very complicated. >> Like spaghetti code, but on network. >> The games begin, in fact, one of our customers called it spaghetti mess. And so, that's where like we thought about where was the industry going and which direction the industry was going into? And we came up with the Alkira where what we are doing is building a common infrastructure across multiple clouds, across in, you know, on-prem locations, be it data centers or physical sites, branches sites, et cetera, with integrated security and network networking services inside. And, you know, nowadays, networking is not only about connectivity, you have to secure everything. So, security has to be built in. Redundancy, high availability, disaster recovery. So all of that needs to be built in. So that's like, you know, kind of a definition of like what we thought at that time, what is turning into supercloud now. >> Yeah. It's interesting too, you mentioned, you know, VPCs is not, configuration of loans a hassle. Nevermind the manual mistakes could be made, but as you decide to do something you got to, "Oh, we got to get these other things." A lot of the hyper scales and a lot of the alpha cloud players now, and cloud native folks, they're kind of in that mode of, "Wow, look at what we've built." Now, they're got to maintain, how do I refresh it? Like, how do I keep the talent? So they got this similar chaotic environment where it's like, okay, now they're already already through, so I think they're going to be okay. But then some people want to bypass it completely. So there's a lot of customers that we see out there that fit the makeup of, I'm cloud first, I've lifted and shifted, I move some stuff to the cloud. But I want to bypass all that learnings from all the people that are gone through the past three years. Can I just skip that and go to a multi-cloud or coherent infrastructure? What do you think about that? What's your view? >> So yeah, so if you look at these enterprises, you know, many of them just to find like the talent, which for one cloud as far as the IT staff is concerned, it's hard enough. And now, when you have multiple clouds, it's hard to find people the talent which is, you know, which has expertise across different clouds. So that's where we come into the picture. So our vision was always to simplify all of this stuff. And simplification, it cannot be just simplification because you cannot just automate the workflows of the cloud providers underneath. So you have to, you know, provide your full data plane on top of it, fed full control plane, management plane, policy and management on top of it. And coming back to like your question, so these nowadays, those people who are working on networking, you know, before it used to be like CLI. You used to learn about Cisco CLI or Juniper CLI, and you used to work on it. Nowadays, it's very different. So automation, programmability, all of that stuff is the key. So now, you know, Ops guys, the DevOps guys, so these are the people who are in high demand. >> So what do you think about the folks out there that are saying, okay, you got a lot of fragmentation. I got the stacks, I got a lot of stove pipes, if you will, out there on the stack. I got to learn this from Azure. Can you guys have with your product abstract the way that's so developers don't need to know the ins and outs of stack's, almost like a gateway, if you will, the old days. But like I'm a developer or team develop, why should I have to learn the management layer of Azure? >> That's exactly what we started, you know, out with to solve. So it's, what we have built is a platform and the platform sits inside the cloud. And customers are able to build their own network or a virtual network on top using that platform. So the platform has its own data plane, own control plane and management plane with a policy layer on top of it. So now, it's the platform which is sitting in different clouds, but from a customer's point of view, it's one way of doing networking. One way of instantiating or bringing in services or security services in the middle. Whether those are our security services or whether those are like services from our partners, like Palo Alto or Checkpoint or Cisco. >> So you guys brought the SD-WAN mojo and refactored it for the cloud it sounds like. >> No. >> No? (chuckles) >> We cannot said. >> All right, explain. >> It's way more than that. >> I mean, SD-WAN was wan. I mean, you're talking about wide area networks, talking about connected, so explain the difference. >> SD-WAN was primarily done for one major reason. MPLS was expensive, very strong SLAs, but very low speed. Internet, on the other hand, you sat at home and you could access your applications much faster. No SLA, very low cost, right? So we wanted to marry the two together so you could have a purely private infrastructure and a public infrastructure and secure both of them by creating a common secure fabric across all those environments. And then seamlessly tying it into your internal branch and data center and cloud network. So, it merely brought you to the edge of the cloud. It didn't do anything inside the cloud. Now, the major problem resides inside the clouds where you have to optimize the clouds themselves. Take a step back. How were the clouds built? Basically, the cloud providers went to the Ciscos and Junipers and the rest of the world, built the network in the data centers or across wide area infrastructure, and brought it all together and tried to create a virtualized layer on top of that. But there were many limitations of this underlying infrastructure that they had built. So number of routes per region, how inter region connectivity worked, or how many routes you could carry to the VPCs of V nets? That all those were becoming no common policy across, you know, these environments, no segmentation across these environments, right? So the networking constructs that the enterprise customers were used to as enterprise class carry class capabilities, they did not exist in the cloud. So what did the customer do? They ended up stitching it together all manually. And that's why Atif was alluding to earlier that it became a spaghetti mess for the customers. And then what happens is, as a result, day two operations, you know, troubleshooting, everything becomes a nightmare. So what do you do? You have to build an infrastructure inside the cloud. Cloud has enough raw capabilities to build the solutions inside there. Netflix's of the world. And many different companies have been born in the cloud and evolved from there. So why could we not take the raw capabilities of the clouds and build a network cloud or a supercloud on top of these clouds to optimize the whole infrastructure and seamlessly connecting it into the on-premise and remote user locations, right? So that's your, you know, hybrid multi-cloud solution. >> Well, great call out on the SD-WAN in common versus cloud. 'Cause I think this is important because you're building a network layer in the cloud that spans out so the customers don't have to get into the, there's a gap in the system that I'm used to, my operating environment, of having lockdown security and network. >> So yeah. So what you do is you use the raw capabilities like bandwidth or virtual machines, or you know, containers, or, you know, different types of serverless capabilities. And you bring it all together in a way to solve the networking problems, thereby creating a supercloud, which is an abstraction layer which hides all the complexity of the underlying clouds from the customer, right? And it provides a common infrastructure across all environments to that customer, right? That's the beauty of it. And it does it in a way that it looks like, if they have the networking knowledge, they can apply it to this new environment and carry it forward. One way of doing security across all clouds and hybrid environments. One way of doing routing. One way of doing large-scale network address translation. One way of doing IPAM services. So people are tired of doing individual things and individual clouds and on-premise locations, right? So now they're getting something common. >> You guys brought that, you brought all that to bear and flexible for the customer to essentially self-serve their network cloud. >> Yes, yeah. Is that the wave? >> And nowadays, from business perspective, agility is the key, right? You have to move at the pace of the business. If you don't, you are losing. >> So, would it be safe to say that you guys have a network supercloud? >> Absolutely, yeah. >> We, pretty much, yeah. Absolutely. >> What does that mean to our customer? What's in it for them? What's the benefit to the customer? I got a network supercloud, it connects, provides SLA, all the capabilities I need. What do they get? What's the end point for them? What's the end? >> Atif, maybe you can talk some examples. >> The IT infrastructure is all like distributed now, right? So you have applications running in data centers. You have applications running in one cloud. Other cloud, public clouds, enterprises are depending on so many SaaS applications. So now, these are, you can call these endpoints. So a supercloud or a network cloud, from our perspective, it's a cloud in the middle or a network in the middle, which provides connectivity from any endpoint to any endpoint. So, you are able to connect to the supercloud or network cloud in one way no matter where you are. So now, whichever cloud you are in, whichever cloud you need to connect to. And also, it's not just connecting to the cloud. So you need to do a lot of stuff, a lot of networking inside the cloud also. So now, as Amir was saying, every cloud has its own from a networking, you know, the concept perspective or the construct, they are different. There are limitations in there also. So this supercloud, which is sitting on top, basically, your platform is sitting into the cloud, but the supercloud is built on top of using your platform. So that abstracts all those complexities, all those limitations. So now your limitations are whatever the limitations of that platform are. So now your platform, that platform is in our control. So we can keep building it, we can keep scaling it horizontally. Because one of the things is that, you know, in this cloud era, one of the things is autoscaling these services. So why can't the network now autoscale also, just like your other services. >> Network autoscaling is a genius idea, and I think that's a killer. I want to ask the the follow on question because I think, first of all, I love what you guys are doing. So, I think it's a great example of this new innovation. It's not obvious until you see it, right? Geographical is huge. So, you know, single instance, global instances, multiple instances, you're seeing global. How do you guys look at that global equation? Because as companies expand their clouds into geos, and then ultimately, you know, it's obviously continent, region and locales. You're going to have geographic issues. So, this is an extension of your network cloud? >> Amir: It is the extension of the network cloud because if you look at this hyperscalers, they're sitting pretty much everywhere in the globe. So, wherever their regions are, the beauty of building a supercloud is that you can by definition, be available in those regions. It literally takes a day or two of testing for our stack to run in those regions, to make sure there are no nuances that we run into, you know, for that region. The moment we bring it up in that region, all customers can onboard into that solution. So literally, what used to take months or years to build a global infrastructure, now, you can configure it in 10 minutes basically, and bring it up in less than one hour. Since when did we see any solution- >> And by the way, >> that can come up with. >> when the edge comes out too, you're going to start to see more clouds get bolted on. >> Exactly. And you can expand to the edge of the network. That's why we call cloud the new edge, right? >> John: Yeah, it is. Now, I think you guys got a good solutions, network clouds, superclouds, good. So the question on the premise side, so I get the cloud play. It's very cool. You can expand out. It's a nice layer. I'm sure you manage the SLAs between latency and all kinds of things. Knowing when not to do things. Physics or physics. Okay. Now, you've got the on-premise. What's the on-premise equation look like? >> So on-premise, the kind of customers, we are working with large enterprises, mid-size enterprises. So they have on-prem networks, they have deployed, in many cases, they have deployed SD-WAN. In many cases, they have MPLS. They have data centers also. And a lot of these companies are, you know, moving the applications from the data center into the cloud. But we still have large enterprise- >> But for you guys, you can sit there too with non server or is it a box or what is it? >> It's a software stack, right? So, we are a software company. >> Okay, so no box. >> No box. >> Okay, got it. >> No box. >> It's even better. So, we can connect any, as I mentioned, any endpoint, whether it's data centers. So, what happens is usually these enterprises from the data centers- >> John: It's a cloud endpoint for you. >> Cloud endpoint for us. And they need highspeed connectivity into the cloud. And our network cloud is sitting inside the or supercloud is sitting inside the cloud. So we need highspeed connectivity from the data centers. This is like multi-gig type of connectivity. So we enable that connectivity as a service. And as Amir was saying, you are able to bring it up in minutes, pretty much. >> John: Well, you guys have a great handle on supercloud. I really appreciate you guys coming on. I have to ask you guys, since you have so much experience in the industry, multiple inflection points you've guys lived through and we're all old, and we can remember those glory days. What's the big deal going on right now? Because you can connect the dots and you can imagine, okay, like a Lambda function spinning up some connectivity. I need instant access to a new route, throw some, I need to send compute to an edge point for process data. A lot of these kind of ad hoc services are going to start flying around, which used to be manually configured as you guys remember. >> Amir: And that's been the problem, right? The shadow IT, that was the biggest problem in the enterprise environment. So that's what we are trying to get the customers away from. Cloud teams came in, individuals or small groups of people spun up instances in the cloud. It was completely disconnected from the on-premise environment or the existing IT environment that the customer had. So, how do you bring it together? And that's what we are trying to solve for, right? At a large scale, in a carrier cloud center (indistinct). >> What do you call that? Shift right or shift left? Shift left is in the cloud native world security. >> Amir: Yes. >> Networking and security, the two hottest areas. What are you shifting? Up or down? I mean, the network's moving up the stack. I mean, you're seeing the run times at Kubernetes later' >> Amir: Right, right. It's true we're end-to-end virtualization. So you have plumbing, which is the physical infrastructure. Then on top of that, now for the first time, you have true end-to-end virtualization, which the cloud-like constructs are providing to us. We tried to virtualize the routers, we try to virtualize instances at the server level. Now, we are bringing it all together in a truly end-to-end virtualized manner to connect any endpoint anywhere across the globe. Whether it's on-premise, home, multiple clouds, or SaaS type environments. >> Yeah. If you talk about the technical benefits beyond virtualizations, you kind of see in virtualization be abstracted away. So you got end-to-end virtualization, but you don't need to know virtualization to take advantage of it. >> Exactly. Exactly. >> What are some of the tech involved where, what's the trend around on top of virtual? What's the easy button for that? >> So there are many, many use cases from the customers and they're, you know, some of those use cases, they used to deliver out of their data centers before. So now, because you, know, it takes a long time to spend something up in the data center and stuff. So the trend is and what enterprises are looking for is agility. And to achieve that agility, they are moving those services or those use cases into the cloud. So another technical benefit of like something like a supercloud and what we are doing is we allow customers to, you know, move their services from existing data centers into the cloud as well. And I'll give you some examples. You know, these enterprises have, you know, tons of partners. They provide connectivity to their partners, to select resources. It used to happen inside the data center. You would bring in connectivity into the data center and apply like tons of ACLs and whatnot to make sure that you are able to only connect. And now those use cases are, they need to be enabled inside the cloud. And the customer's customers are also, it's not just coming from the on-prem, they're coming from the cloud as well. So, if they're coming from the cloud as well as from on-prem, so you need like an infrastructure like supercloud, which is sitting inside the cloud and is able to handle all these use cases. So all of these use cases have to be, so that requires like moving those services from the data center into the cloud or into the supercloud. So, they're, oh, as we started building this service over the last four years, we have come across so many use cases. And to deliver those use cases, you have to have a platform. So you have to have your own platform because otherwise you are depending on somebody else's, you know, capabilities. And every time their capabilities change, you have to change. >> John: I'm glad you brought up the platform 'cause I want to get your both reaction to this. So Bob Muglia just said on theCUBE here at Supercloud, that supercloud is a platform that provides programmatically consistent services hosted on heterogeneous cloud providers. So the question is, is supercloud a platform or an architecture in your view? >> That's an interesting view on things, you know? I mean, if you think of it, you have to design or architect a solution before we turn it into a platform. >> John: It's a trick question actually. >> So it's a, you know, so we look at it as that you have to have an architectural approach end to end, right? And then you build a solution based on that approach. So, I don't think that they are mutually exclusive. I think they go hand in hand. It's an architecture that you turn into a solution and provide that agility and high availability and disaster recovery capability that it built into that. >> It's interesting that these definitions might be actually redefined with this new configuration. >> Amir: Yes. >> Because architecture and platform used to mean something, like, aight here's a platform, you buy this platform. >> And then you architecture solution. >> Architect it via vendor. >> Right, right, right. >> Okay. And they have to deal with that architecture in the place of multiple superclouds. If you have too many stove pipes, then what's the purpose of supercloud? >> Right, right, right. And because, you know, historically, you built a router and you sold it to the customer. And the poor customer was supposed to install it all, you know, and interconnect all those things. And if you have 40, 50,000 router network, which we saw in our lifetime, 'cause there used to be many more branches when we were growing up in the networking industry, right? You had to create hierarchy and all kinds of things to figure out how to solve that problem. We are no longer living in that world anymore. You cannot deploy individual virtual instances. And that's what approach a lot of people are taking, which is a pure overly network. You cannot take that approach anymore. You have to evolve the architecture and then build the solution based on that architecture so that it becomes a platform which is readily available, highly scalable, and available. And at the same time, it's very, very easy to deploy. It's a SaaS type solution, right? >> So you're saying, do the architecture to get the solution for the platform that the customer has. >> Amir: Yes. >> They're not buying a platform, they end up with a platform- >> With the platform. >> as a result of Supercloud path. All right. So that's what's, so you mentioned, that's a great point. I want to double click on what you just said. 'Cause I like that what you said. What's the deployment strategy in your mind for supercloud? I'm an architect. I'm at an enterprise in the Midwest. I'm an insurance company, got some cloud action going on. I'm mostly on-premise. I've got the mandate to transform the company. We have apps. We'll be fully transformed in five years. What's my strategy? What do I do? >> Amir: The resources. >> What's the deployment strategy? Single global instance, code in every region, on every cloud? >> It needs to be a solution which is available as a SaaS service, right? So from the customer's perspective, they are onboarding into the supercloud. And then the supercloud is allowing them to do whatever they used to do, you know, historically and in the new world, right? That needs to come together. And that's what we have built is that, we have brought everything together in a way that what used to take months or years, and now taking an hour or two hours, and then people test it for a week or so and deploy it in production. >> I want to bring up something we were talking about before we were on camera about the TCP/IP, the OSI model. That was a concept that destroyed the proprietary narcissist. Work operating systems of the mini computers, which brought in an era of tech prosperity for generations. TCP/IP was kind of the magical moment that allowed for that kind of super networking connection. Inter networking is what's called as a category. It feels like something's going on here with supercloud. The way you describe it, it feels like there's this unification idea. Like the reality is we've got multiple stuff sitting around by default, you either clean it up or get rid of it, right? Or it's almost a, it's either a nuance, a new nuisance or chaos. >> Yeah. And we live in the new world now. We don't have the luxury of time. So we need to move as fast as possible to solve the business problems. And that's what we are running into. If we don't have automated solutions which scale, which solve our problems, then it's going to be a problem. And that's why SaaS is so important in today's world. Why should we have to deploy the network piecemeal? Why can't we have a solution? We solve our problem as we move forward and we accomplish what we need to accomplish and move forward. >> And we don't really need standards here, dude. It's not that we need a standards body if you have unification. >> So because things move so fast, there's no time to create a standards body. And that's why you see companies like ours popping up, which are trying to create a common infrastructure across all clouds. Otherwise if we vent the standardization path may take long. Eventually, we should be going in that direction. But we don't have the luxury of time. That's what I was trying to get to. >> Well, what's interesting is, is that to your point about standards and ratification, what ratifies a defacto anything? In the old days there was some technical bodies involved, but here, I think developers drive everything. So if you look at the developers and how they're voting with their code. They're instantly, organically defining everything as a collective intelligence. >> And just like you're putting out the paper and making it available, everybody's contributing to that. That's why you need to have APIs and terra form type constructs, which are available so that the customers can continue to improve upon that. And that's the Net DevOps, right? So that you need to have. >> What was once sacrilege, just sayin', in business school, back in the days when I got my business degree after my CS degree was, you know, no one wants to have a better mousetrap, a bad business model to have a better mouse trap. In this case, the better mouse trap, the better solution actually could be that thing. >> It is that thing. >> I mean, that can trigger, tips over the industry. >> And that that's where we are seeing our customers. You know, I mean, we have some publicly referenceable customers like Coke or Warner Music Group or, you know, multiple others and chart industries. The way we are solving the problem. They have some of the largest environments in the industry from the cloud perspective. And their whole network infrastructure is running on the Alkira infrastructure. And they're able to adopt new clouds within days rather than waiting for months to architect and then deploy and then figure out how to manage it and operate it. It's available as a service. >> John: And we've heard from your customer, Warner, they were just on the program. >> Amir: Yes. Okay, okay. >> So they're building a supercloud. So superclouds aren't just for tech companies. >> Amir: No. >> You guys build a supercloud for networking. >> Amir: It is. >> But people are building their own superclouds on top of all this new stuff. Talk about that dynamic. >> Healthcare providers, financials, high-tech companies, even startups. One of our startup customers, Tekion, right? They have these dealerships that they provide sales and support services to across the globe. And for them to be able to onboard those dealerships, it is 80% less time to production. That is real money, right? So, maybe Atif can give you a lot more examples of customers who are deploying. >> Talk about some of the customer activity. What are they like? Are they laggards, they innovators? Are they trying to hit the easy button? Are they coming in late or are you got some high customers? >> Actually most of our customers, all of our customers or customers in general. I don't think they have a choice but to move in this direction because, you know, the cloud has, like everything is quick now. So the cloud teams are moving faster in these enterprises. So now that they cannot afford the network nor to keep up pace with the cloud teams. So, they don't have a choice but to go with something similar where you can, you know, build your network on demand and bring up your network as quickly as possible to meet all those use cases. So, I'll give you an example. >> John: So the demand's high for what you guys do. >> Demand is very high because the cloud teams have- >> John: Yeah. They're going fast. >> They're going fast and there's no stopping. And then network teams, they have to keep up with them. And you cannot keep deploying, you know, networks the way you used to deploy back in the day. And as far as the use cases are concerned, there are so many use cases which our customers are using our platform for. One of the use cases, I'll give you an example of these financial customers. Some of the financial customers, they have their customers who they provide data, like stock exchanges, that provide like market data information to their customers out of data centers part. But now, their customers are moving into the cloud as well. So they need to come in from the cloud. So when they're coming in from the cloud, you cannot be giving them data from your data center because that takes time, and your hair pinning everything back. >> Moving data is like moving, moving money, someone said. >> Exactly. >> Exactly. And the other thing is like you have to optimize your traffic flows in the cloud as well because every time you leave the cloud, you get charged a lot. So, you don't want to leave the cloud unless you have to leave the cloud, your traffic. So, you have to come up or use a service which allows you to optimize all those traffic flows as well, you know? >> My final question to you guys, first of all, thanks for coming on Supercloud Program. Really appreciate it. Congratulations on your success. And you guys have a great positioning and I'm a big fan. And I have to ask, you guys are agile, nimble startup, smart on the cutting edge. Supercloud concept seems to resonate with people who are kind of on the front range of this major wave. While all the incumbents like Cisco, Microsoft, even AWS, they're like, I think they're looking at it, like what is that? I think it's coming up really fast, this trend. Because I know people talk about multi-cloud, I get that. But like, this whole supercloud is not just SaaS, it's more going on there. What do you think is going on between the folks who get it, supercloud, get the concept, and some are who are scratching their heads, whether it's the Ciscos or someone, like I don't get it. Why is supercloud important for the folks that aren't really seeing it? >> So first of all, I mean, the customers, what we saw about six months, 12 months ago, were a little slower to adopt the supercloud kind of concept. And there were leading edge customers who were coming and adopting it. Now, all of a sudden, over the last six to nine months, we've seen a flurry of customers coming in and they are from all disciplines or all very diverse set of customers. And they're starting to see the value of that because of the practical implications of what they're doing. You know, these shadow IT type environments are no longer working and there's a lot of pressure from the management to move faster. And then that's where they're coming in. And perhaps, Atif, if you can give a few examples of. >> Yeah. And I'll also just add to your point earlier about the network needing to be there 'cause the cloud teams are like, let's go faster. And the network's always been slow because, but now, it's been almost turbocharged. >> Atif: Yeah. Yeah, exactly. And as I said, like there was no choice here. You had to move in this industry. And the other thing I would add a little bit is now if you look at all these enterprises, most of their traffic is from, even from which is coming from the on-prem, it's going to the cloud SaaS applications or public clouds. And it's more than 50% of traffic, which is leaving your, you know, what you used to call, your network or the private network. So now it's like, you know, before it used to just connect sites to data centers and sites together. Now, it's a cloud as well as the SaaS application. So it's either internet bound or the public cloud bound. So now you have to build a network quickly, which caters to all these use cases. And that's where like something- >> And you guys, your solution to me is you eliminate all that work for the customer. Now, they can treat the cloud like a bag of Legos. And do their thing. Well, I oversimplify. Well, you know I'm talking about. >> Atif: Right, exactly. >> And to answer your question earlier about what about the big companies coming in and, you know, now they slow to adopt? And, you know, what normally happens is when Cisco came up, right? There used to be 16 different protocols suites. And then we finally settled on TCP/IP and DECnet or AppleTalk or X&S or, you know, you name it, right? Those companies did not adapt to the networking the way it was supposed to be done. And guess what happened, right? So if the companies in the networking space do not adopt this new concept or new way of doing things, I think some of them will become extinct over time. >> Well, I think the force and function too is the cloud teams as well. So you got two evolutions. You got architectural relevance. That's real as impact. >> It's very important. >> Cost, speed. >> And I look at it as a very similar disruption to what Cisco's the world, very early days did to, you know, bring the networking out, right? And it became the internet. But now we are going through the cloud. It's the cloud era, right? How does the cloud evolve over the next 10, 15, 20 years? Everything's is going to be offered as a service, right? So slowly data centers go away, the network becomes a plumbing thing. Very, you know, simple to deploy. And everything on top of that is virtualized in the cloud-like manners. >> And that makes the networks hardened and more secure. >> More secure. >> It's a great way to be secure. You remember the glory days, we'll go back 15 years. The Cisco conversation was, we got to move up to stack. All the manager would fight each other. Now, what does that actually mean? Stay where we are. Stay in your lane. This is kind of like the network's version of moving up the stack because not so much up the stack, but the cloud is everywhere. It's almost horizontally scaled. >> It's extending into the on-premise. It is already moving towards the edge, right? So, you will see a lot- >> So, programmability is a big program. So you guys are hitting programmability, compatibility, getting people into an environment they're comfortable operating. So the Ops people love it. >> Exactly. >> Spans the clouds to a level of SLA management. It might not be perfectly spanning applications, but you can actually know latencies between clouds, measure that. And then so you're basically managing your network now as the overall infrastructure. >> Right. And it needs to be a very intelligent infrastructure going forward, right? Because customers do not want to wait to be able to troubleshoot. They don't want to be able to wait to deploy something, right? So, it needs to be a level of automation. >> Okay. So the question for you guys both on we'll end on is what is the enablement that, because you guys are a disruptive enabler, right? You create this fabric. You're going to enable companies to do stuff. What are some of the things that you see and your customers might be seeing as things that they're going to do as a result of having this enablement? So what are some of those things? >> Amir: Atif, perhaps you can talk through the some of the customer experience on that. >> It's agility. And we are allowing these customers to move very, very quickly and build these networks which meet all these requirements inside the cloud. Because as Amir was saying, in the cloud era, networking is changing. And if you look at, you know, going back to your comment about the existing networking vendors. Some of them still think that, you know, just connecting to the cloud using some concepts like Cloud OnRamp is cloud networking, but it's changing now. >> John: 'Cause there's apps that are depending upon. >> Exactly. And it's all distributed. Like IT infrastructure, as I said earlier, is all distributed. And at the end of the day, you have to make sure that wherever your user is, wherever your app is, you are able to connect them securely. >> Historically, it used to be about building a router bigger and bigger and bigger and bigger, you know, and then interconnecting those routers. Now, it's all about horizontal scale. You don't need to build big, you need to scale it, right? And that's what cloud brings to the customer. >> It's a cultural change for Cisco and Juniper because they have to understand that they're still could be in the game and still win. >> Exactly. >> The question I have for you, what are your customers telling you that, what's some of the anecdotal, like, 'cause you guys have a good solution, is it, "Oh my god, you guys saved my butt." Or what are some of the commentary that you hear from the customers in terms of praise and and glory from your solution? >> Oh, some even say, when we do our demo and stuff, they say it's too hard to believe. >> Believe. >> Like, too hard. It's hard, you know, it's >> I dont believe you. They're skeptics. >> I don't believe you that because now you're able to bring up a global network within minutes. With networking services, like let's say you have APAC, you know, on-prem users, cloud also there, cloud here, users here, you can bring up a global network with full routed connectivity between all these endpoints with security services. You can bring up like a firewall from a third party or our services in the middle. This is a matter of minutes now. And this is all high speed connectivity with SLAs. Imagine like before connecting, you know, Singapore to U.S. East or Hong Kong to Frankfurt, you know, if you were putting your infrastructure in columns like E-connects, you would have to go, you know, figure out like, how am I going to- >> Seal line In, connect to it? Yeah. A lot of hassles, >> If you had to put like firewalls in the middle, segmentation, you had to, you know, isolate different entities. >> That's called heavy lifting. >> So what you're seeing is, you know, it's like customer comes in, there's a disbelief, can you really do that? And then they try it out, they go, "Wow, this works." Right? It's deployed in a small environment. And then all of a sudden they start taking off, right? And literally we have seen customers go from few thousand dollars a month or year type deployments to multi-million dollars a year type deployments in very, very short amount of time, in a few months. >> And you guys are pay as you go? >> Pay as you go. >> Pay as go usage cloud-based compatibility. >> Exactly. And it's amazing once they get to deploy the solution. >> What's the variable on the cost? >> On the cost? >> Is it traffic or is it. >> It's multiple different things. It's packaged into the overall solution. And as a matter of fact, we end up saving a lot of money to the customers. And not only in one way, in multiple different ways. And we do a complete TOI analysis for the customers. So it's bandwidth, it's number of connections, it's the amount of compute power that we are using. >> John: Similar things that they're used to. >> Just like the cloud constructs. Yeah. >> All right. Networking supercloud. Great. Congratulations. >> Thank you so much. >> Thanks for coming on Supercloud. >> Atif: Thank you. >> And looking forward to seeing more of the demand. Translate, instant networking. I'm sure it's going to be huge with the edge exploding. >> Oh yeah, yeah, yeah, yeah. >> Congratulations. >> Thank you so much. >> Thank you so much. >> Okay. So this is Supercloud 2 event here in Palo Alto. I'm John Furrier. The network Supercloud is here. Checkout Alkira. I'm John Furry, the host. Thanks for watching. (lively music)

>> John: Hello and welcome to this cube conversation here in Palo Alto, California. I'm John Furrier, host of theCUBE. We got a great conversation on making machine learning easier and more affordable in an era where everybody wants more machine learning and AI. We're featuring Neural Magic with the CEO is also Cube alumni, Brian Steve. CEO, Great to see you Brian. Thanks for coming on this cube conversation. Talk about machine learning. >> Brian: Hey John, happy to be here again. >> John: What a buzz that's going on right now? Machine learning, one of the hottest topics, AI front and center, kind of going mainstream. We're seeing the success of the, of the kind of NextGen capabilities in the enterprise and in apps. It's a really exciting time. So perfect timing. Great, great to have this conversation. Let's start with taking a minute to explain what you guys are doing over there at Neural Magic. I know there's some history there, neural networks, MIT. But the, the convergence of what's going on, this big wave hitting, it's an exciting time for you guys. Take a minute to explain the company and your mission. >> Brian: Sure, sure, sure. So, as you said, the company's Neural Magic and spun out at MIT four plus years ago, along with some people and, and some intellectual property. And you summarize it better than I can cause you said, we're just trying to make, you know, AI that much easier. And so, but like another level of specificity around it is. You know, in the world you have a lot of like data scientists really focusing on making AI work for whatever their use case is. And then the next phase of that, then they're looking at optimizing the models that they built. And then it's not good enough just to work on models. You got to put 'em into production. So, what we do is we make it easier to optimize the models that have been developed and trained and then trying to make it super simple when it comes time to deploying those in production and managing them. >> Brian: You know, we've seen this movie before with the cloud. You start to see abstractions come out. Data science we saw like was like the, the secret art of being like a data scientist now democratization of data. You're kind of seeing a similar wave with machine learning models, foundational models, some call it developers are getting involved. Model complexity's still there, but, but it's getting easier. There's almost like the democratization happening. You got complexity, you got deployment, it's challenges, cost, you got developers involved. So it's like how do you grow it? How do you get more horsepower? And then how do you make developers productive, right? So like, this seems to be the thread. So, so where, where do you see this going? Because there's going to be a massive demand for, I want to do more with my machine learning. But what's the data source? What's the formatting? This kind of a stack develop, what, what are you guys doing to address this? Can you take us through and demystify this, this wave that's hitting, that everyone's seeing? >> Brian: Yeah. Now like you said, like, you know, the democratization of all of it. And that brings me all the way back to like the roots of open source, right? When you think about like, like back in the day you had to build your own tech stack yourself. A lot of people probably probably don't remember that. And then you went, you're building, you're always starting on a body of code or a module that was out there with open source. And I think that's what I equate to where AI has gotten to with what you were talking about the foundational models that didn't really exist years ago. So you really were like putting the layers of your models together in the formulas and it was a lot of heavy lifting. And so there was so much time spent on development. With far too few success cases, you know, to get into production to solve like a business stereo technical need. But as these, what's happening is as these models are becoming foundational. It's meaning people don't have to start from scratch. They're actually able to, you know, the avant-garde now is start with existing model that almost does what you want, but then applying your data set to it. So it's, you know, it's really the industry moving forward. And then we, you know, and, and the best thing about it is open source plays a new dimension, but this time, you know, in the, in the realm of AI. And so to us though, like, you know, I've been like, I spent a career focusing on, I think on like the, not just the technical side, but the consumption of the technology and how it's still way too hard for somebody to actually like, operationalize technology that all those vendors throw at them. So I've always been like empathetic the user around like, you know what their job is once you give them great technology. And so it's still too difficult even with the foundational models because what happens is there's really this impedance mismatch between the development of the model and then where, where the model has to live and run and be deployed and the life cycle of the model, if you will. And so what we've done in our research is we've developed techniques to introduce what's known as sparsity into a machine learning model. It's already been developed and trained. And what that sparsity does is that unlocks by making that model so much smaller. So in many cases we can make a model 90 to 95% smaller, even smaller than that in research. So, and, and so by doing that, we do that in a way that preserves all the accuracy out of the foundational model as you talked about. So now all of a sudden you get this much smaller model just as accurate. And then the even more exciting part about it is we developed a software-based engine called Deep Source. And what that, what the Inference Runtime does is takes that now sparsified model and it runs it, but because you sparsified it, it only needs a fraction of the compute that it, that it would've needed otherwise. So what we've done is make these models much faster, much smaller, and then by pairing that with an inference runtime, you now can actually deploy that model anywhere you want on commodity hardware, right? So X 86 in the cloud, X 86 in the data center arm at the edge, it's like this massive unlock that happens because you get the, the state-of-the-art models, but you get 'em, you know, on the IT assets and the commodity infrastructure. That is where all the applications are running today. >> John: I want to get into the inference piece and the deep sparse you mentioned, but I first have to ask, you mentioned open source, Dave and I with some fellow cube alumnis. We're having a chat about, you know, the iPhone and Android moment where you got proprietary versus open source. You got a similar thing happening with some of these machine learning modules where there's a lot of proprietary things happening and there's open source movement is growing. So is there a balance there? Are they all trying to do the same thing? Is it more like a chip, you know, silicons involved, all kinds of things going on that are really fascinating from a science. What's your, what's your reaction to that? >> Brian: I think it's like anything that, you know, the way we talk about AI you think had been around for decades, but the reality is it's been some of the deep learning models. When we first, when we first started taking models that the brain team was working on at Google and billing APIs around them on Google Cloud where the first cloud to even have AI services was 2015, 2016. So when you think about it, it's really been what, 6 years since like this thing is even getting lift off. So I think with that, everybody's throwing everything at it. You know, there's tons of funded hardware thrown at specialty for training or inference new companies. There's legacy companies that are getting into like AI now and whether it's a, you know, a CPU company that's now building specialized ASEX for training. There's new tech stacks proprietary software and there's a ton of asset service. So it really is, you know, what's gone from nascent 8 years ago is the wild, wild west out there. So there's a, there's a little bit of everything right now and I think that makes sense because at the early part of any industry it really becomes really specialized. And that's the, you know, showing my age of like, you know, the early pilot of the two thousands, you know, red Hat people weren't running X 86 in enterprise back then and they thought it was a toy and they certainly weren't running open source, but you really, and it made sense that they weren't because it didn't deliver what they needed to at that time. So they needed specialty stacks, they needed expensive, they needed expensive hardware that did what an Oracle database needed to do. They needed proprietary software. But what happens is that commoditizes through both hardware and through open source and the same thing's really just starting with with AI. >> John: Yeah. And I think that's a great point before we to call that out because in any industry timing's everything, right? I mean I remember back in the 80s, late 80s and 90s, AI, you know, stuff was going on and it just wasn't, there wasn't enough horsepower, there wasn't enough tech. >> Brian: Yep. >> John: You mentioned some of the processing. So AI is this industry that has all these experts who have been itch scratching that itch for decades. And now with cloud and custom silicon. The tech fundamental at the lower end of the stack, if you will, on the performance side is significantly more performant. It's there you got more capabilities. >> Brian: Yeah. >> John: Now you're kicking into more software, faster software. So it just seems like we're at a tipping point where finally it's here, like that AI moment or machine learning and now data is, is involved. So this is where organizations I see really jumping in with the CEO mandate. Hey team, make ML work for us. Go figure it out. It's got to be an advantage for us. >> Brian: Yeah. >> John: So now they go, okay boss, we will. So what, what do they do? What's the steps does an enterprise take to get machine learning into their organizations? Cause you know, it's coming down from the boards, you know, how does this work for rob? >> Brian: Yeah. Like the, you know, the, what we're seeing is it's like anything, like it's, whether that was source adoption or whether that was cloud adoption, it always starts usually with one person. And increasingly it is the CEO, which realizes they're getting further behind the competition because they're not leaning in, you know, faster. But typically it really comes down to like a really strong practitioner that's inside the organization, right? And, that realizes that the number one goal isn't doing more and just training more models and and necessarily being proprietary about it. It's really around understanding the art of the possible. Something that's grounded in the art of the possible, what, what deep learning can do today and what business outcomes you can deliver, you know, if you can employ. And then there's well proven paths through that. It's just that because of where it's been, it's not that industrialized today. It's very much, you know, you see ML project by ML project is very snowflakey, right? And that was kind of the early days of open source as well. And so, we're just starting to get to the point where it's getting easier, it's getting more industrialized, there's less steps, there's less burdensome on developers, there's less burdensome on, on the deployment side. And we're trying to bring that, that whole last mile by saying, you know what? Deploying deep learning and AI models should be as easy as the as to deploy your application, right? You shouldn't have to take an extra step to deploy an AI model. It shouldn't have to require a new hardware, it shouldn't require a new process, a new DevOps model. It should be as simple as what you're already doing. >> John: What is the best practice for companies to effectively bring an acceptable level of machine learning and performance into their organizations? >> Brian: Yeah, I think like the, the number one start is like what you hinted at before is they, they have to know the use case. They have to, in most cases, you're going to find across every industry you know, that that problem's been tackled by some company, right? And then you have to have the best practice around fine-tuning the models already exist. So fine tuning that existing model. That foundational model on your unique dataset. You, you know, if you are in medical instruments, it's not good enough to identify that it's a medical instrument in the picture. You got to know what type of medical instrument. So there's always a fine tuning step. And so we've created open source tools that make it easy for you to do two things at once. You can fine tune that existing foundational model, whether that's in the language space or whether that's in the vision space. You can fine tune that on your dataset. And at the same time you get an optimized model that comes out the other end. So you get kind of both things. So you, you no longer have to worry about you're, we're freeing you from worrying about the complexity of that transfer learning, if you will. And we're freeing you from worrying about, well where am I going to deploy the model? Where does it need to be? Does it need to be on a device, an edge, a data center, a cloud edge? What kind of hardware is it? Is there enough hardware there? We're liberating you from all of that. Because what you want, what you can count on is there'll always be commodity capability, commodity CPUs where you want to deploy in abundance cause that's where your application is. And so all of a sudden we're just freeing you of that, of that whole step. >> John: Okay. Let's get into deep sparse because you mentioned that earlier. What inspired the creation of deep sparse and how does it differ from any other solutions in the market that are out there? >> Brian: Sure. So, so where unique is it? It starts by, by two things. One is what the industry's pretty good at from the optimization side is they're good at like this thing called quantization, which turns like, you know, big numbers into small numbers, lower precision. So a 32 bit representation of a, of AI weight into a bit. And they're good at like cutting out layers, which also takes away accuracy. What we've figured out is to take those, the industry techniques for those that are best practice, but we combined it with unstructured varsity. So by reducing that model by 90 to 95% in size, that's great because it's made it smaller. But we've taken that when it's the deep sparse engine, when you deploy it that looks at that model and says, because it's so much smaller, I no longer have to run the part of the model that's been essentially sparsified. So what that's done is, it's meant that you no longer need a supercomputer to run models because there's not nearly as much math and processing as there was before the model was optimized. So now what happens is, every CPU platform out there has, has an enormous amount of compute because we've sparsified the rest of it away. So you can pick a, you can pick your, your laptop and you have enough compute to run state-of-the-art models. The second thing that, and you need a software engine to do that cause it ignores the parts of the models. It doesn't need to run, which is what like specialized hardware can't do. The second part is it's then turned into a memory efficiency problem. So it's really around just getting memory, getting the models loaded into the cash of the computer and keeping it there. Never having to go back out to memory. So, so our techniques are both, we reduce the model size and then we only run the part of the model that matters and then we keep it all in cash. And so what that does is it gets us to like these, these low, low latency faster and we're able to increase, you know, the CPU processing by an order magnitude. >> John: Yeah. That low latency is key. And you got developers, you know, co coding super fast. We'll get to the developer angle in a second. I want to just follow up on this, this motivation behind the, the deep sparse because you know, as we were talking earlier before we came on camera about the old days, I mean, not too long ago, virtualization and VMware abstracted away the os from, from the hardware rights and the server virtualization changed the game. >> Brian: Yeah. >> John: And that basically invented cloud computing as we know it today. So, so we see that abstraction. >> Brian: Yeah. >> John: There seems to be a motivation behind abstracting the way the machine learning models away from the hardware. And that seems to be bringing advantages to the AI growth. Can you elaborate on, is that true? And it's, what's your comment? >> Brian: It's true. I think it's true for us. I don't think the industry's there yet, honestly. Cause I think the industry still is of that mindset that if I took, if it took these expensive GPUs to train my model, then I want to run my model on those same expensive GPUs. Because there's often like not a separation between the people that are developing AI and the people that have to manage and deploy at where you need it. So the reality is, is that that's everything that we're after. Like, do we decrease the cost? Yes. Do we make the models smaller? Yes. Do we make them faster? A yes. But I think the most amazing power is that we've turned AI into a docker based microservice. And so like who in the industry wants to deploy their apps the old way on a os without virtualization, without docker, without Kubernetes, without microservices, without service mesh without serverless. You want all those tools for your apps by converting AI models. So they can be run inside a docker container with no apologies around latency and performance cause it's faster. You get the best of that whole world that you just talked about, which is, you know, what we're calling, you know, software delivered AI. So now the AI lives in the same world. Organizations that have gone through that digital cloud transformation with their app infrastructure. AI fits into that world. >> John: And this is where the abstraction concepts matter. When you have these inflection points, the convergence of compute data, machine learning that powers AI, it really becomes a developer opportunity. Because now applications and businesses, when they actually go through the digital transformation, their businesses are completely transformed. There is no IT. Developers are the application. They are the company, right? So AI will be part of whatever business or app will be out there. So there is a application developer angle here. Brian, can you explain >> Brian: Oh completely. >> John: how they're going to use this? Because you mentioned docker container microservice, I mean this really is an insane flipping of the script for developers. >> Brian: Yeah. >> John: So what's that look like? >> Brian: Well speak, it's because like AI's kind of, I mean, again, like it's come so fast. So you figure there's my app team and here's my AI team, right? And they're in different places and the AI team is dragging in specialized infrastructure in support of that as well. And that's not how app developers think. Like they've ran on fungible infrastructure that subtracted and virtualized forever, right? And so what we've done is we've, in addition to fitting into that world that they, that they like, we've also made it simple for them for they don't have to be a machine learning engineer to be able to experiment with these foundational models and transfer learning 'em. We've done that. So they can do that in a couple of commands and it has a simple API that they can either link to their application directly as a library to make difference calls or they can stand it up as a standalone, you know, scale up, scale out inference server. They get two choices. But it really fits into that, you know, you know that world that the modern developer, whether they're just using Python or C or otherwise, we made it just simple. So as opposed to like Go learn something else, they kind of don't have to. So in a way though, it's made it. It's almost made it hard because people expect when we talk to 'em for the first time to be the old way. Like, how do you look like a piece of hardware? Are you compatible with my existing hardware that runs ML? Like, no, we're, we're not. Because you don't need that stack anymore. All you need is a library called to make your prediction and that's it. That's it. >> John: Well, I mean, we were joking on Twitter the other day with someone saying, is AI a pet or a cattle? Right? Because they love their, their AI bots right now. So, so I'd say pet there. But you look at a lot of, there's going to be a lot of AI. So on a more serious note, you mentioned in microservices, will deep sparse have an API for developers? And how does that look like? What do I do? >> Brian: Yeah. >> John: tell me what my, as a developer, what's the roadmap look like? What's the >> Brian: Yeah, it, it really looks, it really can go in both modes. It can go in a standalone server mode where it handles, you know, rest API and it can scale out with ES as the workload comes up and scale back and like try to make hardware do that. Hardware may scale back, but it's just sitting there dormant, you know, so with this, it scales the same way your application needs to. And then for a developer, they basically just, they just, the PIP install de sparse, you know, has one commanded to do an install, and then they do two calls, really. The first call is a library call that the app makes to create the model. And models really already trained, but they, it's called a model create call. And the second command they do is they make a call to do a prediction. And it's as simple as that. So it's, it's AI's as simple as using any other library that the developers are already using, which I, which sounds hard to fathom because it is just so simplified. >> John: Software delivered AI. Okay, that's a cool thing. I believe in it personally. I think that's the way to go. I think there's going to be plenty of hardware options if you look at the advances of cloud players that got more silicon coming out. Yeah. More GPU. I mean, there's more instance, I mean, everything's out there right now. So the question is how does that evolve in your mind? Because that's seems to be key. You have open source projects emerging. What, what path does this take? Is there a parallel mental model that you see, Brian, that is similar? You mentioned open source earlier. Is it more like a VMware virtualization thing or is it more of a cloud thing? Is there Yeah. Is it going to evolve in a, in a trajectory that looks similar to what we might've seen in the past? >> Brian: Yeah, we're, you know, when I, when when I got involved with the company, what I, when I thought about it and I was reasoning about it, like, do you, you know, you want to, like, we all do when you want to join something full-time. I thought about it and said, where will the industry eventually get to? Right? To fully realize the value of, of deep learning and what's plausible as it evolves. And to me, like I, I know it's the old adage of, you know, you know, software, its hardware, cloudy software. But it truly was like, you know, we can solve these problems in software. Like there's nothing special that's happening at the hardware layer and the processing AI. The reality is that it's just early in the industry. So the view that that we had was like, this is eventually the best place where the industry will be, is the liberation of being able to run AI anywhere. Like you're really not democratizing, you democratize the model. But if you can't run the model anywhere you want because these models are getting bigger and bigger with these large language models, then you're kind of not democratizing. And if you got to go and like by a cluster to run this thing on. So the democratization comes by if all of a sudden that model can be consumed anywhere on demand without planning, without provisioning, wherever infrastructure is. And so I think that's with or without Neural Magic, that's where the industry will go and will get to. I think we're the leaders, leaders in getting it there. It's right because we're more advanced on these techniques. >> John: Yeah. And your background too. You've seen OpenStack, pre-cloud, you saw open source grow and still exponentially growing. And so you have the same similar dynamic with machine learning models growing. And they're also segmenting into almost a, an ML stack or foundational model as we talk about. So you're starting to see the formation of tooling inference. So a lot of components coming. It's almost a stack, it's almost a, it literally is like an operating system problem space, you know? How do you run things, how do you link things? How do you bring things together? Is that what's going on here? Is this like a data modeling operating environment kind of red hat type thing going on? Like. >> Brian: Yeah. Yeah. Like I think there is, you know, I thought about that too. And I think there is the role of like distribution, because the industrialization not happening fast enough of this. Like, can I go back to like every customers, every, every user does it in their own kind of way. Like it's not, everyone's a little bit of a snowflake. And I think that's okay. There's definitely plenty of companies that want to come in and say, well, this is the way it's going to be and we industrialize it as long as you do it our way. The reality is technology doesn't get industrialized by one company just saying, do it our way. And so that's why like we've taken the approach through open source by saying like, Hey, you haven't really industrialized it if you said. We made it simple, but you always got to run AI here. Yeah, right. You only like really industrialize it if you break it down into components that are simple to use and they work integrated in the stack the way you want them to. And so to me, that first principles was getting thing into microservices and dockers that could be run on VMware, OpenShare on the cloud in the edge. And so that's the, that's the real part that we're happening with. The other part, like I do agree, like I think it's going to quickly move into less about the model. Less about the training of the model and the transfer learning, you know, the data set of the model. We're taking away the complexity of optimization. Giving liberating deployment to be anywhere. And I think the last mile, John is going to be around the ML ops around that. Because it's easy to think of like soft now that it's just a software problem, we've turned it into a software problem. So it's easy to think of software as like kind of a point release, but that's not the reality, right? It's a life cycle. And it's, and so I think ML very much brings in the what is the lifecycle of that deployment? And, you know, you get into more interesting conversations, to be honest than like, once you've deployed in a docking container is around like model drift and accuracy and the dataset changes and the user changes is how do you become from an ML perspective of where of that sending signal back retraining. And, and that's where I think a lot of the, in more of the innovation's going to start to move there. >> John: Yeah. And software also, the software problem, the software opportunity as well is developer focused. And if you look at the cloud native landscape now, similar stacks developing a lot of components. A lot of things to, to stitch together a lot of things that are automating under the hood. A lot of developer productivity conversations. I think this is going to go down that same road. I want to get your thoughts because developers will set the pace. And this is something that's clear in this next wave developer productivity. They're the defacto standards bodies. They will decide what microservices check, API check. Now, skill gap is going to be a problem because it's relatively new. So model sprawl, model sizes, proprietary versus open. There has to be a way to kind of crunch that down into a, like a DevOps, like just make it, get the developer out of the, the muck. So what's your view? Are we early days like that? Or what's the young kid in college studying CS or whatever degree who comes into this with, with both feet? What are they doing? >> Brian: I'll probably say like the, the non-popular answer to that. A little bit is it's happening so fast that it's going to get kind of boring fast. Meaning like, yeah, you could go to school and go to MIT, right? Sorry. Like, and you could get a hold through end like becoming a model architect, like inventing the next model, right? And the layers and combining 'em and et cetera, et cetera. And then what operators and, and building a model that's bigger than the last one and trains faster, right? And there will be those people, right? That actually, like they're building the engines the same way. You know, I grew up as an infrastructure software developer. There's not a lot of companies that hire those anymore because they're all sitting inside of three big clouds. Yeah. Right? So you better be a good app developer, but I think what you're going to see is before you had to be everything, you had to be the, if you were going to use infrastructure, you had to know how to build infrastructure. And I think the same thing's true around is quickly exiting ML is to be able to use ML in your company, you better be like, great at every aspect of ML, including every intricacy inside of the model and every operation's doing, that's quickly changing. Like, you're going to start with a starting point. You know, in the future you're not going to be like cracking open these GPT models, you're going to just be pulling them off the shelf, fine tuning 'em and go. You don't have to invent it. You don't have to understand it. And I think that's going to be a pivot point, you know, in the industry between, you know, what's the future? What's, what's the future of a, a data scientist? ML engineer researcher look like? >> John: I think that's, the outcome's going to be determined. I mean, you mentioned, you know, doing it yourself what an SRE is for a Google with the servers scale's huge. So yeah, it might have to, at the beginning get boring, you get obsolete quickly, but that means it's progressing. So, The scale becomes huge. And that's where I think it's going to be interesting when we see that scale. >> Brian: Yep. Yeah, I think that's right. I think that's right. And we always, and, and what I've always said, and much the, again, the distribute into my ML team is that I want every developer to be as adept at being able take advantage of ML as non ML engineer, right? It's got to be that simple. And I think, I think it's getting there. I really do. >> John: Well, Brian, great, great to have you on theCUBE here on this cube conversation. As part of the startup showcase that's coming up. You're going to be featured. Or your company would featured on the upcoming ABRA startup showcase on making machine learning easier and more affordable as more machine learning models come in. You guys got deep sparse and some great technology. We're going to dig into that next time. I'll give you the final word right now. What do you see for the company? What are you guys looking for? Give a plug for the company right now. >> Brian: Oh, give a plug that I haven't already doubled in as the plug. >> John: You're hiring engineers, I assume from MIT and other places. >> Brian: Yep. I think like the, the biggest thing is like, like we're on the developer side. We're here to make this easy. The majority of inference today is, is on CPUs already, believe it or not, as much as kind of, we like to talk about hardware and specialized hardware. The majority is already on CPUs. We're basically bringing 95% cost savings to CPUs through this acceleration. So, but we're trying to do it in a way that makes it community first. So I think the, the shout out would be come find the Neural Magic community and engage with us and you'll find, you know, a thousand other like-minded people in Slack that are willing to help you as well as our engineers. And, and let's, let's go take on some successful AI deployments. >> John: Exciting times. This is, I think one of the pivotal moments, NextGen data, machine learning, and now starting to see AI not be that chat bot, just, you know, customer support or some basic natural language processing thing. You're starting to see real innovation. Brian Stevens, CEO of Neural Magic, bringing the magic here. Thanks for the time. Great conversation. >> Brian: Thanks John. >> John: Thanks for joining me. >> Brian: Cheers. Thank you. >> John: Okay. I'm John Furrier, host of theCUBE here in Palo Alto, California for this cube conversation with Brian Stevens. Thanks for watching.

>> From theCUBE studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vellante. >> Confidential computing is a technology that aims to enhance data privacy and security by providing encrypted computation on sensitive data and isolating data from apps in a fenced off enclave during processing. The concept of confidential computing is gaining popularity, especially in the cloud computing space where sensitive data is often stored and of course processed. However, there are some who view confidential computing as an unnecessary technology in a marketing ploy by cloud providers aimed at calming customers who are cloud phobic. Hello and welcome to this week's Wikibon CUBE Insights powered by ETR. In this Breaking Analysis, we revisit the notion of confidential computing, and to do so, we'll invite two Google experts to the show, but before we get there, let's summarize briefly. There's not a ton of ETR data on the topic of confidential computing. I mean, it's a technology that's deeply embedded into silicon and computing architectures. But at the highest level, security remains the number one priority being addressed by IT decision makers in the coming year as shown here. And this data is pretty much across the board by industry, by region, by size of company. I mean we dug into it and the only slight deviation from the mean is in financial services. The second and third most cited priorities, cloud migration and analytics, are noticeably closer to cybersecurity in financial services than in other sectors, likely because financial services has always been hyper security conscious, but security is still a clear number one priority in that sector. The idea behind confidential computing is to better address threat models for data in execution. Protecting data at rest and data and transit have long been a focus of security approaches, but more recently, silicon manufacturers have introduced architectures that separate data and applications from the host system. Arm, Intel, AMD, Nvidia and other suppliers are all on board, as are the big cloud players. Now the argument against confidential computing is that it narrowly focuses on memory encryption and it doesn't solve the biggest problems in security. Multiple system images updates different services and the entire code flow aren't directly addressed by memory encryption, rather to truly attack these problems, many believe that OSs need to be re-engineered with the attacker and hacker in mind. There are so many variables and at the end of the day, critics say the emphasis on confidential computing made by cloud providers is overstated and largely hype. This tweet from security researcher Rodrigo Branco sums up the sentiment of many skeptics. He says, "Confidential computing is mostly a marketing campaign for memory encryption. It's not driving the industry towards the hard open problems. It is selling an illusion." Okay. Nonetheless, encrypting data in use and fencing off key components of the system isn't a bad thing, especially if it comes with the package essentially for free. There has been a lack of standardization and interoperability between different confidential computing approaches. But the confidential computing consortium was established in 2019 ostensibly to accelerate the market and influence standards. Notably, AWS is not part of the consortium, likely because the politics of the consortium were probably a conundrum for AWS because the base technology defined by the the consortium is seen as limiting by AWS. This is my guess, not AWS's words, and but I think joining the consortium would validate a definition which AWS isn't aligned with. And two, it's got a lead with this Annapurna acquisition. This was way ahead with Arm integration and so it probably doesn't feel the need to validate its competitors. Anyway, one of the premier members of the confidential computing consortium is Google, along with many high profile names including Arm, Intel, Meta, Red Hat, Microsoft, and others. And we're pleased to welcome two experts on confidential computing from Google to unpack the topic, Nelly Porter is head of product for GCP confidential computing and encryption, and Dr. Patricia Florissi is the technical director for the office of the CTO at Google Cloud. Welcome Nelly and Patricia, great to have you. >> Great to be here. >> Thank you so much for having us. >> You're very welcome. Nelly, why don't you start and then Patricia, you can weigh in. Just tell the audience a little bit about each of your roles at Google Cloud. >> So I'll start, I'm owning a lot of interesting activities in Google and again security or infrastructure securities that I usually own. And we are talking about encryption and when encryption and confidential computing is a part of portfolio in additional areas that I contribute together with my team to Google and our customers is secure software supply chain. Because you need to trust your software. Is it operate in your confidential environment to have end-to-end story about if you believe that your software and your environment doing what you expect, it's my role. >> Got it. Okay. Patricia? >> Well, I am a technical director in the office of the CTO, OCTO for short, in Google Cloud. And we are a global team. We include former CTOs like myself and senior technologists from large corporations, institutions and a lot of success, we're startups as well. And we have two main goals. First, we walk side by side with some of our largest, more strategic or most strategical customers and we help them solve complex engineering technical problems. And second, we are devise Google and Google Cloud engineering and product management and tech on there, on emerging trends and technologies to guide the trajectory of our business. We are unique group, I think, because we have created this collaborative culture with our customers. And within OCTO, I spend a lot of time collaborating with customers and the industry at large on technologies that can address privacy, security, and sovereignty of data in general. >> Excellent. Thank you for that both of you. Let's get into it. So Nelly, what is confidential computing? From Google's perspective, how do you define it? >> Confidential computing is a tool and it's still one of the tools in our toolbox. And confidential computing is a way how we would help our customers to complete this very interesting end-to-end lifecycle of the data. And when customers bring in the data to cloud and want to protect it as they ingest it to the cloud, they protect it at rest when they store data in the cloud. But what was missing for many, many years is ability for us to continue protecting data and workloads of our customers when they running them. And again, because data is not brought to cloud to have huge graveyard, we need to ensure that this data is actually indexed. Again, there is some insights driven and drawn from this data. You have to process this data and confidential computing here to help. Now we have end to end protection of our customer's data when they bring the workloads and data to cloud, thanks to confidential computing. >> Thank you for that. Okay, we're going to get into the architecture a bit, but before we do, Patricia, why do you think this topic of confidential computing is such an important technology? Can you explain, do you think it's transformative for customers and if so, why? >> Yeah, I would maybe like to use one thought, one way, one intuition behind why confidential commuting matters, because at the end of the day, it reduces more and more the customer's thresh boundaries and the attack surface. That's about reducing that periphery, the boundary in which the customer needs to mind about trust and safety. And in a way, is a natural progression that you're using encryption to secure and protect the data. In the same way that we are encrypting data in transit and at rest, now we are also encrypting data while in use. And among other beneficials, I would say one of the most transformative ones is that organizations will be able to collaborate with each other and retain the confidentiality of the data. And that is across industry, even though it's highly focused on, I wouldn't say highly focused, but very beneficial for highly regulated industries. It applies to all of industries. And if you look at financing for example, where bankers are trying to detect fraud, and specifically double finance where you are, a customer is actually trying to get a finance on an asset, let's say a boat or a house, and then it goes to another bank and gets another finance on that asset. Now bankers would be able to collaborate and detect fraud while preserving confidentiality and privacy of the data. >> Interesting. And I want to understand that a little bit more but I'm going to push you a little bit on this, Nelly, if I can because there's a narrative out there that says confidential computing is a marketing ploy, I talked about this upfront, by cloud providers that are just trying to placate people that are scared of the cloud. And I'm presuming you don't agree with that, but I'd like you to weigh in here. The argument is confidential computing is just memory encryption and it doesn't address many other problems. It is over hyped by cloud providers. What do you say to that line of thinking? >> I absolutely disagree, as you can imagine, with this statement, but the most importantly is we mixing multiple concepts, I guess. And exactly as Patricia said, we need to look at the end-to-end story, not again the mechanism how confidential computing trying to again, execute and protect a customer's data and why it's so critically important because what confidential computing was able to do, it's in addition to isolate our tenants in multi-tenant environments the cloud covering to offer additional stronger isolation. They called it cryptographic isolation. It's why customers will have more trust to customers and to other customers, the tenant that's running on the same host but also us because they don't need to worry about against threats and more malicious attempts to penetrate the environment. So what confidential computing is helping us to offer our customers, stronger isolation between tenants in this multi-tenant environment, but also incredibly important, stronger isolation of our customers, so tenants from us. We also writing code, we also software providers will also make mistakes or have some zero days. Sometimes again us introduced, sometimes introduced by our adversaries. But what I'm trying to say by creating this cryptographic layer of isolation between us and our tenants and amongst those tenants, we're really providing meaningful security to our customers and eliminate some of the worries that they have running on multi-tenant spaces or even collaborating to gather this very sensitive data knowing that this particular protection is available to them. >> Okay, thank you. Appreciate that. And I think malicious code is often a threat model missed in these narratives. Operator access, yeah, maybe I trust my clouds provider, but if I can fence off your access even better, I'll sleep better at night. Separating a code from the data, everybody's, Arm, Intel, AMD, Nvidia, others, they're all doing it. I wonder if, Nelly, if we could stay with you and bring up the slide on the architecture. What's architecturally different with confidential computing versus how operating systems and VMs have worked traditionally. We're showing a slide here with some VMs, maybe you could take us through that. >> Absolutely. And Dave, the whole idea for Google and now industry way of dealing with confidential computing is to ensure that three main property is actually preserved. Customers don't need to change the code. They can operate on those VMs exactly as they would with normal non-confidential VMs, but to give them this opportunity of lift and shift or no changing their apps and performing and having very, very, very low latency and scale as any cloud can, something that Google actually pioneer in confidential computing. I think we need to open and explain how this magic was actually done. And as I said, it's again the whole entire system have to change to be able to provide this magic. And I would start with we have this concept of root of trust and root of trust where we will ensure that this machine, when the whole entire post has integrity guarantee, means nobody changing my code on the most low level of system. And we introduce this in 2017 called Titan. It was our specific ASIC, specific, again, inch by inch system on every single motherboard that we have that ensures that your low level former, your actually system code, your kernel, the most powerful system is actually proper configured and not changed, not tampered. We do it for everybody, confidential computing included. But for confidential computing, what we have to change, we bring in AMD, or again, future silicon vendors and we have to trust their former, their way to deal with our confidential environments. And that's why we have obligation to validate integrity, not only our software and our former but also former and software of our vendors, silicon vendors. So we actually, when we booting this machine, as you can see, we validate that integrity of all of the system is in place. It means nobody touching, nobody changing, nobody modifying it. But then we have this concept of AMD secure processor, it's special ASICs, best specific things that generate a key for every single VM that our customers will run or every single node in Kubernetes or every single worker thread in our Hadoop or Spark capability. We offer all of that. And those keys are not available to us. It's the best keys ever in encryption space because when we are talking about encryption, the first question that I'm receiving all the time, where's the key, who will have access to the key? Because if you have access to the key then it doesn't matter if you encrypted or not. So, but the case in confidential computing provides so revolutionary technology, us cloud providers, who don't have access to the keys. They sitting in the hardware and they head to memory controller. And it means when hypervisors that also know about these wonderful things saying I need to get access to the memories that this particular VM trying to get access to, they do not decrypt the data, they don't have access to the key because those keys are random, ephemeral and per VM, but the most importantly, in hardware not exportable. And it means now you would be able to have this very interesting role that customers or cloud providers will not be able to get access to your memory. And what we do, again, as you can see our customers don't need to change their applications, their VMs are running exactly as it should run and what you're running in VM, you actually see your memory in clear, it's not encrypted, but God forbid is trying somebody to do it outside of my confidential box. No, no, no, no, no, they would not be able to do it. Now you'll see cyber and it's exactly what combination of these multiple hardware pieces and software pieces have to do. So OS is also modified. And OS is modified such way to provide integrity. It means even OS that you're running in your VM box is not modifiable and you, as customer, can verify. But the most interesting thing, I guess, how to ensure the super performance of this environment because you can imagine, Dave, that encrypting and it's additional performance, additional time, additional latency. So we were able to mitigate all of that by providing incredibly interesting capability in the OS itself. So our customers will get no changes needed, fantastic performance and scales as they would expect from cloud providers like Google. >> Okay, thank you. Excellent. Appreciate that explanation. So, again, the narrative on this as well, you've already given me guarantees as a cloud provider that you don't have access to my data, but this gives another level of assurance, key management as they say is key. Now humans aren't managing the keys, the machines are managing them. So Patricia, my question to you is, in addition to, let's go pre confidential computing days, what are the sort of new guarantees that these hardware-based technologies are going to provide to customers? >> So if I am a customer, I am saying I now have full guarantee of confidentiality and integrity of the data and of the code. So if you look at code and data confidentiality, the customer cares and they want to know whether their systems are protected from outside or unauthorized access, and that recovered with Nelly, that it is. Confidential computing actually ensures that the applications and data internals remain secret, right? The code is actually looking at the data, the only the memory is decrypting the data with a key that is ephemeral and per VM and generated on demand. Then you have the second point where you have code and data integrity, and now customers want to know whether their data was corrupted, tampered with or impacted by outside actors. And what confidential computing ensures is that application internals are not tampered with. So the application, the workload as we call it, that is processing the data, it's also, it has not been tampered and preserves integrity. I would also say that this is all verifiable. So you have attestation and these attestation actually generates a log trail and the log trail guarantees that, provides a proof that it was preserved. And I think that the offer's also a guarantee of what we call ceiling, this idea that the secrets have been preserved and not tampered with, confidentiality and integrity of code and data. >> Got it. Okay, thank you. Nelly, you mentioned, I think I heard you say that the applications, it's transparent, you don't have to change the application, it just comes for free essentially. And we showed some various parts of the stack before. I'm curious as to what's affected, but really more importantly, what is specifically Google's value add? How do partners participate in this, the ecosystem, or maybe said another way, how does Google ensure the compatibility of confidential computing with existing systems and applications? >> And a fantastic question by the way. And it's very difficult and definitely complicated world because to be able to provide these guarantees, actually a lot of work was done by community. Google is very much operate in open, so again, our operating system, we working with operating system repository OSs, OS vendors to ensure that all capabilities that we need is part of the kernels, are part of the releases and it's available for customers to understand and even explore if they have fun to explore a lot of code. We have also modified together with our silicon vendors a kernel, host kernel to support this capability and it means working this community to ensure that all of those patches are there. We also worked with every single silicon vendor as you've seen, and that's what I probably feel that Google contributed quite a bit in this whole, we moved our industry, our community, our vendors to understand the value of easy to use confidential computing or removing barriers. And now I don't know if you noticed, Intel is pulling the lead and also announcing their trusted domain extension, very similar architecture. And no surprise, it's, again, a lot of work done with our partners to, again, convince, work with them and make this capability available. The same with Arm this year, actually last year, Arm announced their future design for confidential computing. It's called Confidential Computing Architecture. And it's also influenced very heavily with similar ideas by Google and industry overall. So it's a lot of work in confidential computing consortiums that we are doing, for example, simply to mention, to ensure interop, as you mentioned, between different confidential environments of cloud providers. They want to ensure that they can attest to each other because when you're communicating with different environments, you need to trust them. And if it's running on different cloud providers, you need to ensure that you can trust your receiver when you are sharing your sensitive data workloads or secret with them. So we coming as a community and we have this attestation sig, the, again, the community based systems that we want to build and influence and work with Arm and every other cloud providers to ensure that we can interrupt and it means it doesn't matter where confidential workloads will be hosted, but they can exchange the data in secure, verifiable and controlled by customers way. And to do it, we need to continue what we are doing, working open, again, and contribute with our ideas and ideas of our partners to this role to become what we see confidential computing has to become, it has to become utility. It doesn't need to be so special, but it's what we want it to become. >> Let's talk about, thank you for that explanation. Let's talk about data sovereignty because when you think about data sharing, you think about data sharing across the ecosystem and different regions and then of course data sovereignty comes up. Typically public policy lags, the technology industry and sometimes is problematic. I know there's a lot of discussions about exceptions, but Patricia, we have a graphic on data sovereignty. I'm interested in how confidential computing ensures that data sovereignty and privacy edicts are adhered to, even if they're out of alignment maybe with the pace of technology. One of the frequent examples is when you delete data, can you actually prove that data is deleted with a hundred percent certainty? You got to prove that and a lot of other issues. So looking at this slide, maybe you could take us through your thinking on data sovereignty. >> Perfect. So for us, data sovereignty is only one of the three pillars of digital sovereignty. And I don't want to give the impression that confidential computing addresses it all. That's why we want to step back and say, hey, digital sovereignty includes data sovereignty where we are giving you full control and ownership of the location, encryption and access to your data. Operational sovereignty where the goal is to give our Google Cloud customers full visibility and control over the provider operations, right? So if there are any updates on hardware, software stack, any operations, there is full transparency, full visibility. And then the third pillar is around software sovereignty where the customer wants to ensure that they can run their workloads without dependency on the provider's software. So they have sometimes is often referred as survivability, that you can actually survive if you are untethered to the cloud and that you can use open source. Now let's take a deep dive on data sovereignty, which by the way is one of my favorite topics. And we typically focus on saying, hey, we need to care about data residency. We care where the data resides because where the data is at rest or in processing, it typically abides to the jurisdiction, the regulations of the jurisdiction where the data resides. And others say, hey, let's focus on data protection. We want to ensure the confidentiality and integrity and availability of the data, which confidential computing is at the heart of that data protection. But it is yet another element that people typically don't talk about when talking about data sovereignty, which is the element of user control. And here, Dave, is about what happens to the data when I give you access to my data. And this reminds me of security two decades ago, even a decade ago, where we started the security movement by putting firewall protections and login accesses. But once you were in, you were able to do everything you wanted with the data. An insider had access to all the infrastructure, the data and the code. And that's similar because with data sovereignty we care about whether it resides, where, who is operating on the data. But the moment that the data is being processed, I need to trust that the processing of the data will abide by user control, by the policies that I put in place of how my data is going to be used. And if you look at a lot of the regulation today and a lot of the initiatives around the International Data Space Association, IDSA, and Gaia-X, there is a movement of saying the two parties, the provider of the data and the receiver of the data are going to agree on a contract that describes what my data can be used for. The challenge is to ensure that once the data crosses boundaries, that the data will be used for the purposes that it was intended and specified in the contract. And if you actually bring together, and this is the exciting part, confidential computing together with policy enforcement, now the policy enforcement can guarantee that the data is only processed within the confines of a confidential computing environment, that the workload is cryptographically verified that there is the workload that was meant to process the data and that the data will be only used when abiding to the confidentiality and integrity safety of the confidential computing environment. And that's why we believe confidential computing is one necessary and essential technology that will allow us to ensure data sovereignty, especially when it comes to user control. >> Thank you for that. I mean it was a deep dive, I mean brief, but really detailed. So I appreciate that, especially the verification of the enforcement. Last question, I met you two because as part of my year end prediction post, you guys sent in some predictions and I wasn't able to get to them in the predictions post. So I'm thrilled that you were able to make the time to come on the program. How widespread do you think the adoption of confidential computing will be in 23 and what's the maturity curve look like, this decade in your opinion? Maybe each of you could give us a brief answer. >> So my prediction in five, seven years, as I started, it'll become utility. It'll become TLS as of, again, 10 years ago we couldn't believe that websites will have certificates and we will support encrypted traffic. Now we do and it's become ubiquity. It's exactly where confidential computing is getting and heading, I don't know we deserve yet. It'll take a few years of maturity for us, but we will be there. >> Thank you. And Patricia, what's your prediction? >> I will double that and say, hey, in the future, in the very near future, you will not be able to afford not having it. I believe as digital sovereignty becomes evermore top of mind with sovereign states and also for multi national organizations and for organizations that want to collaborate with each other, confidential computing will become the norm. It'll become the default, if I say, mode of operation. I like to compare that today is inconceivable. If we talk to the young technologists, it's inconceivable to think that at some point in history, and I happen to be alive that we had data at rest that was not encrypted, data in transit that was not encrypted, and I think that will be inconceivable at some point in the near future that to have unencrypted data while in use. >> And plus I think the beauty of the this industry is because there's so much competition, this essentially comes for free. I want to thank you both for spending some time on Breaking Analysis. There's so much more we could cover. I hope you'll come back to share the progress that you're making in this area and we can double click on some of these topics. Really appreciate your time. >> Anytime. >> Thank you so much. >> In summary, while confidential computing is being touted by the cloud players as a promising technology for enhancing data privacy and security, there are also those, as we said, who remain skeptical. The truth probably lies somewhere in between and it will depend on the specific implementation and the use case as to how effective confidential computing will be. Look, as with any new tech, it's important to carefully evaluate the potential benefits, the drawbacks, and make informed decisions based on the specific requirements in the situation and the constraints of each individual customer. But the bottom line is silicon manufacturers are working with cloud providers and other system companies to include confidential computing into their architectures. Competition, in our view, will moderate price hikes. And at the end of the day, this is under the covers technology that essentially will come for free. So we'll take it. I want to thank our guests today, Nelly and Patricia from Google, and thanks to Alex Myerson who's on production and manages the podcast. Ken Schiffman as well out of our Boston studio, Kristin Martin and Cheryl Knight help get the word out on social media and in our newsletters. And Rob Hof is our editor-in-chief over at siliconangle.com. Does some great editing for us, thank you all. Remember all these episodes are available as podcasts. Wherever you listen, just search Breaking Analysis podcast. I publish each week on wikibon.com and siliconangle.com where you can get all the news. If you want to get in touch, you can email me at david.vellante@siliconangle.com or dm me @DVellante. And you can also comment on my LinkedIn post. Definitely you want to check out etr.ai for the best survey data in the enterprise tech business. I know we didn't hit on a lot today, but there's some amazing data and it's always being updated, so check that out. This is Dave Vellante for theCUBE Insights, powered by ETR. Thanks for watching and we'll see you next time on Breaking Analysis. (upbeat music)

>> Welcome Nelly and Patricia, great to have you. >> Great to be here. >> Thank you so much for having us. >> You're very welcome. Nelly, why don't you start, and then Patricia you can weigh in. Just tell the audience a little bit about each of your roles at Google Cloud. >> So I'll start, I'm honing a lot of interesting activities in Google and again, security or infrastructure securities that I usually hone, and we're talking about encryption, Antware encryption, and confidential computing is a part of portfolio. In additional areas that I contribute to get with my team to Google and our customers is secure software supply chain. Because you need to trust your software. Is it operating your confidential environment to have end to end story about if you believe that your software and your environment doing what you expect, it's my role. >> Got it, okay. Patricia? >> Well I am a technical director in the office of the CTO, OCTO for short, in Google Cloud. And we are a global team. We include former CTOs like myself and senior technologies from large corporations, institutions, and a lot of success for startups as well. And we have two main goals. First, we work side by side with some of our largest, more strategic or most strategic customers and we help them solve complex engineering technical problems. And second, we are device Google and Google Cloud engineering and product management on emerging trends in technologies to guide the trajectory of our business. We are unique group, I think, because we have created this collaborative culture with our customers. And within OCTO I spend a lot of time collaborating with customers in the industry at large on technologies that can address privacy, security, and sovereignty of data in general. >> Excellent, thank you for that both of you. Let's get into it. So Nelly, what is confidential computing from Google's perspective? How do you define it? >> Confidential computing is a tool. And it's one of the tools in our toolbox. And confidential computing is a way how would help our customers to complete this very interesting end to end lifecycle of their data. And when customers bring in the data to Cloud and want to protect it, as they ingest it to the Cloud, they protect it address when they store data in the Cloud. But what was missing for many, many years is ability for us to continue protecting data and workloads of our customers when they running them. And again, because data is not brought to Cloud to have huge graveyard, we need to ensure that this data is actually indexed. Again there is some insights driven and drawn from this data. You have to process this data and confidential computing here to help. Now we have end to end protection of our customer's data when they bring the workloads and data to Cloud, thanks to confidential computing. >> Thank you for that. Okay, we're going to get into the architecture a bit but before we do Patricia, why do you think this topic of confidential computing is such an important technology? Can you explain, do you think it's transformative for customers and if so, why? >> Yeah, I would maybe like to use one thought, one way, one intuition behind why confidential matters. Because at the end of the day it reduces more and more the customers thrush boundaries and the attack surface, that's about reducing that periphery, the boundary, in which the customer needs to mind about trust and safety. And in a way is a natural progression that you're using encryption to secure and protect data in the same way that we are encrypting data in transit and at rest. Now we are also encrypting data while in use. And among other beneficial I would say one of the most transformative ones is that organizations will be able to collaborate with each other and retain the confidentiality of the data. And that is across industry. Even though it's highly focused on, I wouldn't say highly focused, but very beneficial for highly regulated industries. It applies to all of industries. And if you look at financing for example, where bankers are trying to detect fraud and specifically double finance where you are a customer is actually trying to get a finance on an asset, let's say a boat or a house and then it goes to another bank and gets another finance on that asset. Now bankers would be able to collaborate and detect fraud while preserving confidentiality and privacy of the of the data. >> Interesting, and I want to understand that a little bit more but I'm going to push you a little bit on this, Nelly, if I can, because there's a narrative out there that says confidential computing is a marketing ploy. I talked about this upfront, by Cloud providers that are just trying to placate people that are scared of the Cloud. And I'm presuming you don't agree with that but I'd like you to weigh in here. The argument is confidential computing is just memory encryption, it doesn't address many other problems, it is overhyped by Cloud providers. What do you say to that line of thinking? >> I absolutely disagree as you can imagine, it's a crazy statement. But the most importantly is we mixing multiple concepts I guess. And exactly as Patricia said, we need to look at the end-to-end story not again the mechanism of how confidential computing trying to again execute and protect customer's data, and why it's so critically important. Because what confidential computing was able to do it's in addition to isolate our tenants in multi-tenant environments the Cloud over. To offer additional stronger isolation, we called it cryptographic isolation. It's why customers will have more trust to customers and to other customers, the tenants that's running on the same host but also us, because they don't need to worry about against threats and more malicious attempts to penetrate the environment. So what confidential computing is helping us to offer our customers, stronger isolation between tenants in this multi-tenant environment but also incredibly important, stronger isolation of our customers. So tenants from us, we also writing code, we also software providers will also make mistakes or have some zero days sometimes again us introduced, sometimes introduced by our adversaries. But what I'm trying to say by creating this cryptographic layer of isolation between us and our tenants, and amongst those tenants, they're really providing meaningful security to our customers and eliminate some of the worries that they have running on multi-tenant spaces or even collaborating together this very sensitive data, knowing that this particular protection is available to them. >> Okay, thank you, appreciate that. And I, you know, I think malicious code is often a threat model missed in these narratives. You know, operator access, yeah, could maybe I trust my Clouds provider, but if I can fence off your access even better I'll sleep better at night. Separating a code from the data, everybody's arm Intel, AM, Invidia, others, they're all doing it. I wonder if Nell, if we could stay with you and bring up the slide on the architecture. What's architecturally different with confidential computing versus how operating systems and VMs have worked traditionally? We're showing a slide here with some VMs, maybe you could take us through that. >> Absolutely, and Dave, the whole idea for Google and industry way of dealing with confidential computing is to ensure as it's three main property is actually preserved. Customers don't need to change the code. They can operate in those VMs exactly as they would with normal non-confidential VMs. But to give them this opportunity of lift and shift or no changing their apps and performing and having very, very, very low latency and scale as any Cloud can, something that Google actually pioneered in confidential computing. I think we need to open and explain how this magic was actually done. And as I said, it's again the whole entire system have to change to be able to provide this magic. And I would start with we have this concept of root of trust and root of trust where we will ensure that this machine, the whole entire post has integrity guarantee, means nobody changing my code on the most low level of system. And we introduce this in 2017 code Titan. Those our specific ASIC specific, again inch by inch system on every single motherboard that we have, that ensures that your low level former, your actually system code, your kernel, the most powerful system, is actually proper configured and not changed, not tempered. We do it for everybody, confidential computing concluded. But for confidential computing what we have to change we bring in a MD again, future silicon vendors, and we have to trust their former, their way to deal with our confidential environments. And that's why we have obligation to validate integrity not only our software and our firmware but also firmware and software of our vendors, silicon vendors. So we actually, when we booting this machine as you can see, we validate that integrity of all of this system is in place. It means nobody touching, nobody changing, nobody modifying it. But then we have this concept of the secure processor. It's special Asics best, specific things that generate a key for every single VM that our customers will run or every single node in Kubernetes, or every single worker thread in our Spark capability. We offer all of that, and those keys are not available to us. It's the best keys ever in encryption space. Because when we are talking about encryption the first question that I'm receiving all the time, where's the key, who will have access to the key? Because if you have access to the key then it doesn't matter if you encrypt it enough. But the case in confidential computing quite so revolutionary technology, ask Cloud providers who don't have access to the keys. They're sitting in the hardware and they fed to memory controller. And it means when Hypervisors that also know about these wonderful things, saying I need to get access to the memories that this particular VM I'm trying to get access to. They do not encrypt the data, they don't have access to the key. Because those keys are random, ephemeral and VM, but the most importantly in hardware not exportable. And it means now you will be able to have this very interesting role that customers all Cloud providers, will not be able to get access to your memory. And what we do, again, as you can see our customers don't need to change their applications. Their VMs are running exactly as it should run. And what you're running in VM you actually see your memory in clear, it's not encrypted. But God forbid is trying somebody to do it outside of my confidential box. No, no, no, no, no, you will not be able to do it. Now you'll see cybernet. And it's exactly what combination of these multiple hardware pieces and software pieces have to do. So OS is also modified, and OS is modified such way to provide integrity. It means even OS that you're running in UVM bucks is not modifiable and you as customer can verify. But the most interesting thing I guess how to ensure the super performance of this environment because you can imagine, Dave, that's increasing it's additional performance, additional time, additional latency. So we're able to mitigate all of that by providing incredibly interesting capability in the OS itself. So our customers will get no changes needed, fantastic performance, and scales as they would expect from Cloud providers like Google. >> Okay, thank you. Excellent, appreciate that explanation. So you know again, the narrative on this is, well you know you've already given me guarantees as a Cloud provider that you don't have access to my data but this gives another level of assurance. Key management as they say is key. Now you're not, humans aren't managing the keys the machines are managing them. So Patricia, my question to you is in addition to, you know, let's go pre-confidential computing days what are the sort of new guarantees that these hardware-based technologies are going to provide to customers? >> So if I am a customer, I am saying I now have full guarantee of confidentiality and integrity of the data and of the code. So if you look at code and data confidentiality the customer cares then they want to know whether their systems are protected from outside or unauthorized access. And that we covered with Nelly that it is. Confidential computing actually ensures that the applications and data antennas remain secret, right? The code is actually looking at the data only the memory is decrypting the data with a key that is ephemeral, and per VM, and generated on demand. Then you have the second point where you have code and data integrity and now customers want to know whether their data was corrupted, tempered, with or impacted by outside actors. And what confidential computing insures is that application internals are not tampered with. So the application, the workload as we call it, that is processing the data it's also it has not been tempered and preserves integrity. I would also say that this is all verifiable. So you have attestation, and this attestation actually generates a log trail and the log trail guarantees that provides a proof that it was preserved. And I think that the offers also a guarantee of what we call ceiling, this idea that the secrets have been preserved and not tempered with. Confidentiality and integrity of code and data. >> Got it, okay, thank you. You know, Nelly, you mentioned, I think I heard you say that the applications, it's transparent,you don't have to change the application it just comes for free essentially. And I'm, we showed some various parts of the stack before. I'm curious as to what's affected but really more importantly what is specifically Google's value add? You know, how do partners, you know, participate in this? The ecosystem or maybe said another way how does Google ensure the compatibility of confidential computing with existing systems and applications? >> And a fantastic question by the way. And it's very difficult and definitely complicated world because to be able to provide these guarantees actually a lot of works was done by community. Google is very much operate and open. So again, our operating system we working in this operating system repository OS vendors to ensure that all capabilities that we need is part of their kernels, are part of their releases, and it's available for customers to understand and even explore if they have fun to explore a lot of code. We have also modified together with our silicon vendors, kernel, host kernel, to support this capability and it means working this community to ensure that all of those patches are there. We also worked with every single silicon vendor as you've seen, and that's what I probably feel that Google contributed quite a bit in this role. We moved our industry, our community, our vendors to understand the value of easy to use confidential computing or removing barriers. And now I don't know if you noticed Intel is pulling the lead and also announcing the trusted domain extension very similar architecture and no surprise, it's again a lot of work done with our partners to again, convince, work with them, and make this capability available. The same with ARM this year, actually last year, ARM unknowns are future design for confidential computing. It's called confidential computing architecture. And it's also influenced very heavily with similar ideas by Google and industry overall. So it's a lot of work in confidential computing consortiums that we are doing. For example, simply to mention to ensure interop, as you mentioned, between different confidential environments of Cloud providers. We want to ensure that they can attest to each other. Because when you're communicating with different environments, you need to trust them. And if it's running on different Cloud providers you need to ensure that you can trust your receiver when you are sharing your sensitive data workloads or secret with them. So we coming as a community and we have this at the station, the community based systems that we want to build and influence and work with ARM and every other Cloud providers to ensure that they can interrupt. And it means it doesn't matter where confidential workloads will be hosted but they can exchange the data in secure, verifiable, and controlled by customers way. And to do it, we need to continue what we are doing. Working open again and contribute with our ideas and ideas of our partners to this role to become what we see confidential computing has to become, it has to become utility. It doesn't need to be so special but it's what what we've wanted to become. >> Let's talk about, thank you for that explanation. Let talk about data sovereignty, because when you think about data sharing you think about data sharing across, you know, the ecosystem and different regions and then of course data sovereignty comes up. Typically public policy lags, you know, the technology industry and sometimes is problematic. I know, you know, there's a lot of discussions about exceptions, but Patricia, we have a graphic on data sovereignty. I'm interested in how confidential computing ensures that data sovereignty and privacy edicts are adhered to even if they're out of alignment maybe with the pace of technology. One of the frequent examples is when you you know, when you delete data, can you actually prove the data is deleted with a hundred percent certainty? You got to prove that and a lot of other issues. So looking at this slide, maybe you could take us through your thinking on data sovereignty. >> Perfect, so for us, data sovereignty is only one of the three pillars of digital sovereignty. And I don't want to give the impression that confidential computing addresses at all. That's why we want to step back and say, hey, digital sovereignty includes data sovereignty where we are giving you full control and ownership of the location, encryption, and access to your data. Operational sovereignty where the goal is to give our Google Cloud customers full visibility and control over the provider operations, right? So if there are any updates on hardware, software, stack, any operations, that is full transparency, full visibility. And then the third pillar is around software sovereignty where the customer wants to ensure that they can run their workloads without dependency on the provider's software. So they have sometimes is often referred as survivability that you can actually survive if you are untethered to the Cloud and that you can use open source. Now let's take a deep dive on data sovereignty, which by the way is one of my favorite topics. And we typically focus on saying, hey, we need to care about data residency. We care where the data resides because where the data is at rest or in processing it typically abides to the jurisdiction, the regulations of the jurisdiction where the data resides. And others say, hey, let's focus on data protection. We want to ensure the confidentiality and integrity and availability of the data which confidential computing is at the heart of that data protection. But it is yet another element that people typically don't talk about when talking about data sovereignty, which is the element of user control. And here Dave, is about what happens to the data when I give you access to my data. And this reminds me of security two decades ago, even a decade ago, where we started the security movement by putting firewall protections and login accesses. But once you were in, you were able to do everything you wanted with the data, an insider had access to all the infrastructure, the data, and the code. And that's similar because with data sovereignty we care about whether it resides, who is operating on the data. But the moment that the data is being processed, I need to trust that the processing of the data will abide by user control, by the policies that I put in place of how my data is going to be used. And if you look at a lot of the regulation today and a lot of the initiatives around the International Data Space Association, IDSA, and Gaia X, there is a movement of saying the two parties, the provider of the data and the receiver of the data going to agree on a contract that describes what my data can be used for. The challenge is to ensure that once the data crosses boundaries, that the data will be used for the purposes that it was intended and specified in the contract. And if you actually bring together, and this is the exciting part, confidential computing together with policy enforcement. Now the policy enforcement can guarantee that the data is only processed within the confines of a confidential computing environment. That the workload is cryptographically verified that there is the workload that was meant to process the data and that the data will be only used when abiding to the confidentiality and integrity, safety of the confidential computing environment. And that's why we believe confidential computing is one, necessary and essential technology that will allow us to ensure data sovereignty especially when it comes to user control. >> Thank you for that. I mean it was a deep dive, I mean brief, but really detailed, so I appreciate that, especially the verification of the enforcement. Last question, I met you two because as part of my year end prediction post you guys sent in some predictions, and I wasn't able to get to them in the predictions post. So I'm thrilled that you were able to make the time to come on the program. How widespread do you think the adoption of confidential computing will be in '23 and what's the maturity curve look like, you know, this decade in, in your opinion? Maybe each of you could give us a brief answer. >> So my prediction in five, seven years as I started, it'll become utility. It'll become TLS. As of, again, 10 years ago we couldn't believe that websites will have certificates and we will support encrypted traffic. Now we do, and it's become ubiquity. It's exactly where our confidential computing is heading and heading, I don't know if we are there yet yet. It'll take a few years of maturity for us, but we'll do that. >> Thank you, and Patricia, what's your prediction? >> I would double that and say, hey, in the future, in the very near future you will not be able to afford not having it. I believe as digital sovereignty becomes ever more top of mind with sovereign states and also for multinational organizations and for organizations that want to collaborate with each other, confidential computing will become the norm. It'll become the default, If I say mode of operation, I like to compare that, today is inconceivable if we talk to the young technologists. It's inconceivable to think that at some point in history and I happen to be alive that we had data at address that was not encrypted. Data in transit, that was not encrypted. And I think that we will be inconceivable at some point in the near future that to have unencrypted data while we use. >> You know, and plus, I think the beauty of the this industry is because there's so much competition this essentially comes for free. I want to thank you both for spending some time on Breaking Analysis. There's so much more we could cover. I hope you'll come back to share the progress that you're making in this area and we can double click on some of these topics. Really appreciate your time. >> Anytime. >> Thank you so much.

>> From theCUBE Studios in Palo Alto in Boston, bringing you data-driven insights from theCUBE and ETR. This is Breaking Analysis with Dave Vellante. >> Confidential computing is a technology that aims to enhance data privacy and security, by providing encrypted computation on sensitive data and isolating data, and apps that are fenced off enclave during processing. The concept of, I got to start over. I fucked that up, I'm sorry. That's not right, what I said was not right. On Dave in five, four, three. Confidential computing is a technology that aims to enhance data privacy and security by providing encrypted computation on sensitive data, isolating data from apps and a fenced off enclave during processing. The concept of confidential computing is gaining popularity, especially in the cloud computing space, where sensitive data is often stored and of course processed. However, there are some who view confidential computing as an unnecessary technology in a marketing ploy by cloud providers aimed at calming customers who are cloud phobic. Hello and welcome to this week's Wikibon Cube Insights powered by ETR. In this Breaking Analysis, we revisit the notion of confidential computing, and to do so, we'll invite two Google experts to the show. But before we get there, let's summarize briefly. There's not a ton of ETR data on the topic of confidential computing, I mean, it's a technology that's deeply embedded into silicon and computing architectures. But at the highest level, security remains the number one priority being addressed by IT decision makers in the coming year as shown here. And this data is pretty much across the board by industry, by region, by size of company. I mean we dug into it and the only slight deviation from the mean is in financial services. The second and third most cited priorities, cloud migration and analytics are noticeably closer to cybersecurity in financial services than in other sectors, likely because financial services has always been hyper security conscious, but security is still a clear number one priority in that sector. The idea behind confidential computing is to better address threat models for data in execution. Protecting data at rest and data in transit have long been a focus of security approaches, but more recently, silicon manufacturers have introduced architectures that separate data and applications from the host system, ARM, Intel, AMD, Nvidia and other suppliers are all on board, as are the big cloud players. Now, the argument against confidential computing is that it narrowly focuses on memory encryption and it doesn't solve the biggest problems in security. Multiple system images, updates, different services and the entire code flow aren't directly addressed by memory encryption. Rather to truly attack these problems, many believe that OSs need to be re-engineered with the attacker and hacker in mind. There are so many variables and at the end of the day, critics say the emphasis on confidential computing made by cloud providers is overstated and largely hype. This tweet from security researcher Rodrigo Bronco, sums up the sentiment of many skeptics. He says, "Confidential computing is mostly a marketing campaign from memory encryption. It's not driving the industry towards the hard open problems. It is selling an illusion." Okay. Nonetheless, encrypting data in use and fencing off key components of the system isn't a bad thing, especially if it comes with the package essentially for free. There has been a lack of standardization and interoperability between different confidential computing approaches. But the confidential computing consortium was established in 2019 ostensibly to accelerate the market and influence standards. Notably, AWS is not part of the consortium, likely because the politics of the consortium were probably a conundrum for AWS because the base technology defined by the consortium is seen as limiting by AWS. This is my guess, not AWS' words. But I think joining the consortium would validate a definition which AWS isn't aligned with. And two, it's got to lead with this Annapurna acquisition. It was way ahead with ARM integration, and so it's probably doesn't feel the need to validate its competitors. Anyway, one of the premier members of the confidential computing consortium is Google, along with many high profile names, including Aem, Intel, Meta, Red Hat, Microsoft, and others. And we're pleased to welcome two experts on confidential computing from Google to unpack the topic. Nelly Porter is Head of Product for GCP Confidential Computing and Encryption and Dr. Patricia Florissi is the Technical Director for the Office of the CTO at Google Cloud. Welcome Nelly and Patricia, great to have you. >> Great to be here. >> Thank you so much for having us. >> You're very welcome. Nelly, why don't you start and then Patricia, you can weigh in. Just tell the audience a little bit about each of your roles at Google Cloud. >> So I'll start, I'm owning a lot of interesting activities in Google and again, security or infrastructure securities that I usually own. And we are talking about encryption, end-to-end encryption, and confidential computing is a part of portfolio. Additional areas that I contribute to get with my team to Google and our customers is secure software supply chain because you need to trust your software. Is it operate in your confidential environment to have end-to-end security, about if you believe that your software and your environment doing what you expect, it's my role. >> Got it. Okay, Patricia? >> Well, I am a Technical Director in the Office of the CTO, OCTO for short in Google Cloud. And we are a global team, we include former CTOs like myself and senior technologies from large corporations, institutions and a lot of success for startups as well. And we have two main goals, first, we walk side by side with some of our largest, more strategic or most strategical customers and we help them solve complex engineering technical problems. And second, we advice Google and Google Cloud Engineering, product management on emerging trends and technologies to guide the trajectory of our business. We are unique group, I think, because we have created this collaborative culture with our customers. And within OCTO I spend a lot of time collaborating with customers in the industry at large on technologies that can address privacy, security, and sovereignty of data in general. >> Excellent. Thank you for that both of you. Let's get into it. So Nelly, what is confidential computing from Google's perspective? How do you define it? >> Confidential computing is a tool and one of the tools in our toolbox. And confidential computing is a way how we would help our customers to complete this very interesting end-to-end lifecycle of the data. And when customers bring in the data to cloud and want to protect it as they ingest it to the cloud, they protect it at rest when they store data in the cloud. But what was missing for many, many years is ability for us to continue protecting data and workloads of our customers when they run them. And again, because data is not brought to cloud to have huge graveyard, we need to ensure that this data is actually indexed. Again, there is some insights driven and drawn from this data. You have to process this data and confidential computing here to help. Now we have end-to-end protection of our customer's data when they bring the workloads and data to cloud thanks to confidential computing. >> Thank you for that. Okay, we're going to get into the architecture a bit, but before we do Patricia, why do you think this topic of confidential computing is such an important technology? Can you explain? Do you think it's transformative for customers and if so, why? >> Yeah, I would maybe like to use one thought, one way, one intuition behind why confidential computing matters because at the end of the day, it reduces more and more the customer's thrush boundaries and the attack surface. That's about reducing that periphery, the boundary in which the customer needs to mind about trust and safety. And in a way is a natural progression that you're using encryption to secure and protect data in the same way that we are encrypting data in transit and at rest. Now, we are also encrypting data while in the use. And among other beneficials, I would say one of the most transformative ones is that organizations will be able to collaborate with each other and retain the confidentiality of the data. And that is across industry, even though it's highly focused on, I wouldn't say highly focused but very beneficial for highly regulated industries, it applies to all of industries. And if you look at financing for example, where bankers are trying to detect fraud and specifically double finance where a customer is actually trying to get a finance on an asset, let's say a boat or a house, and then it goes to another bank and gets another finance on that asset. Now bankers would be able to collaborate and detect fraud while preserving confidentiality and privacy of the data. >> Interesting and I want to understand that a little bit more but I got to push you a little bit on this, Nellie if I can, because there's a narrative out there that says confidential computing is a marketing ploy I talked about this up front, by cloud providers that are just trying to placate people that are scared of the cloud. And I'm presuming you don't agree with that, but I'd like you to weigh in here. The argument is confidential computing is just memory encryption, it doesn't address many other problems. It is over hyped by cloud providers. What do you say to that line of thinking? >> I absolutely disagree as you can imagine Dave, with this statement. But the most importantly is we mixing a multiple concepts I guess, and exactly as Patricia said, we need to look at the end-to-end story, not again, is a mechanism. How confidential computing trying to execute and protect customer's data and why it's so critically important. Because what confidential computing was able to do, it's in addition to isolate our tenants in multi-tenant environments the cloud offering to offer additional stronger isolation, they called it cryptographic isolation. It's why customers will have more trust to customers and to other customers, the tenants running on the same host but also us because they don't need to worry about against rats and more malicious attempts to penetrate the environment. So what confidential computing is helping us to offer our customers stronger isolation between tenants in this multi-tenant environment, but also incredibly important, stronger isolation of our customers to tenants from us. We also writing code, we also software providers, we also make mistakes or have some zero days. Sometimes again us introduce, sometimes introduced by our adversaries. But what I'm trying to say by creating this cryptographic layer of isolation between us and our tenants and among those tenants, we really providing meaningful security to our customers and eliminate some of the worries that they have running on multi-tenant spaces or even collaborating together with very sensitive data knowing that this particular protection is available to them. >> Okay, thank you. Appreciate that. And I think malicious code is often a threat model missed in these narratives. You know, operator access. Yeah, maybe I trust my cloud's provider, but if I can fence off your access even better, I'll sleep better at night separating a code from the data. Everybody's ARM, Intel, AMD, Nvidia and others, they're all doing it. I wonder if Nell, if we could stay with you and bring up the slide on the architecture. What's architecturally different with confidential computing versus how operating systems and VMs have worked traditionally? We're showing a slide here with some VMs, maybe you could take us through that. >> Absolutely, and Dave, the whole idea for Google and now industry way of dealing with confidential computing is to ensure that three main property is actually preserved. Customers don't need to change the code. They can operate in those VMs exactly as they would with normal non-confidential VMs. But to give them this opportunity of lift and shift though, no changing the apps and performing and having very, very, very low latency and scale as any cloud can, some things that Google actually pioneer in confidential computing. I think we need to open and explain how this magic was actually done, and as I said, it's again the whole entire system have to change to be able to provide this magic. And I would start with we have this concept of root of trust and root of trust where we will ensure that this machine within the whole entire host has integrity guarantee, means nobody changing my code on the most low level of system, and we introduce this in 2017 called Titan. So our specific ASIC, specific inch by inch system on every single motherboard that we have that ensures that your low level former, your actually system code, your kernel, the most powerful system is actually proper configured and not changed, not tempered. We do it for everybody, confidential computing included, but for confidential computing is what we have to change, we bring in AMD or future silicon vendors and we have to trust their former, their way to deal with our confidential environments. And that's why we have obligation to validate intelligent not only our software and our former but also former and software of our vendors, silicon vendors. So we actually, when we booting this machine as you can see, we validate that integrity of all of this system is in place. It means nobody touching, nobody changing, nobody modifying it. But then we have this concept of AMD Secure Processor, it's special ASIC best specific things that generate a key for every single VM that our customers will run or every single node in Kubernetes or every single worker thread in our Hadoop spark capability. We offer all of that and those keys are not available to us. It's the best case ever in encryption space because when we are talking about encryption, the first question that I'm receiving all the time, "Where's the key? Who will have access to the key?" because if you have access to the key then it doesn't matter if you encrypted or not. So, but the case in confidential computing why it's so revolutionary technology, us cloud providers who don't have access to the keys, they're sitting in the hardware and they fed to memory controller. And it means when hypervisors that also know about this wonderful things saying I need to get access to the memories, that this particular VM I'm trying to get access to. They do not decrypt the data, they don't have access to the key because those keys are random, ephemeral and per VM, but most importantly in hardware not exportable. And it means now you will be able to have this very interesting world that customers or cloud providers will not be able to get access to your memory. And what we do, again as you can see, our customers don't need to change their applications. Their VMs are running exactly as it should run. And what you've running in VM, you actually see your memory clear, it's not encrypted. But God forbid is trying somebody to do it outside of my confidential box, no, no, no, no, no, you will now be able to do it. Now, you'll see cyber test and it's exactly what combination of these multiple hardware pieces and software pieces have to do. So OS is also modified and OS is modified such way to provide integrity. It means even OS that you're running in your VM box is not modifiable and you as customer can verify. But the most interesting thing I guess how to ensure the super performance of this environment because you can imagine Dave, that's increasing and it's additional performance, additional time, additional latency. So we're able to mitigate all of that by providing incredibly interesting capability in the OS itself. So our customers will get no changes needed, fantastic performance and scales as they would expect from cloud providers like Google. >> Okay, thank you. Excellent, appreciate that explanation. So you know again, the narrative on this is, well, you've already given me guarantees as a cloud provider that you don't have access to my data, but this gives another level of assurance, key management as they say is key. Now humans aren't managing the keys, the machines are managing them. So Patricia, my question to you is in addition to, let's go pre-confidential computing days, what are the sort of new guarantees that these hardware based technologies are going to provide to customers? >> So if I am a customer, I am saying I now have full guarantee of confidentiality and integrity of the data and of the code. So if you look at code and data confidentiality, the customer cares and they want to know whether their systems are protected from outside or unauthorized access, and that we covered with Nelly that it is. Confidential computing actually ensures that the applications and data antennas remain secret. The code is actually looking at the data, only the memory is decrypting the data with a key that is ephemeral, and per VM, and generated on demand. Then you have the second point where you have code and data integrity and now customers want to know whether their data was corrupted, tempered with or impacted by outside actors. And what confidential computing ensures is that application internals are not tempered with. So the application, the workload as we call it, that is processing the data is also has not been tempered and preserves integrity. I would also say that this is all verifiable, so you have attestation and this attestation actually generates a log trail and the log trail guarantees that provides a proof that it was preserved. And I think that the offers also a guarantee of what we call sealing, this idea that the secrets have been preserved and not tempered with, confidentiality and integrity of code and data. >> Got it. Okay, thank you. Nelly, you mentioned, I think I heard you say that the applications is transparent, you don't have to change the application, it just comes for free essentially. And we showed some various parts of the stack before, I'm curious as to what's affected, but really more importantly, what is specifically Google's value add? How do partners participate in this, the ecosystem or maybe said another way, how does Google ensure the compatibility of confidential computing with existing systems and applications? >> And a fantastic question by the way, and it's very difficult and definitely complicated world because to be able to provide these guarantees, actually a lot of work was done by community. Google is very much operate and open. So again our operating system, we working this operating system repository OS is OS vendors to ensure that all capabilities that we need is part of the kernels are part of the releases and it's available for customers to understand and even explore if they have fun to explore a lot of code. We have also modified together with our silicon vendors kernel, host kernel to support this capability and it means working this community to ensure that all of those pages are there. We also worked with every single silicon vendor as you've seen, and it's what I probably feel that Google contributed quite a bit in this world. We moved our industry, our community, our vendors to understand the value of easy to use confidential computing or removing barriers. And now I don't know if you noticed Intel is following the lead and also announcing a trusted domain extension, very similar architecture and no surprise, it's a lot of work done with our partners to convince work with them and make this capability available. The same with ARM this year, actually last year, ARM announced future design for confidential computing, it's called confidential computing architecture. And it's also influenced very heavily with similar ideas by Google and industry overall. So it's a lot of work in confidential computing consortiums that we are doing, for example, simply to mention, to ensure interop as you mentioned, between different confidential environments of cloud providers. They want to ensure that they can attest to each other because when you're communicating with different environments, you need to trust them. And if it's running on different cloud providers, you need to ensure that you can trust your receiver when you sharing your sensitive data workloads or secret with them. So we coming as a community and we have this at Station Sig, the community-based systems that we want to build, and influence, and work with ARM and every other cloud providers to ensure that they can interop. And it means it doesn't matter where confidential workloads will be hosted, but they can exchange the data in secure, verifiable and controlled by customers really. And to do it, we need to continue what we are doing, working open and contribute with our ideas and ideas of our partners to this role to become what we see confidential computing has to become, it has to become utility. It doesn't need to be so special, but it's what what we've wanted to become. >> Let's talk about, thank you for that explanation. Let's talk about data sovereignty because when you think about data sharing, you think about data sharing across the ecosystem in different regions and then of course data sovereignty comes up, typically public policy, lags, the technology industry and sometimes it's problematic. I know there's a lot of discussions about exceptions but Patricia, we have a graphic on data sovereignty. I'm interested in how confidential computing ensures that data sovereignty and privacy edicts are adhered to, even if they're out of alignment maybe with the pace of technology. One of the frequent examples is when you delete data, can you actually prove the data is deleted with a hundred percent certainty, you got to prove that and a lot of other issues. So looking at this slide, maybe you could take us through your thinking on data sovereignty. >> Perfect. So for us, data sovereignty is only one of the three pillars of digital sovereignty. And I don't want to give the impression that confidential computing addresses it at all, that's why we want to step back and say, hey, digital sovereignty includes data sovereignty where we are giving you full control and ownership of the location, encryption and access to your data. Operational sovereignty where the goal is to give our Google Cloud customers full visibility and control over the provider operations, right? So if there are any updates on hardware, software stack, any operations, there is full transparency, full visibility. And then the third pillar is around software sovereignty, where the customer wants to ensure that they can run their workloads without dependency on the provider's software. So they have sometimes is often referred as survivability that you can actually survive if you are untethered to the cloud and that you can use open source. Now, let's take a deep dive on data sovereignty, which by the way is one of my favorite topics. And we typically focus on saying, hey, we need to care about data residency. We care where the data resides because where the data is at rest or in processing need to typically abides to the jurisdiction, the regulations of the jurisdiction where the data resides. And others say, hey, let's focus on data protection, we want to ensure the confidentiality, and integrity, and availability of the data, which confidential computing is at the heart of that data protection. But it is yet another element that people typically don't talk about when talking about data sovereignty, which is the element of user control. And here Dave, is about what happens to the data when I give you access to my data, and this reminds me of security two decades ago, even a decade ago, where we started the security movement by putting firewall protections and logging accesses. But once you were in, you were able to do everything you wanted with the data. An insider had access to all the infrastructure, the data, and the code. And that's similar because with data sovereignty, we care about whether it resides, who is operating on the data, but the moment that the data is being processed, I need to trust that the processing of the data we abide by user's control, by the policies that I put in place of how my data is going to be used. And if you look at a lot of the regulation today and a lot of the initiatives around the International Data Space Association, IDSA and Gaia-X, there is a movement of saying the two parties, the provider of the data and the receiver of the data going to agree on a contract that describes what my data can be used for. The challenge is to ensure that once the data crosses boundaries, that the data will be used for the purposes that it was intended and specified in the contract. And if you actually bring together, and this is the exciting part, confidential computing together with policy enforcement. Now, the policy enforcement can guarantee that the data is only processed within the confines of a confidential computing environment, that the workload is in cryptographically verified that there is the workload that was meant to process the data and that the data will be only used when abiding to the confidentiality and integrity safety of the confidential computing environment. And that's why we believe confidential computing is one necessary and essential technology that will allow us to ensure data sovereignty, especially when it comes to user's control. >> Thank you for that. I mean it was a deep dive, I mean brief, but really detailed. So I appreciate that, especially the verification of the enforcement. Last question, I met you two because as part of my year-end prediction post, you guys sent in some predictions and I wasn't able to get to them in the predictions post, so I'm thrilled that you were able to make the time to come on the program. How widespread do you think the adoption of confidential computing will be in '23 and what's the maturity curve look like this decade in your opinion? Maybe each of you could give us a brief answer. >> So my prediction in five, seven years as I started, it will become utility, it will become TLS. As of freakin' 10 years ago, we couldn't believe that websites will have certificates and we will support encrypted traffic. Now we do, and it's become ubiquity. It's exactly where our confidential computing is heeding and heading, I don't know we deserve yet. It'll take a few years of maturity for us, but we'll do that. >> Thank you. And Patricia, what's your prediction? >> I would double that and say, hey, in the very near future, you will not be able to afford not having it. I believe as digital sovereignty becomes ever more top of mind with sovereign states and also for multinational organizations, and for organizations that want to collaborate with each other, confidential computing will become the norm, it will become the default, if I say mode of operation. I like to compare that today is inconceivable if we talk to the young technologists, it's inconceivable to think that at some point in history and I happen to be alive, that we had data at rest that was non-encrypted, data in transit that was not encrypted. And I think that we'll be inconceivable at some point in the near future that to have unencrypted data while we use. >> You know, and plus I think the beauty of the this industry is because there's so much competition, this essentially comes for free. I want to thank you both for spending some time on Breaking Analysis, there's so much more we could cover. I hope you'll come back to share the progress that you're making in this area and we can double click on some of these topics. Really appreciate your time. >> Anytime. >> Thank you so much, yeah. >> In summary, while confidential computing is being touted by the cloud players as a promising technology for enhancing data privacy and security, there are also those as we said, who remain skeptical. The truth probably lies somewhere in between and it will depend on the specific implementation and the use case as to how effective confidential computing will be. Look as with any new tech, it's important to carefully evaluate the potential benefits, the drawbacks, and make informed decisions based on the specific requirements in the situation and the constraints of each individual customer. But the bottom line is silicon manufacturers are working with cloud providers and other system companies to include confidential computing into their architectures. Competition in our view will moderate price hikes and at the end of the day, this is under-the-covers technology that essentially will come for free, so we'll take it. I want to thank our guests today, Nelly and Patricia from Google. And thanks to Alex Myerson who's on production and manages the podcast. Ken Schiffman as well out of our Boston studio. Kristin Martin and Cheryl Knight help get the word out on social media and in our newsletters, and Rob Hoof is our editor-in-chief over at siliconangle.com, does some great editing for us. Thank you all. Remember all these episodes are available as podcasts. Wherever you listen, just search Breaking Analysis podcast. I publish each week on wikibon.com and siliconangle.com where you can get all the news. If you want to get in touch, you can email me at david.vellante@siliconangle.com or DM me at D Vellante, and you can also comment on my LinkedIn post. Definitely you want to check out etr.ai for the best survey data in the enterprise tech business. I know we didn't hit on a lot today, but there's some amazing data and it's always being updated, so check that out. This is Dave Vellante for theCUBE Insights powered by ETR. Thanks for watching and we'll see you next time on Breaking Analysis. (subtle music)

>> From the Cube Studios in Palo Alto in Boston bringing you data-driven insights from the Cube and ETR. This is Breaking Analysis with Dave Vellante. >> The unraveling of market enthusiasm continued in Q4 of 2022 with the earnings reports from the US hyperscalers, the big three now all in. As we said earlier this year, even the cloud is an immune from the macro headwinds and the cracks in the armor that we saw from the data that we shared last summer, they're playing out into 2023. For the most part actuals are disappointing beyond expectations including our own. It turns out that our estimates for the big three hyperscaler's revenue missed by 1.2 billion or 2.7% lower than we had forecast from even our most recent November estimates. And we expect continued decelerating growth rates for the hyperscalers through the summer of 2023 and we don't think that's going to abate until comparisons get easier. Hello and welcome to this week's Wikibon Cube Insights powered by ETR. In this Breaking Analysis, we share our view of what's happening in cloud markets not just for the hyperscalers but other firms that have hitched a ride on the cloud. And we'll share new ETR data that shows why these trends are playing out tactics that customers are employing to deal with their cost challenges and how long the pain is likely to last. You know, riding the cloud wave, it's a two-edged sword. Let's look at the players that have gone all in on or are exposed to both the positive and negative trends of cloud. Look the cloud has been a huge tailwind for so many companies like Snowflake and Databricks, Workday, Salesforce, Mongo's move with Atlas, Red Hats Cloud strategy with OpenShift and so forth. And you know, the flip side is because cloud is elastic what comes up can also go down very easily. Here's an XY graphic from ETR that shows spending momentum or net score on the vertical axis and market presence in the dataset on the horizontal axis provision or called overlap. This is data from the January 2023 survey and that the red dotted lines show the positions of several companies that we've highlighted going back to January 2021. So let's unpack this for a bit starting with the big three hyperscalers. The first point is AWS and Azure continue to solidify their moat relative to Google Cloud platform. And we're going to get into this in a moment, but Azure and AWS revenues are five to six times that of GCP for IaaS. And at those deltas, Google should be gaining ground much faster than the big two. The second point on Google is notice the red line on GCP relative to its starting point. While it appears to be gaining ground on the horizontal axis, its net score is now below that of AWS and Azure in the survey. So despite its significantly smaller size it's just not keeping pace with the leaders in terms of market momentum. Now looking at AWS and Microsoft, what we see is basically AWS is holding serve. As we know both Google and Microsoft benefit from including SaaS in their cloud numbers. So the fact that AWS hasn't seen a huge downward momentum relative to a January 2021 position is one positive in the data. And both companies are well above that magic 40% line on the Y-axis, anything above 40% we consider to be highly elevated. But the fact remains that they're down as are most of the names on this chart. So let's take a closer look. I want to start with Snowflake and Databricks. Snowflake, as we reported from several quarters back came down to Earth, it was up in the 80% range in the Y-axis here. And it's still highly elevated in the 60% range and it continues to move to the right, which is positive but as we'll address in a moment it's customers can dial down consumption just as in any cloud. Now, Databricks is really interesting. It's not a public company, it never made it to IPO during the sort of tech bubble. So we don't have the same level of transparency that we do with other companies that did make it through. But look at how much more prominent it is on the X-axis relative to January 2021. And it's net score is basically held up over that period of time. So that's a real positive for Databricks. Next, look at Workday and Salesforce. They've held up relatively well, both inching to the right and generally holding their net scores. Same from Mongo, which is the brown dot above its name that says Elastic, it says a little gets a little crowded which Elastic's actually the blue dot above it. But generally, SaaS is harder to dial down, Workday, Salesforce, Oracles, SaaS and others. So it's harder to dial down because commitments have been made in advance, they're kind of locked in. Now, one of the discussions from last summer was as Mongo, less discretionary than analytics i.e. Snowflake. And it's an interesting debate but maybe Snowflake customers, you know, they're also generally committed to a dollar amount. So over time the spending is going to be there. But in the short term, yeah maybe Snowflake customers can dial down. Now that highlighted dotted red line, that bolded one is Datadog and you can see it's made major strides on the X-axis but its net score has decelerated quite dramatically. Openshift's momentum in the survey has dropped although IBM just announced that OpenShift has a a billion dollar ARR and I suspect what's happening there is IBM consulting is bundling OpenShift into its modernization projects. It's got a, that sort of captive base if you will. And as such it's probably not as top of mind to the respondents but I'll bet you the developers are certainly aware of it. Now the other really notable call out here is CloudFlare, We've reported on them earlier. Cloudflare's net score has held up really well since January of 2021. It really hasn't seen the downdraft of some of these others, but it's making major major moves to the right gaining market presence. We really like how CloudFlare is performing. And the last comment is on Oracle which as you can see, despite its much, much lower net score continues to gain ground in the market and thrive from a profitability standpoint. But the data pretty clearly shows that there's a downdraft in the market. Okay, so what's happening here? Let's dig deeper into this data. Here's a graphic from the most recent ETR drill down asking customers that said they were going to cut spending what technique they're using to do so. Now, as we've previously reported, consolidating redundant vendors is by far the most cited approach but there's two key points we want to make here. One is reducing excess cloud resources. As you can see in the bars is the second most cited technique and it's up from the previous polling period. The second we're not showing, you know directly but we've got some red call outs there. Reducing cloud costs jumps to 29% and 28% respectively in financial services and tech telco. And it's much closer to second. It's basically neck and neck with consolidating redundant vendors in those two industries. So they're being really aggressive about optimizing cloud cost. Okay, so as we said, cloud is great 'cause you can dial it up but it's just as easy to dial down. We've identified six factors that customers tell us are affecting their cloud consumption and there are probably more, if you got more we'd love to hear them but these are the ones that are fairly prominent that have hit our radar. First, rising mortgage rates mean banks are processing fewer loans means less cloud. The crypto crash means less trading activity and that means less cloud resources. Third lower ad spend has led companies to reduce not only you know, their ad buying but also their frequency of running their analytics and their calculations. And they're also often using less data, maybe compressing the timeframe of the corpus down to a shorter time period. Also very prominent is down to the bottom left, using lower cost compute instances. For example, Graviton from AWS or AMD chips and tiering storage to cheaper S3 or deep archived tiers. And finally, optimizing based on better pricing plans. So customers are moving from, you know, smaller companies in particular moving maybe from on demand or other larger companies that are experimenting using on demand or they're moving to spot pricing or reserved instances or optimized savings plans. That all lowers cost and that means less cloud resource consumption and less cloud revenue. Now in the days when everything was on prem CFOs, what would they do? They would freeze CapEx and IT Pros would have to try to do more with less and often that meant a lot of manual tasks. With the cloud it's much easier to move things around. It still takes some thinking and some effort but it's dramatically simpler to do so. So you can get those savings a lot faster. Now of course the other huge factor is you can cut or you can freeze. And this graphic shows data from a recent ETR survey with 159 respondents and you can see the meaningful uptick in hiring freezes, freezing new IT deployments and layoffs. And as we've been reporting, this has been trending up since earlier last year. And note the call out, this is especially prominent in retail sectors, all three of these techniques jump up in retail and that's a bit of a concern because oftentimes consumer spending helps the economy make a softer landing out of a pullback. But this is a potential canary in the coal mine. If retail firms are pulling back it's because consumers aren't spending as much. And so we're keeping a close eye on that. So let's boil this down to the market data and what this all means. So in this graphic we show our estimates for Q4 IaaS revenues compared to the "actual" IaaS revenues. And we say quote because AWS is the only one that reports, you know clean revenue and IaaS, Azure and GCP don't report actuals. Why would they? Because it would make them look even, you know smaller relative to AWS. Rather, they bury the figures in overall cloud which includes their, you know G-Suite for Google and all the Microsoft SaaS. And then they give us little tidbits about in Microsoft's case, Azure, they give growth rates. Google gives kind of relative growth of GCP. So, and we use survey data and you know, other data to try to really pinpoint and we've been covering this for, I don't know, five or six years ever since the cloud really became a thing. But looking at the data, we had AWS growing at 25% this quarter and it came in at 20%. So a significant decline relative to our expectations. AWS announced that it exited December, actually, sorry it's January data showed about a 15% mid-teens growth rate. So that's, you know, something we're watching. Azure was two points off our forecast coming in at 38% growth. It said it exited December in the 35% growth range and it said that it's expecting five points of deceleration off of that. So think 30% for Azure. GCP came in three points off our expectation coming in 35% and Alibaba has yet to report but we've shaved a bid off that forecast based on some survey data and you know what maybe 9% is even still not enough. Now for the year, the big four hyperscalers generated almost 160 billion of revenue, but that was 7 billion lower than what what we expected coming into 2022. For 2023, we're expecting 21% growth for a total of 193.3 billion. And while it's, you know, lower, you know, significantly lower than historical expectations it's still four to five times the overall spending forecast that we just shared with you in our predictions post of between 4 and 5% for the overall market. We think AWS is going to come in in around 93 billion this year with Azure closing in at over 71 billion. This is, again, we're talking IaaS here. Now, despite Amazon focusing investors on the fact that AWS's absolute dollar growth is still larger than its competitors. By our estimates Azure will come in at more than 75% of AWS's forecasted revenue. That's a significant milestone. AWS is operating margins by the way declined significantly this past quarter, dropping from 30% of revenue to 24%, 30% the year earlier to 24%. Now that's still extremely healthy and we've seen wild fluctuations like this before so I don't get too freaked out about that. But I'll say this, Microsoft has a marginal cost advantage relative to AWS because one, it has a captive cloud on which to run its massive software estate. So it can just throw software at its own cloud and two software marginal costs. Marginal economics despite AWS's awesomeness in high degrees of automation, software is just a better business. Now the upshot for AWS is the ecosystem. AWS is essentially in our view positioning very smartly as a platform for data partners like Snowflake and Databricks, security partners like CrowdStrike and Okta and Palo Alto and many others and SaaS companies. You know, Microsoft is more competitive even though AWS does have competitive products. Now of course Amazon's competitive to retail companies so that's another factor but generally speaking for tech players, Amazon is a really thriving ecosystem that is a secret weapon in our view. AWS happy to spin the meter with its partners even though it sells competitive products, you know, more so in our view than other cloud players. Microsoft, of course is, don't forget is hyping now, we're hearing a lot OpenAI and ChatGPT we reported last week in our predictions post. How OpenAI is shot up in terms of market sentiment in ETR's emerging technology company surveys and people are moving to Azure to get OpenAI and get ChatGPT that is a an interesting lever. Amazon in our view has to have a response. They have lots of AI and they're going to have to make some moves there. Meanwhile, Google is emphasizing itself as an AI first company. In fact, Google spent at least five minutes of continuous dialogue, nonstop on its AI chops during its latest earnings call. So that's an area that we're watching very closely as the buzz around large language models continues. All right, let's wrap up with some assumptions for 2023. We think SaaS players are going to continue to be sticky. They're going to be somewhat insulated from all these downdrafts because they're so tied in and customers, you know they make the commitment up front, you've got the lock in. Now having said that, we do expect some backlash over time on the onerous and generally customer unfriendly pricing models of most large SaaS companies. But that's going to play out over a longer period of time. Now for cloud generally and the hyperscalers specifically we do expect accelerating growth rates into Q3 but the amplitude of the demand swings from this rubber band economy, we expect to continue to compress and become more predictable throughout the year. Estimates are coming down, CEOs we think are going to be more cautious when the market snaps back more cautious about hiring and spending and as such a perhaps we expect a more orderly return to growth which we think will slightly accelerate in Q4 as comps get easier. Now of course the big risk to these scenarios is of course the economy, the FED, consumer spending, inflation, supply chain, energy prices, wars, geopolitics, China relations, you know, all the usual stuff. But as always with our partners at ETR and the Cube community, we're here for you. We have the data and we'll be the first to report when we see a change at the margin. Okay, that's a wrap for today. I want to thank Alex Morrison who's on production and manages the podcast, Ken Schiffman as well out of our Boston studio getting this up on LinkedIn Live. Thank you for that. Kristen Martin also and Cheryl Knight help get the word out on social media and in our newsletters. And Rob Hof is our Editor-in-Chief over at siliconangle.com. He does some great editing for us. Thank you all. Remember all these episodes are available as podcast. Wherever you listen, just search Breaking Analysis podcast. I publish each week on wikibon.com, at siliconangle.com where you can see all the data and you want to get in touch. Just all you can do is email me david.vellante@siliconangle.com or DM me @dvellante if you if you got something interesting, I'll respond. If you don't, it's either 'cause I'm swamped or it's just not tickling me. You can comment on our LinkedIn post as well. And please check out ETR.ai for the best survey data in the enterprise tech business. This is Dave Vellante for the Cube Insights powered by ETR. Thanks for watching and we'll see you next time on Breaking Analysis. (gentle upbeat music)

>>The cube presents Ignite 22, brought to you by Palo Alto Networks. >>Good morning. Live from the MGM Grand. It's the cube at Palo Alto Networks Ignite 2022. Lisa Martin here with Dave Valante, day two, Dave of our coverage, or last live day of the year, which I can't believe, lots of good news coming out from Palo Alto Networks. We're gonna sit down with its Chief product officer next and dissect all of that. >>Yeah. You know, oftentimes in, in events like this, day two is product day. And look, it's all about products and sales. Yeah, I mean those, that's the, the, the golden rule. Get the product right, get the sales right, and everything else will take care of itself. So let's talk product. >>Yeah, let's talk product. Lee Claridge joins us, the Chief Product Officer at Palo Alto Networks. Welcome Lee. Great to have >>You. Thank you so much. >>So we didn't get to see your keynote yesterday, but we heard one of the things, you know, we've been talking about the threat landscape, the challenges. We had Unit 42, Wendy on yesterday. We had Nash on and near talking about the massive challenges in the threat landscape. But we understand, despite that you are optimistic. I am. Talk about your optimism given the massive challenges that every organization is facing today. >>Look, cybersecurity's hard and often in cybersecurity in the industry, a lot of people get sort of really focused on what the threat actors are doing, why they're successful. We investigate breaches and we think of it, it just starts to feel somewhat overwhelming for a lot of folks. And I just happen to think a little bit differently. I, I look at it and I think it's actually a solvable problem. >>Talk about cyber resilience. How does Palo Alto Networks define that and how does it help customers achieve that? Cuz that's the, that's the holy grail these days. >>Yes. Look, the, the way I think about cyber resilience is basically in two pieces. One, it's all about how do we prevent the threat actors from actually being successful in the first place. Second, we also have to be prepared for what happens if they happen to find a way to get through, and how do we make sure that that happens? The blast radius is, is as narrowly contained as possible. And so the, the way that we approach this is, you know, I, I kind of think in terms of like threes three core principles. Number one, we have to have amazing technology and we have to constantly be, keep keeping up with and ideally ahead of what attackers are doing. It's a big part of my job as the chief product officer, right? Second is we, you know, one of the, the big transformations that's happened is the advent of, of AI and the opportunity, as long as we can do it, a great job of collecting great data, we can drive AI and machine learning models that can start to be used for our advantage as defenders, and then further use that to drive automation. >>So we take the human out of the response as much as possible. What that allows us to do is actually to start using AI and automation to disrupt attackers as it's happening. The third piece then becomes natively integrating these capabilities into a platform. And when we do that, what allows us to do is to make sure that we are consistently delivering cybersecurity everywhere that it needs to happen. That we don't have gaps. Yeah. So great tech AI and automation deliver natively integrated through platforms. This is how we achieve cyber resilience. >>So I like the positivity. In fact, Steven Schmidt, who's now the CSO of, of Amazon, you know, Steven, and it was the CSO at AWS at the time, the first reinforced, he stood up on stage and said, listen, this narrative that's all gloom and doom is not the right approach. We actually are doing a good job and we have the capability. So I was like, yeah, you know, okay. I'm, I'm down with that. Now when I, my question is around the, the portfolio. I, I was looking at, you know, some of your alternatives and options and the website. I mean, you got network security, cloud security, you got sassy, you got capp, you got endpoint, pretty much everything. You got cider security, which you just recently acquired for, you know, this whole shift left stuff, you know, nothing in there on identity yet. That's good. You partner for that, but, so could you describe sort of how you think about the portfolio from a product standpoint? How you continue to evolve it and what's the direction? Yes. >>So the, the, the cybersecurity industry has long had this, I'm gonna call it a major flaw. And the major flaw of the cybersecurity industry has been that every time there is a problem to be solved, there's another 10 or 20 startups that get funded to solve that problem. And so pretty soon what you have is you're, if you're a customer of this is you have 50, a hundred, the, the record is over 400 different cybersecurity products that as a customer you're trying to operationalize. >>It's not a good record to have. >>No, it's not a good record. No. This is, this is the opposite of Yes. Not a good personal best. So the, so the reason I start there in answering your question is the, the way that, so that's one end of the extreme, the other end of the extreme view to say, is there such a thing as a single platform that does everything? No, there's not. That would be nice. That was, that sounds nice. But the reality is that cybersecurity has to be much broader than any one single thing can do. And so the, the way that we approach this is, is three fundamental areas that, that we, Palo Alto Networks are going to be the best at. One is network security within network security. This includes hardware, NextGen, firewalls, software NextGen, firewalls, sassy, all the different security services that tie into that. All of that makes up our network security platforms. >>So everything to do with network security is integrated in that one place. Second is around cloud security. The shift to the cloud is happening is very real. That's where Prisma Cloud takes center stage. C a P is the industry acronym. If if five letters thrown together can be called an acronym. The, so cloud native application protection platform, right? So this is where we bring all of the different cloud security capabilities integrated together, delivered through one platform. And then security, security operations is the third for us. This is Cortex. And this is where we bring together endpoint security, edr, ndr, attack, surface management automation, all of this. And what we had, what we announced earlier this year is x Im, which is a Cortex product for actually integrating all of that together into one SOC transformation platform. So those are the three platforms, and that's how we deliver much, much, much greater levels of native integration of capabilities, but in a logical way where we're not trying to overdo it. >>And cider will fit into two or three >>Into Prisma cloud into the second cloud to two. Yeah. As part of the shift left strategy of how we secure makes sense applications in the cloud >>When you're in customer conversations. You mentioned the record of 400 different product. That's crazy. Nash was saying yesterday between 30 and 50 and we talked with him and near about what's realistic in terms of getting organizations to, to be able to consolidate. I'd love to understand what does cybersecurity transformation look like for the average organization that's running 30 to 50 point >>Solutions? Yeah, look, 30 to 50 is probably, maybe normal. A hundred is not unusual. Obviously 400 is the extreme example. But all of those are, those numbers are too big right now. I think, I think realistic is high. Single digits, low double digits is probably somewhat realistic for most organizations, the most complex organizations that might go a bit above that if we're really doing a good job. That's, that's what I think. Now second, I do really want to point out on, on the product guy. So, so maybe this is just my way of thinking, consolidation is an outcome of having more tightly and natively integrated capabilities. Got you. And the reason I flip that around is if I just went to you and say, Hey, would you like to consolidate? That just means maybe fewer vendors that that helps the procurement person. Yes. You know, have to negotiate with fewer companies. Yeah. Integration is actually a technology statement. It's delivering better outcomes because we've designed multiple capabilities to work together natively ourselves as the developers so that the customer doesn't have to figure out how to do it. It just happens that by, by doing that, the customer gets all this wonderful technical benefit. And then there's this outcome sitting there called, you've just consolidated your complexity. How >>Specialized is the customer? I think a data pipelines, and I think I have a data engineer, have a data scientists, a data analyst, but hyper specialized roles. If, if, let's say I have, you know, 30 or 40, and one of 'em is an SD wan, you know, security product. Yeah. I'm best of breed an SD wan. Okay, great. Palo Alto comes in as you, you pointed out, I'm gonna help you with your procurement side. Are there hyper specialized individuals that are aligned to that? And how that's kind of part A and B, how, assuming that's the case, how does that integration, you know, carry through to the business case? So >>Obviously there are specializations, this is the, and, and cybersecurity is really important. And so there, this is why there had, there's this tendency in the past to head toward, well I have this problem, so who's the best at solving this one problem? And if you only had one problem to solve, you would go find the specialist. The, the, the, the challenge becomes, well, what do you have a hundred problems to solve? I is the right answer, a hundred specialized solutions for your a hundred problems. And what what I think is missing in this approach is, is understanding that almost every problem that needs to be solved is interconnected with other problems to be solved. It's that interconnectedness of the problems where all of a sudden, so, so you mentioned SD wan. Okay, great. I have Estee wan, I need it. Well what are you connecting SD WAN to? >>Well, ideally our view is you would connect SD WAN and branch to the cloud. Well, would you run in the cloud? Well, in our case, we can take our SD wan, connect it to Prisma access, which is our cloud security solution, and we can natively integrate those two things together such that when you use 'em together, way easier. Right? All of a sudden we took what seemed like two separate problems. We said, no, actually these problems are related and we can deliver a solution where those, those things are actually brought together. And that's just one simple example, but you could, you could extend that across a lot of these other areas. And so that's the difference. And that's how the, the, the mindset shift that is happening. And, and I I was gonna say needs to happen, but it's starting to happen. I'm talking to customers where they're telling me this as opposed to me telling them. >>So when you walk around the floor here, there's a visual, it's called a day in the life of a fuel member. And basically what it has, it's got like, I dunno, six or seven different roles or personas, you know, one is management, one is a network engineer, one's a coder, and it gives you an X and an O. And it says, okay, put the X on things that you spend your time doing, put the o on things that you wanna spend your time doing a across all different sort of activities that a SecOps pro would do. There's Xs and O's in every one of 'em. You know, to your point, there's so much overlap going on. This was really difficult to discern, you know, any kind of consistent pattern because it, it, it, unlike the hyper specialization and data pipelines that I just described, it, it's, it's not, it, it, there's way more overlap between those, those specialization roles. >>And there's a, there's a second challenge that, that I've observed and that we are, we've, we've been trying to solve this and now I'd say we've become, started to become a lot more purposeful in, in, in trying to solve this, which is, I believe cybersecurity, in order for cyber security vendors to become partners, we actually have to start to become more opinionated. We actually have to start, guys >>Are pretty opinionated. >>Well, yes, but, but the industry large. So yes, we're opinionated. We build these products, but that have, that have our, I'll call our opinions built into it, and then we, we sell the, the product and then, and then what happens? Customer says, great, thank you for the product. I'm going to deploy it however I want to, which is fine. Obviously it's their choice at the end of the day, but we actually should start to exert an opinion to say, well, here's what we would recommend, here's why we would recommend that. Here's how we envisioned it providing the most value to you. And actually starting to build that into the products themselves so that they start to guide the customer toward these outcomes as opposed to just saying, here's a product, good luck. >>What's, what's the customer lifecycle, not lifecycle, but really kind of that, that collaboration, like it's one thing to, to have products that you're saying that have opinions to be able to inform customers how to deploy, how to use, but where is their feedback in this cycle of product development? >>Oh, look, my, this, this is, this is my life. I'm, this is, this is why I'm here. This is like, you know, all day long I'm meeting with customers and, and I share what we're doing. But, but it's, it's a, it's a 50 50, I'm half the time I'm listening as well to understand what they're trying to do, what they're trying to accomplish, and how, what they need us to do better in order to help them solve the problem. So the, the, and, and so my entire organization is oriented around not just telling customers, here's what we did, but listening and understanding and bringing that feedback in and constantly making the products better. That's, that's the, the main way in which we do this. Now there's a second way, which is we also allow our products to be customized. You know, I can say, here's our best practices, we see it, but then allowing our customer to, to customize that and tailor it to their environment, because there are going to be uniquenesses for different customers in parti, we need more complex environments. Explain >>Why fire firewalls won't go away >>From your perspective. Oh, Nikesh actually did a great job of explaining this yesterday, and although he gave me credit for it, so this is like a, a circular kind of reference here. But if you think about the firewalls slightly more abstract, and you basically say a NextGen firewalls job is to inspect every connection in order to make sure the connection should be allowed. And then if it is allowed to make sure that it's secure, >>Which that is the definition of an NextGen firewall, by the way, exactly what I just said. Now what you noticed is, I didn't describe it as a hardware device, right? It can be delivered in hardware because there are environments where you need super high throughput, low latency, guess what? Hardware is the best way of delivering that functionality. There's other use cases cloud where you can't, you, you can't ship hardware to a cloud provider and say, can you install this hardware in front of my cloud? No, no, no. You deployed in a software. So you take that same functionality, you instantly in a software, then you have other use cases, branch offices, remote workforce, et cetera, where you say, actually, I just want it delivered from the cloud. This is what sassy is. So when I, when I look at and say, the firewall's not going away, what, what, what I see is the functionality needed is not only not going away, it's actually expanding. But how we deliver it is going to be across these three form factors. And then the customer's going to decide how they need to intermix these form factors for their environment. >>We put forth this notion of super cloud a while about a year ago. And the idea being you're gonna leverage the hyperscale infrastructure and you're gonna build a, a, you're gonna solve a common problem across clouds and even on-prem, super cloud above the cloud. Not Superman, but super as in Latin. But it turned into this sort of, you know, superlative, which is fun. But the, my, my question to you is, is, is, is Palo Alto essentially building a common cross-cloud on-prem, presumably out to the edge consistent experience that we would call a super cloud? >>Yeah, I don't know that we've ever used the term surfer cloud to describe it. Oh, you don't have to, but yeah. But yes, based on how you describe it, absolutely. And it has three main benefits that I describe to customers all the time. The first is the end user experience. So imagine your employee, and you might work from the office, you might work from home, you might work while from, from traveling and hotels and conferences. And, and by the way, in one day you might actually work from all of those places. So, so the first part is the end user experience becomes way better when it doesn't matter where they're working from. They always get the same experience, huge benefit from productivity perspective, no second benefit security operations. You think about the, the people who are actually administering these policies and analyzing the security events. >>Imagine how much better it is for them when it's all common and consistent across everywhere that has to happen. Cloud, on-prem branch, remote workforce, et cetera. So there's a operational benefit that is super valuable. Third, security benefit. Imagine if in this, this platform-based approach, if we come out with some new amazing innovation that is able to detect and block, you know, new types of attacks, guess what, we can deliver that across hardware, software, and sassi uniformly and keep it all up to date. So from a security perspective, way better than trying to figure out, okay, there's some new technology, you know, does my hardware provider have that technology or not? Does my soft provider? So it's bringing that in to one place. >>From a developer perspective, is there a, a, a PAs layer, forgive me super PAs, that a allows the developers to have a common experience across irrespective of physical location with the explicit purpose of serving the objective of your platform. >>So normally when I think of the context of developers, I'm thinking of the context of, of the people who are building the applications that are being deployed. And those applications may be deployed in a data center, increasing the data centers, depending private clouds might be deployed into, into public cloud. It might even be hybrid in nature. And so if you think about what the developer wants, the developer actually wants to not have to think about security, quite frankly. Yeah. They want to think about how do I develop the functionality I need as quickly as possible with the highest quality >>Possible, but they are being forced to think about it more and more. Well, but anyway, I didn't mean to >>Interrupt you. No, it's a, it is a good, it's a, it's, it's a great point. The >>Well we're trying to do is we're trying to enable our security capabilities to work in a way that actually enables what the developer wants that actually allows them to develop faster that actually allows them to focus on the things they want to focus. And, and the way we do that is by actually surfacing the security information that they need to know in the tools that they use as opposed to trying to bring them to our tools. So you think about this, so our customer is a security customer. Yet in the application development lifecycle, the developer is often the user. So we, we we're selling, we're so providing a solution to security and then we're enabling them to surface it in the developer tools. And by, by doing this, we actually make life easier for the developers such that they're not actually thinking about security so much as they're just saying, oh, I pulled down the wrong open source package, it's outdated, it has vulnerabilities. I was notified the second I did it, and I was told which one I should pull down. So I pulled down the right one. Now, if you're a developer, do you think that's security getting your way? Not at all. No. If you're a developer, you're thinking, thank god, thank you, thank, thank you. Yeah. You told me at a point where it was easy as opposed to waiting a week or two and then telling me where it's gonna be really hard to fix it. Yeah. Nothing >>More than, so maybe be talking to Terraform or some other hash corp, you know, environment. I got it. Okay. >>Absolutely. >>We're 30 seconds. We're almost out of time. Sure. But I'd love to get your snapshot. Here we are at the end of calendar 2022. What are you, we know you're optimistic in this threat landscape, which we're gonna see obviously more dynamics next year. What kind of nuggets can you drop about what we might hear and see in 23? >>You're gonna see across everything. We do a lot more focus on the use of AI and machine learning to drive automated outcomes for our customers. And you're gonna see us across everything we do. And that's going to be the big transformation. It'll be a multi-year transformation, but you're gonna see significant progress in the next 12 months. All >>Right, well >>What will be the sign of that progress? If I had to make a prediction, which >>I'm better security with less effort. >>Okay, great. I feel like that's, we can measure that. I >>Feel, I feel like that's a mic drop moment. Lee, it's been great having you on the program. Thank you for walking us through such great detail. What's going on in the organization, what you're doing for customers, where you're meeting, how you're meeting the developers, where they are. We'll have to have you back. There's just, just too much to unpack. Thank you both so much. Actually, our pleasure for Lee Cler and Dave Valante. I'm Lisa Martin. You're watching The Cube Live from Palo Alto Networks Ignite 22, the Cube, the leader in live, emerging and enterprise tech coverage.

(cheerful music) >> Welcome to theCUBE's continuing coverage of AMD's 4th Generation EPYC launch. I'm Dave Nicholson, and I'm here in our Palo Alto studios talking to Greg Gibby, senior product manager, data center products from AMD, and Mohan Rokkam, technical marketing engineer at Dell. Welcome, gentlemen. >> Mohan: Hello, hello. >> Greg: Thank you. Glad to be here. >> Good to see each of you. Just really quickly, I want to start out. Let us know a little bit about yourselves. Mohan, let's start with you. What do you do at Dell exactly? >> So I'm a technical marketing engineer at Dell. I've been with Dell for around 15 years now and my goal is to really look at the Dell powered servers and see how do customers take advantage of some of the features we have, especially with the AMD EPYC processors that have just come out. >> Greg, and what do you do at AMD? >> Yeah, so I manage our software-defined infrastructure solutions team, and really it's a cradle to grave where we work with the ISVs in the market, so VMware, Nutanix, Microsoft, et cetera, to integrate the features that we're putting into our processors and make sure they're ready to go and enabled. And then we work with our valued partners like Dell on putting those into actual solutions that customers can buy and then we work with them to sell those solutions into the market. >> Before we get into the details on the 4th Generation EPYC launch and what that means and why people should care. Mohan, maybe you can tell us a little about the relationship between Dell and AMD, how that works, and then Greg, if you've got commentary on that afterwards, that'd be great. Yeah, Mohan. >> Absolutely. Dell and AMD have a long standing partnership, right? Especially now with EPYC series. We have had products since EPYC first generation. We have been doing solutions across the whole range of Dell ecosystem. We have integrated AMD quite thoroughly and effectively and we really love how performant these systems are. So, yeah. >> Dave: Greg, what are your thoughts? >> Yeah, I would say the other thing too is, is that we need to point out is that we both have really strong relationships across the entire ecosystem. So memory vendors, the software providers, et cetera, we have technical relationships. We're working with them to optimize solutions so that ultimately when the customer buys that, they get a great user experience right out of the box. >> So, Mohan, I know that you and your team do a lot of performance validation testing as time goes by. I suspect that you had early releases of the 4th Gen EPYC processor technology. What have you been seeing so far? What can you tell us? >> AMD has definitely knocked it out of the park. Time and again, in the past four generations, in the past five years alone, we have done some database work where in five years, we have seen five exit performance. And across the board, AMD is the leader in benchmarks. We have done virtualization where we would consolidate from five into one system. We have world records in AI, we have world records in databases, we have world records in virtualization. The AMD EPYC solutions has been absolutely performant. I'll leave you with one number here. When we went from top of Stack Milan to top of Stack Genoa, we saw a performance bump of 120%. And that number just blew my mind. >> So that prompts a question for Greg. Often we, in industry insiders, think in terms of performance gains over the last generation or the current generation. A lot of customers in the real world, however, are N - 2. They're a ways back, so I guess two points on that. First of all, the kinds of increases the average person is going to see when they move to this architecture, correct me if I'm wrong, but it's even more significant than a lot of the headline numbers because they're moving two generations, number one. Correct me if I'm wrong on that, but then the other thing is the question to you, Greg. I like very long complicated questions, as you can tell. The question is, is it okay for people to skip generations or make the case for upgrades, I guess is the problem? >> Well, yeah, so a couple thoughts on that first too. Mohan talked about that five X over the generation improvements that we've seen. The other key point with that too is that we've made significant process improvements along the way moving to seven nanocomputer to now five nanocomputer and that's really reducing the total amount of power or the performance per watt the customers can realize as well. And when we look at why would a customer want to upgrade, right? And I want to rephrase that as to why aren't you? And there is a real cost of not upgrading. And so when you look at infrastructure, the average age of a server in the data center is over five years old. And if you look at the most popular processors that were sold in that timeframe, it's 8, 10, 12 cores. So now you've got a bunch of servers that you need in order to deliver the applications and meet your SLAs to your end users, and all those servers pull power. They require maintenance. They have the opportunity to go down, et cetera. You got to pay licensing and service and support costs and all those. And when you look at all the costs that roll up, even though the hardware is paid for just to keep the lights on, and not even talking about the soft costs of unplanned downtime, and, "I'm not meeting your SLAs," et cetera, it's very expensive to keep those servers running. Now, if you refresh, and now you have processors that have 32, 64, 96 cores, now you can consolidate that infrastructure and reduce your total power bill. You can reduce your CapEx, you reduce your ongoing OpEx, you improve your performance, and you improve your security profile. So it really is more cost effective to refresh than not to refresh. >> So, Mohan, what has your experience been double clicking on this topic of consolidation? I know that we're going to talk about virtualization in some of the results that you've seen. What have you seen in that regard? Does this favor better consolidation and virtualized environments? And are you both assuring us that the ROI and TCO pencil out on these new big, bad machines? >> Greg definitely hit the nail on the head, right? We are seeing tremendous savings really, if you're consolidating from two generations old. We went from, as I said, five is to one. You're going from five full servers, probably paid off down to one single server. That itself is, if you look at licensing costs, which again, with things like VMware does get pretty expensive. If you move to a single system, yes, we are at 32, 64, 96 cores, but if you compare to the licensing costs of 10 cores, two sockets, that's still pretty significant, right? That's one huge thing. Another thing which actually really drives the thing is we are looking at security, and in today's environment, security becomes a major driving factor for upgrades. Dell has its own setups, cyber-resilient architecture, as we call it, and that really is integrated from processor all the way up into the OS. And those are some of the features which customers really can take advantage of and help protect their ecosystems. >> So what kinds of virtualized environments did you test? >> We have done virtualization across primary codes with VMware, but the Azure Stack, we have looked at Nutanix. PowerFlex is another one within Dell. We have vSAN Ready Nodes. All of these, OpenShift, we have a broad variety of solutions from Dell and AMD really fits into almost every one of them very well. >> So where does hyper-converged infrastructure fit into this puzzle? We can think of a server as something that contains not only AMD's latest architecture but also latest PCIe bus technology and all of the faster memory, faster storage cards, faster nicks, all of that comes together. But how does that play out in Dell's hyper-converged infrastructure or HCI strategy? >> Dell is a leader in hyper-converged infrastructure. We have the very popular VxRail line, we have the PowerFlex, which is now going into the AWS ecosystem as well, Nutanix, and of course, Azure Stack. With all these, when you look at AMD, we have up to 96 cores coming in. We have PCIe Gen 5 which means you can now connect dual port, 100 and 200 gig nicks and get line rate on those so you can connect to your ecosystem. And I don't know if you've seen the news, 200, 400 gig routers and switchers are selling out. That's not slowing down. The network infrastructure is booming. If you want to look at the AI/ML side of things, the VDI side of things, accelerator cards are becoming more and more powerful, more and more popular. And of course they need that higher end data path that PCIe Gen 5 brings to the table. GDDR5 is another huge improvement in terms of performance and latencies. So when we take all this together, you talk about hyper-converged, all of them add into making sure that A, with hyper-converged, you get ease of management, but B, just 'cause you have ease of management doesn't mean you need to compromise on anything. And the AMD servers effectively are a no compromise offering that we at Dell are able to offer to our customers. >> So Greg, I've got a question a little bit from left field for you. We covered Supercompute Conference 2022. We were in Dallas a couple of weeks ago, and there was a lot of discussion of the current processor manufacturer battles, and a lot of buzz around 4th Gen EPYC being launched and what's coming over the next year. Do you have any thoughts on what this architecture can deliver for us in terms of things like AI? We talk about virtualization, but if you look out over the next year, do you see this kind of architecture driving significant change in the world? >> Yeah, yeah, yeah, yeah. It has the real potential to do that from just the building blocks. So we have our chiplet architecture we call it. So you have an IO die and then you have your core complexes that go around that. And we integrate it all with our infinity fabric. That architecture allows you, if we wanted to, replace some of those CCDs with specific accelerators. And so when we look two, three, four years down the road, that architecture and that capability already built into what we're delivering and can easily be moved in. We just need to make sure that when you look at doing that, that the power that's required to do that and the software, et cetera, and those accelerators actually deliver better performance as a dedicated engine versus just using standard CPUs. The other things that I would say too is if you look at emerging workloads. So data center modernization is one of the buzzwords in cloud native, right? And these container environments, well, AMD'S architecture really just screams support for those type of environments, right? Where when you get into these larger core accounts and the consolidation that Mohan talked about. Now when I'm in a container environment, that blast radius so a lot of customers have concerns around, "Hey, having a single point of failure and having more than X number of cores concerns me." If I'm in containers, that becomes less of a concern. And so when you look at cloud native, containerized applications, data center modernization, AMD's extremely well positioned to take advantage of those use cases as well. >> Yeah, Mohan, and when we talk about virtualization, I think sometimes we have to remind everyone that yeah, we're talking about not only virtualization that has a full-blown operating system in the bucket, but also virtualization where the containers have microservices and things like that. I think you had something to add, Mohan. >> I did, and I think going back to the accelerator side of business, right? When we are looking at the current technology and looking at accelerators, AMD has done a fantastic job of adding in features like AVX-512, we have the bfloat16 and eight features. And some of what these do is they're effectively built-in accelerators for certain workloads especially in the AI and media spaces. And in some of these use cases we look at, for example, are inference. Traditionally we have used external accelerator cards, but for some of the entry level and mid-level use cases, CPU is going to work just fine especially with the newer CPUs that we are seeing this fantastic performance from. The accelerators just help get us to the point where if I'm at the edge, if I'm in certain use cases, I don't need to have an accelerator in there. I can run most of my inference workloads right on the CPU. >> Yeah, yeah. You know the game. It's an endless chase to find the bottleneck. And once we've solved the puzzle, we've created a bottleneck somewhere else. Back to the supercompute conversations we had, specifically about some of the AMD EPYC processor technology and the way that Dell is packaging it up and leveraging things like connectivity. That was one of the things that was also highlighted. This idea that increasingly connectivity is critically important, not just for supercomputing, but for high-performance computing that's finding its way out of the realms of Los Alamos and down to the enterprise level. Gentlemen, any more thoughts about the partnership or maybe a hint at what's coming in the future? I know that the original AMD announcement was announcing and previewing some things that are rolling out over the next several months. So let me just toss it to Greg. What are we going to see in 2023 in terms of rollouts that you can share with us? >> That I can share with you? Yeah, so I think look forward to see more advancements in the technology at the core level. I think we've already announced our product code name Bergamo, where we'll have up to 128 cores per socket. And then as we look in, how do we continually address this demand for data, this demand for, I need actionable insights immediately, look for us to continue to drive performance leadership in our products that are coming out and address specific workloads and accelerators where appropriate and where we see a growing market. >> Mohan, final thoughts. >> On the Dell side, of course, we have four very rich and configurable options with AMD EPYC servers. But beyond that, you'll see a lot more solutions. Some of what Greg has been talking about around the next generation of processors or the next updated processors, you'll start seeing some of those. and you'll definitely see more use cases from us and how customers can implement them and take advantage of the features that. It's just exciting stuff. >> Exciting stuff indeed. Gentlemen, we have a great year ahead of us. As we approach possibly the holiday seasons, I wish both of you well. Thank you for joining us. From here in the Palo Alto studios, again, Dave Nicholson here. Stay tuned for our continuing coverage of AMD's 4th Generation EPYC launch. Thanks for joining us. (cheerful music)

>>Welcome to the Cube's Continuing coverage of AMD's fourth generation Epic launch. I'm Dave Nicholson and I'm joining you here in our Palo Alto Studios. We have two very interesting guests to dive into some of the announcements that have been made and maybe take a look at this from an AI and ML perspective. Our first guest is Milland Doley. He's a senior director for software and solutions at amd, and we're also joined by Shamus Jones, who's a director of server engineering at Dell Technologies. Welcome gentlemen. How are you? >>Very good, thank >>You. Welcome to the Cube. So let's start out really quickly, Shamus, what, give us a thumbnail sketch of what you do at Dell. >>Yeah, so I'm the director of technical marketing engineering here at Dell, and our team really takes a look at the technical server portfolio and solutions and ensures that we can look at, you know, the performance metrics, benchmarks, and performance characteristics, so that way we can give customers a good idea of what they can expect from the server portfolio when they're looking to buy Power Edge from Dell. >>Milland, how about you? What's, what's new at a M D? What do you do there? >>Great to be here. Thank you for having me at amd, I'm the senior director of performance engineering and ISV ecosystem enablement, which is a long winter way of saying we do a lot of benchmarks, improved performance and demonstrate with wonderful partners such as Shamus and Dell, the combined leverage that AMD four generation processes and Dell systems can bring to bear on a multitude of applications across the industry spectrum. >>Shamus, talk about that relationship a little bit more. The relationship between a M D and Dell. How far back does it go? What does it look like in practical terms? >>Absolutely. So, you know, ever since AM MD reentered the server space, we've had a very close relationship. You know, it's one of those things where we are offering solutions that are out there to our customers no matter what generation A portfolio, if they're, if they're demanding either from their competitor or a m d, we offer a portfolio solutions that are out there. What we're finding is that within their generational improvements, they're just getting better and better and better. Really exciting things happening from a m D at the moment, and we're seeing that as we engineer those CPU stacks into our, our server portfolio, you know, we're really seeing unprecedented performance across the board. So excited about the, the history, you know, my team and Lin's team work very closely together, so much so that we were communicating almost on a daily basis around portfolio platforms and updates around the, the, the benchmarks testing and, and validation efforts. >>So Melind, are you happy with these PowerEdge boxes that Seamus is building to, to house, to house your baby? >>We are delighted, you know, it's hard to find stronger partners than Shamus and Dell with AMD's, second generation epic service CPUs. We already had undisputable industry performance leadership, and then with the third and now the fourth generation CPUs, we've just increased our lead with competition. We've got so many outstanding features at the platform, at the CPU level, everybody focuses on the high core counts, but there's also the DDR five, the memory, the io, and the storage subsystem. So we believe we have a fantastic performance and performance per dollar performance per what edge over competition, and we look to partners such as Dell to help us showcase that leadership. >>Well. So Shay Yeah, through Yeah, go ahead >>Dave. What, what I'd add, Dave, is that through the, the partnership that we've had, you know, we've been able to develop subsystems and platform features that historically we couldn't have really things around thermals power efficiency and, and efficiency within the platform. That means that customers can get the most out of their compute infrastructure. >>So this is gonna be a big question moving forward as next generation platforms are rolled out, there's the potential for people to have sticker shock. You talk about something that has eight or 12 cores in a, in a physical enclosure versus 96 cores, and, and I guess the, the question is, do the ROI and TCO numbers look good for someone to make that upgrade? Shamus, you wanna, you wanna hit that first or you guys are integrated? >>Absolutely, yeah, sorry. Absolutely. So we, I'll tell you what, at the moment, customers really can't afford not to upgrade at the moment, right? We've taken a look at the cost basis of keeping older infrastructure in place, let's say five or seven year old infrastructure servers that are, that are drawing more power maybe are, are poorly utilized within the infrastructure and take more and more effort and time to manage, maintain and, and really keep in production. So as customers look to upgrade or refresh their platforms, what we're finding right is that they can take a dynamic consolidation sometimes 5, 7, 8 to one consolidation depending on which platform they have as a historical and which one they're looking to upgrade to. Within AI specifically and machine learning frameworks, we're seeing really unprecedented performance. Lin's team partnered with us to deliver multiple benchmarks for the launch, some of which we're still continuing to see the goodness from things like TP C X AI as a framework, and I'm talking about here specifically the CPU U based performance. >>Even though in a lot of those AI frameworks, you would also expect to have GPUs, which all of the four platforms that we're offering on the AM MD portfolio today offer multiple G P U offerings. So we're seeing a balance between a huge amount of C P U gain and performance, as well as more and more GPU offerings within the platform. That was real, that was a real challenge for us because of the thermal challenges. I mean, you think GPUs are going up 300, 400 watt, these CPUs at 96 core are, are quite demanding thermally, but what we're able to do is through some, some unique smart cooling engineering within the, the PowerEdge portfolio, we can take a look at those platforms and make the most efficient use case by having things like telemetry within the platform so that way we can dynamically change fan speeds to get customers the best performance without throttling based on their need. >>Melin the cube was at the Supercomputing conference in Dallas this year, supercomputing conference 2022, and a lot of the discussion was around not only advances in microprocessor technology, but also advances in interconnect technology. How do you manage that sort of research partnership with Dell when you aren't strictly just focusing on the piece that you are bringing to the party? It's kind of a potluck, you know, we, we, we, we mentioned P C I E Gen five or 5.0, whatever you want to call it, new DDR storage cards, Nicks, accelerators, all of those, all of those things. How do you keep that straight when those aren't things that you actually build? >>Well, excellent question, Dave. And you know, as we are developing the next platform, obviously the, the ongoing relationship is there with Dell, but we start way before launch, right? Sometimes it's multiple years before launch. So we are not just focusing on the super high core counts at the CPU level and the platform configurations, whether it's single socket or dual socket, we are looking at it from the memory subsystem from the IO subsystem, P c i lanes for storage is a big deal, for example, in this generation. So it's really a holistic approach. And look, core counts are, you know, more important at the higher end for some customers h HPC space, some of the AI applications. But on the lower end you have database applications or some other is s v applications that care a lot about those. So it's, I guess different things matter to different folks across verticals. >>So we partnered with Dell very early in the cycle, and it's really a joint co-engineering. Shamus talked about the focus on AI with TP C X xci, I, so we set five world records in that space just on that one benchmark with AD and Dell. So fantastic kick kick off to that across a multitude of scale factors. But PPP c Xci is not just the only thing we are focusing on. We are also collaborating with Dell and des e i on some of the transformer based natural language processing models that we worked on, for example. So it's not just a steep CPU story, it's CPU platform, es subsystem software and the whole thing delivering goodness across the board to solve end user problems in AI and and other verticals. >>Yeah, the two of you are at the tip of the spear from a performance perspective. So I know it's easy to get excited about world records and, and they're, they're fantastic. I know Shamus, you know, that, you know, end user customers might, might immediately have the reaction, well, I don't need a Ferrari in my data center, or, you know, what I need is to be able to do more with less. Well, aren't we delivering that also? And you know, you imagine you milland you mentioned natural, natural language processing. Shamus, are you thinking in 2023 that a lot more enterprises are gonna be able to afford to do things like that? I mean, what are you hearing from customers on this front? >>I mean, while the adoption of the top bin CPU stack is, is definitely the exception, not the rule today we are seeing marked performance, even when we look at the mid bin CPU offerings from from a m d, those are, you know, the most common sold SKUs. And when we look at customers implementations, really what we're seeing is the fact that they're trying to make the most, not just of dollar spend, but also the whole subsystem that Melin was talking about. You know, the fact that balanced memory configs can give you marked performance improvements, not just at the CPU level, but as actually all the way through to the, to the application performance. So it's, it's trying to find the correct balance between the application needs, your budget, power draw and infrastructure within the, the data center, right? Because not only could you, you could be purchasing and, and look to deploy the most powerful systems, but if you don't have an infrastructure that's, that's got the right power, right, that's a large challenge that's happening right now and the right cooling to deal with the thermal differences of the systems, might you wanna ensure that, that you can accommodate those for not just today but in the future, right? >>So it's, it's planning that balance. >>If I may just add onto that, right? So when we launched, not just the fourth generation, but any generation in the past, there's a natural tendency to zero in on the top bin and say, wow, we've got so many cores. But as Shamus correctly said, it's not just that one core count opn, it's, it's the whole stack. And we believe with our four gen CPU processor stack, we've simplified things so much. We don't have, you know, dozens and dozens of offerings. We have a fairly simple skew stack, but we also have a very efficient skew stack. So even, even though at the top end we've got 96 scores, the thermal budget that we require is fairly reasonable. And look, with all the energy crisis going around, especially in Europe, this is a big deal. Not only do customers want performance, but they're also super focused on performance per want. And so we believe with this generation, we really delivered not just on raw performance, but also on performance per dollar and performance per one. >>Yeah. And it's not just Europe, I'm, we're, we are here in Palo Alto right now, which is in California where we all know the cost of an individual kilowatt hour of electricity because it's quite, because it's quite high. So, so thermals, power cooling, all of that, all of that goes together and that, and that drives cost. So it's a question of how much can you get done per dollar shame as you made the point that you, you're not, you don't just have a one size fits all solution that it's, that it's fit for function. I, I'm, I'm curious to hear from you from the two of you what your thoughts are from a, from a general AI and ML perspective. We're starting to see right now, if you hang out on any kind of social media, the rise of these experimental AI programs that are being presented to the public, some will write stories for you based on prom, some will create images for you. One of the more popular ones will create sort of a, your superhero alter ego for, I, I can't wait to do it, I just got the app on my phone. So those are all fun and they're trivial, but they sort of get us used to this idea that, wow, these systems can do things. They can think on their own in a certain way. W what do, what do you see the future of that looking like over the next year in terms of enterprises, what they're going to do for it with it >>Melan? Yeah, I can go first. Yeah, yeah, yeah, yeah, >>Sure. Yeah. Good. >>So the couple of examples, Dave, that you mentioned are, I, I guess it's a blend of novelty and curiosity. You know, people using AI to write stories or poems or, you know, even carve out little jokes, check grammar and spelling very useful, but still, you know, kind of in the realm of novelty in the mainstream, in the enterprise. Look, in my opinion, AI is not just gonna be a vertical, it's gonna be a horizontal capability. We are seeing AI deployed across the board once the models have been suitably trained for disparate functions ranging from fraud detection or anomaly detection, both in the financial markets in manufacturing to things like image classification or object detection that you talked about in, in the sort of a core AI space itself, right? So we don't think of AI necessarily as a vertical, although we are showcasing it with a specific benchmark for launch, but we really look at AI emerging as a horizontal capability and frankly, companies that don't adopt AI on a massive scale run the risk of being left behind. >>Yeah, absolutely. There's an, an AI as an outcome is really something that companies, I, I think of it in the fact that they're adopting that and the frameworks that you're now seeing as the novelty pieces that Melin was talking about is, is really indicative of the under the covers activity that's been happening within infrastructures and within enterprises for the past, let's say 5, 6, 7 years, right? The fact that you have object detection within manufacturing to be able to, to be able to do defect detection within manufacturing lines. Now that can be done on edge platforms all the way at the device. So you're no longer only having to have things be done, you know, in the data center, you can bring it right out to the edge and have that high performance, you know, inferencing training models. Now, not necessarily training at the edge, but the inferencing models especially, so that way you can, you know, have more and, and better use cases for some of these, these instances things like, you know, smart cities with, with video detection. >>So that way they can see, especially during covid, we saw a lot of hospitals and a lot of customers that were using using image and, and spatial detection within their, their video feeds to be able to determine who and what employees were at risk during covid. So there's a lot of different use cases that have been coming around. I think the novelty aspect of it is really interesting and I, I know my kids, my daughters love that, that portion of it, but really what's been happening has been exciting for quite a, quite a period of time in the enterprise space. We're just now starting to actually see those come to light in more of a, a consumer relevant kind of use case. So the technology that's been developed in the data center around all of these different use cases is now starting to feed in because we do have more powerful compute at our fingertips. We do have the ability to talk more about the framework and infrastructure that's that's right out at the edge. You know, I know Dave in the past you've said things like the data center of, you know, 20 years ago is now in my hand as, as my cell phone. That's right. And, and that's, that's a fact and I'm, it's exciting to think where it's gonna be in the next 10 or 20 years. >>One terabyte baby. Yeah. One terabyte. Yeah. It's mind bo. Exactly. It's mind boggling. Yeah. And it makes me feel old. >>Yeah, >>Me too. And, and that and, and Shamus, that all sounded great. A all I want is a picture of me as a superhero though, so you guys are already way ahead of the curve, you know, with, with, with that on that note, Seamus wrap us up with, with a, with kind of a summary of the, the highlights of what we just went through in terms of the performance you're seeing out of this latest gen architecture from a md. >>Absolutely. So within the TPC xai frameworks that Melin and my team have worked together to do, you know, we're seeing unprecedented price performance. So the fact that you can get 220% uplift gen on gen for some of these benchmarks and, you know, you can have a five to one consolidation means that if you're looking to refresh platforms that are historically legacy, you can get a, a huge amount of benefit, both in reduction in the number of units that you need to deploy and the, the amount of performance that you can get per unit. You know, Melinda had mentioned earlier around CPU performance and performance per wat, specifically on the Tu socket two U platform using the fourth generation a m d Epic, you know, we're seeing a 55% higher C P U performance per wat that is that, you know, when for people who aren't necessarily looking at these statistics, every generation of servers, that that's, that is a huge jump leap forward. >>That combined with 121% higher spec scores, you know, as a benchmark, those are huge. Normally we see, let's say a 40 to 60% performance improvement on the spec benchmarks, we're seeing 121%. So while that's really impressive at the top bin, we're actually seeing, you know, large percentile improvements across the mid bins as well, you know, things in the range of like 70 to 90% performance improvements in those standard bins. So it, it's a, it's a huge performance improvement, a power efficiency, which means customers are able to save energy, space and time based on, on their deployment size. >>Thanks for that Shamus, sadly, gentlemen, our time has expired. With that, I want to thank both of you. It's a very interesting conversation. Thanks for, thanks for being with us, both of you. Thanks for joining us here on the Cube for our coverage of AMD's fourth generation Epic launch. Additional information, including white papers and benchmarks plus editorial coverage can be found on does hardware matter.com.

(upbeat music) >> Welcome to theCUBE's continuing coverage of AMD's fourth generation EPYC launch. I've got a special guest with me today from Prowess Consulting. His name is Evan Touger, he's a senior technical writer with Prowess. Evan, welcome. >> Hi, great to be here. Thanks. >> So tell us a little bit about Prowess, what does Prowess do? >> Yeah, we're a consulting firm. We've been around for quite a few years, based in Bellevue, Washington. And we do quite a few projects with folks from Dell to a lot of other companies, and dive in. We have engineers, writers, production folks, so pretty much end-to-end work, doing research testing and writing, and diving into different technical topics. >> So you- in this case what we're going to be talking about is some validation studies that you've done, looking at Dell PowerEdge servers that happened to be integrating in fourth-gen EPYC processors from AMD. What were the specific workloads that you were focused on in this study? >> Yeah, this particular one was honing in on virtualization, right? You know, obviously it's pretty much ubiquitous in the industry, everybody works with virtualization in one way or another. So just getting optimal performance for virtualization was critical, or is critical for most businesses. So we just wanted to look a little deeper into, you know, how do companies evaluate that? What are they going to use to make the determination for virtualization performance as it relates to their workloads? So that led us to this study, where we looked at some benchmarks, and then went a little deeper under the hood to see what led to the results that we saw from those benchmarks. >> So when you say virtualization, does that include virtual desktop infrastructure or are we just talking about virtual machines in general? >> No, it can include both. We looked at VMs, thinking in terms of what about database performance when you're working in VMs, all the way through to VDI and companies like healthcare organizations and so forth, where it's common to roll out lots of virtual desktops, and performance is critical there as well. >> Okay, you alluded to, sort of, looking under the covers to see, you know, where these performance results were coming from. I assume what you're referencing is the idea that it's not just all about the CPU when you talk about a system. Am I correct in that assumption and- >> Yeah, absolutely. >> What can you tell us? >> Well, you know, for companies evaluating, there's quite a bit to consider, obviously. So they're looking at not just raw performance but power performance. So that was part of it, and then what makes up that- those factors, right? So certainly CPU is critical to that, but then other things come into play, like the RAID controllers. So we looked a little bit there. And then networking, of course can be critical for configurations that are relying on good performance on their networks, both in terms of bandwidth and just reducing latency overall. So interconnects as well would be a big part of that. So with, with PCIe gen 5 or 5.0 pick your moniker. You know in this- in the infrastructure game, we're often playing a game of whack-a-mole, looking for the bottlenecks, you know, chasing the bottlenecks. PCIe 5 opens up a lot of bandwidth for memory and things like RAID controllers and NICs. I mean, is the bottleneck now just our imagination, Evan, have we reached a point where there are no bottlenecks? What did you see when you ran these tests? What, you know, what were you able to stress to a point where it was saturated, if anything? >> Yeah. Well, first of all, we didn't- these are particular tests were ones that we looked at industry benchmarks, and we were examining in particular to see where world records were set. And so we uncovered a few specific servers, PowerEdge servers that were pretty key there, or had a lot of- were leading in the category in a lot of areas. So that's what led us to then, okay, well why is that? What's in these servers, and what's responsible for that? So in a lot of cases they, we saw these results even with, you know, gen 4, PCIe gen 4. So there were situations where clearly there was benefit from faster interconnects and, and especially NVMe for RAID, you know, for supporting NVMe and SSDs. But all of that just leads you to the understanding that it means it can only get better, right? So going from gen 4 to- if you're seeing great results on gen 4, then gen 5 is probably going to be, you know, blow that away. >> And in this case, >> It'll be even better. >> In this case, gen 5 you're referencing PCIe >> PCIe right. Yeah, that's right. >> (indistinct) >> And then the same thing with EPYC actually holds true, some of the records, we saw records set for both 3rd and 4th gen, so- with EPYC, so the same thing there. Anywhere there's a record set on the 3rd gen, you know, makes us really- we're really looking forward to going back and seeing over the next few months, which of those records fall and are broken by newer generation versions of these servers, once they actually wrap to the newer generation processors. You know, based on, on what we're seeing for the- for what those processors can do, not only in. >> (indistinct) Go ahead. >> Sorry, just want to say, not only in terms of raw performance, but as I mentioned before, the power performance, 'cause they're very efficient, and that's a really critical consideration, right? I don't think you can overstate that for companies who are looking at, you know, have to consider expenditures and power and cooling and meeting sustainability goals and so forth. So that was really an important category in terms of what we looked at, was that power performance, not just raw performance. >> Yeah, I want to get back to that, that's a really good point. We should probably give credit where credit is due. Which Dell PowerEdge servers are we talking about that were tested and what did those interconnect components look like from a (indistinct) perspective? >> Yeah, so we focused primarily on a couple benchmarks that seemed most important for real world performance results for virtualization. TPCx-V and VMmark 3.x. the TPCx-V, that's where we saw PowerEdge R7525, R7515. They both had top scores in different categories there. That benchmark is great for looking at database workloads in particular, right? Running in virtualization settings. And then the VMmark 3.x was critical. We saw good, good results there for the 7525 and the R 7515 as well as the R 6525, in that one and that included, sorry, just checking notes to see what- >> Yeah, no, no, no, no, (indistinct) >> Included results for power performance, as I mentioned earlier, that's where we could see that. So we kind of, we saw this in a range of servers that included both 3rd gen AMD EPYC and newer 4th gen as well as I mentioned. The RAID controllers were critical in the TPCx-V. I don't think that came into play in the VM mark test, but they were definitely part of the TPCx-V benchmarks. So that's where the RAID controllers would make a difference, right? And in those tests, I think they're using PERC 11. So, you know, the newer PERC 12 controllers there, again we'd expect >> (indistinct) >> To see continued, you know, gains in newer benchmarks. That's what we'll be looking for over the next several months. >> Yeah. So I think if I've got my Dell nomenclature down, performance, no no, PowerEdge RAID Controller, is that right? >> Exactly, yeah, there you go. Right? >> With Broadcom, you know, powered by Broadcom. >> That's right. There you go. Yeah. Isn't the Dell naming scheme there PERC? >> Yeah, exactly, exactly. Back to your comment about power. So you've had a chance to take a pretty deep look at the latest stuff coming out. You're confident that- 'cause some of these servers are going to be more expensive than previous generation. Now a server is not a server is not a server, but some are awakening to the idea that there might be some sticker shock. You're confident that the bang for your buck, the bang for your kilowatt hour is actually going to be beneficial. We're actually making things better, faster, stronger, cheaper, more energy efficient. We're continuing on that curve? >> That's what I would expect to see, right. I mean, of course can't speak to to pricing without knowing, you know, where the dollars are going to land on the servers. But I would expect to see that because you're getting gains in a couple of ways. I mean, one, if the performance increases to the point where you can run more VMs, right? Get more performance out of your VMs and run more total VMs or more BDIs, then there's obviously a good, you know, payback on your investment there. And then as we were discussing earlier, just the power performance ratio, right? So if you're bringing down your power and cooling costs, if these machines are just more efficient overall, then you should see some gains there as well. So, you know, I think the key is looking at what's the total cost of ownership over, you know, a standard like a three-year period or something and what you're going to get out of it for your number of sessions, the performance for the sessions, and the overall efficiency of the machines. >> So just just to be clear with these Dell PowerEdge servers, you were able to validate world record performance. But this isn't, if you, if you look at CPU architecture, PCIe bus architecture, memory, you know, the class of memory, the class of RAID controller, the class of NIC. Those were not all state of the art in terms of at least what has been recently announced. Correct? >> Right. >> Because (indistinct) the PCI 4.0, So to your point- world records with that, you've got next-gen RAID controllers coming out, and NICs coming out. If the motherboard was PCIe 5, with commensurate memory, all of those things are getting better. >> Exactly, right. I mean you're, you're really you're just eliminating bandwidth constraints latency constraints, you know, all of that should be improved. NVMe, you know, just collectively all these things just open the doors, you know, letting more bandwidth through reducing all the latency. Those are, those are all pieces of the puzzle, right? That come together and it's all about finding the weakest link and eliminating it. And I think we're reaching the point where we're removing the biggest constraints from the systems. >> Okay. So I guess is it fair to summarize to say that with this infrastructure that you tested, you were able to set world records. This, during this year, I mean, over the next several months, things are just going to get faster and faster and faster and faster. >> That's what I would anticipate, exactly, right. If they're setting world records with these machines before some of the components are, you know, the absolute latest, it seems to me we're going to just see a continuing trend there, and more and more records should fall. So I'm really looking forward to seeing how that goes, 'cause it's already good and I think the return on investment is pretty good there. So I think it's only going to get better as these roll out. >> So let me ask you a question that's a little bit off topic. >> Okay. >> Kind of, you know, we see these gains, you know, we're all familiar with Moore's Law, we're familiar with, you know, the advancements in memory and bus architecture and everything else. We just covered SuperCompute 2022 in Dallas a couple of weeks ago. And it was fascinating talking to people about advances in AI that will be possible with new architectures. You know, most of these supercomputers that are running right now are n minus 1 or n minus 2 infrastructure, you know, they're, they're, they're PCI 3, right. And maybe two generations of processors old, because you don't just throw out a 100,000 CPU super computing environment every 18 months. It doesn't work that way. >> Exactly. >> Do you have an opinion on this question of the qualitative versus quantitative increase in computing moving forward? And, I mean, do you think that this new stuff that you're starting to do tests on is going to power a fundamental shift in computing? Or is it just going to be more consolidation, better power consumption? Do you think there's an inflection point coming? What do you think? >> That's a great question. That's a hard one to answer. I mean, it's probably a little bit of both, 'cause certainly there will be better consolidation, right? But I think that, you know, the systems, it works both ways. It just allows you to do more with less, right? And you can go either direction, you can do what you're doing now on fewer machines, you know, and get better value for it, or reduce your footprint. Or you can go the other way and say, wow, this lets us add more machines into the mix and take our our level of performance from here to here, right? So it just depends on what your focus is. Certainly with, with areas like, you know, HPC and AI and ML, having the ability to expand what you already are capable of by adding more machines that can do more is going to be your main concern. But if you're more like a small to medium sized business and the opportunity to do what you were doing on, on a much smaller footprint and for lower costs, that's really your goal, right? So I think you can use this in either direction and it should, should pay back in a lot of dividends. >> Yeah. Thanks for your thoughts. It's an interesting subject moving forward. You know, sometimes it's easy to get lost in the minutiae of the bits and bites and bobs of all the components we're studying, but they're powering something that that's going to effect effectively all of humanity as we move forward. So what else do we need to consider when it comes to what you've just validated in the virtualization testing? Anything else, anything we left out? >> I think we hit all the key points, or most of them it's, you know, really, it's just keeping in mind that it's all about the full system, the components not- you know, the processor is a obviously a key, but just removing blockages, right? Freeing up, getting rid of latency, improving bandwidth, all these things come to play. And then the power performance, as I said, I know I keep coming back to that but you know, we just, and a lot of what we work on, we just see that businesses, that's a really big concern for businesses and finding efficiency, right? And especially in an age of constrained budgets, that's a big deal. So, it's really important to have that power performance ratio. And that's one of the key things we saw that stood out to us in, in some of these benchmarks, so. >> Well, it's a big deal for me. >> It's all good. >> Yeah, I live in California and I know exactly how much I pay for a kilowatt hour of electricity. >> I bet, yeah. >> My friends in other places don't even know. So I totally understand the power constraint question. >> Yeah, it's not going to get better, so, anything you can do there, right? >> Yeah. Well Evan, this has been great. Thanks for sharing the results that Prowess has come up with, third party validation that, you know, even without the latest and greatest components in all categories, Dell PowerEdge servers are able to set world records. And I anticipate that those world records will be broken in 2023 and I expect that Prowess will be part of that process, So Thanks for that. For the rest of us- >> (indistinct) >> Here at theCUBE, I want to thank you for joining us. Stay tuned for continuing coverage of AMD's fourth generation EPYC launch, for myself and for Evan Touger. Thanks so much for joining us. (upbeat music)

>>Hey everyone, welcome to the Cube's coverage of AWS Reinvent 2022. Lisa Martin here with you with Subaru ier, one of our alumni who's now the CEO of Aerospike. Sabu. Great to have you on the program. Thank you for joining us. >>Great as always, to be on the cube. Luisa, good to meet you. >>So, you know, every company these days has got to be a data company, whether it's a retailer, a manufacturer, a grocer, a automotive company. But for a lot of companies, data is underutilized, yet a huge asset that is value added. Why do you think companies are struggling so much to make data a value added asset? >>Well, you know, we, we see this across the board when I talk to customers and prospects. There's a desire from the business and from it actually to leverage data to really fuel newer applications, newer services, newer business lines, if you will, for companies. I think the struggle is one, I think one the, you know, the plethora of data that is created, you know, surveys say that over the next three years data is gonna be, you know, by 2025, around 175 zetabytes, right? A hundred and zetabytes of data is gonna be created. And that's really a, a, a growth of north of 30% year over year. But the more important, and the interesting thing is the real time component of that data is actually growing at, you know, 35% cagr. And what enterprises desire is decisions that are made in real time or near real time. >>And a lot of the challenges that do exist today is that either the infrastructure that enterprises have in place was never built to actually manipulate data in real time. The second is really the ability to actually put something in place which can handle spikes yet be cost efficient if you'll, so you can build for really peak loads, but then it's very expensive to operate that particular service at normal loads. So how do you build something which actually works for you, for both you, both users, so to speak? And the last point that we see out there is even if you're able to, you know, bring all that data, you don't have the processing capability to run through that data. So as a result, most enterprises struggle with one, capturing the data, you know, making decisions from it in real time and really operating it at the cost point that they need to operate it at. >>You know, you bring up a great point with respect to real time data access. And I think one of the things that we've learned the last couple of years is that access to real time data, it's not a nice to have anymore. It's business critical for organizations in any industry. Talk about that as one of the challenges that organizations are facing. >>Yeah. When, when, when we started Aerospike, right when the company started, it started with the premise that data is gonna grow, number one, exponentially. Two, when applications open up to the internet, there's gonna be a flood of users and demands on those applications. And that was true primarily when we started the company in the ad tech vertical. So ad tech was the first vertical where there was a lot of data both on the supply side and the demand side from an inventory of ads that were available. And on the other hand, they had like microseconds or milliseconds in which they could make a decision on which ad to put in front of you and I so that we would click or engage with that particular ad. But over the last three to five years, what we've seen is as digitization has actually permeated every industry out there, the need to harness data in real time is pretty much present in every industry. >>Whether that's retail, whether that's financial services, telecommunications, e-commerce, gaming and entertainment. Every industry has a desire. One, the innovative companies, the small companies rather, are innovating at a pace and standing up new businesses to compete with the larger companies in each of these verticals. And the larger companies don't wanna be left behind. So they're standing up their own competing services or getting into new lines of business that really harness and are driven by real time data. So this compelling pressures, one, the customer exp you know, customer experience is paramount and we as customers expect answers in, you know, an instant in real time. And on the other hand, the way they make decisions is based on a large data set because you know, larger data sets actually propel better decisions. So there's competing pressures here, which essentially drive the need. One from a business perspective, two from a customer perspective to harness all of this data in real time. So that's what's driving an inces need to actually make decisions in real or near real time. >>You know, I think one of the things that's been in short supply over the last couple of years is patients we do expect as consumers, whether we're in our business lives, our personal lives that we're going to be getting, be given information and data that's relevant, it's personal to help us make those real time decisions. So having access to real time data is really business critical for organizations across any industries. Talk about some of the main capabilities that modern data applications and data platforms need to have. What are some of the key capabilities of a modern data platform that need to be delivered to meet demanding customer expectations? >>So, you know, going back to your initial question Lisa, around why is data really a high value but underutilized or underleveraged asset? One of the reasons we see is a lot of the data platforms that, you know, some of these applications were built on have been then around for a decade plus and they were never built for the needs of today, which is really driving a lot of data and driving insight in real time from a lot of data. So there are four major capabilities that we see that are essential ingredients of any modern data platform. One is really the ability to, you know, operate at unlimited scale. So what we mean by that is really the ability to scale from gigabytes to even petabytes without any degradation in performance or latency or throughput. The second is really, you know, predictable performance. So can you actually deliver predictable performance as your data size grows or your throughput grows or your concurrent user on that application of service grows? >>It's really easy to build an application that operates at low scale or low throughput or low concurrency, but performance usually starts degrading as you start scaling one of these attributes. The third thing is the ability to operate and always on globally resilient application. And that requires a, a really robust data platform that can be up on a five, nine basis globally, can support global distribution because a lot of these applications have global users. And the last point is, goes back to my first answer, which is, can you operate all of this at a cost point? Which is not prohibitive, but it makes sense from a TCO perspective. Cuz a lot of times what we see is people make choices of data platforms and as ironically their service or applications become more successful and more users join their journey, the revenue starts going up, the user base starts going up, but the cost basis starts crossing over the revenue and they're losing money on the service, ironically, as the service becomes more popular. So really unlimited scale, predictable performance always on, on a globally resilient basis and low tco. These are the four essential capabilities of any modern data platform. >>So then talk to me with those as the four main core functionalities of a modern data platform. How does aerospace deliver that? >>So we were built, as I said, from the from day one to operate at unlimited scale and deliver predictable performance. And then over the years as we work with customers, we build this incredible high availability capability which helps us deliver the always on, you know, operations. So we have customers who are, who have been on the platform 10 years with no downtime for example, right? So we are talking about an amazing continuum of high availability that we provide for customers who operate these, you know, globally resilient services. The key to our innovation here is what we call the hybrid memory architecture. So, you know, going a little bit technically deep here, essentially what we built out in our architecture is the ability on each node or each server to treat a bank of SSDs or solid state devices as essentially extended memory. So you're getting memory performance, but you're accessing these SSDs, you're not paying memory prices, but you're getting memory performance as a result of that. >>You can attach a lot more data to each node or each server in your distributed cluster. And when you kind of scale that across basically a distributed cluster you can do with aerospike, the same things at 60 to 80% lower server count and as a result 60 to 80% lower TCO compared to some of the other options that are available in the market. Then basically, as I said, that's the key kind of starting point to the innovation. We layer around capabilities like, you know, replication change, data notification, you know, synchronous and asynchronous replication. The ability to actually stretch a single cluster across multiple regions. So for example, if you're operating a global service, you can have a single aerospace cluster with one node in San Francisco, one northern New York, another one in London. And this would be basically seamlessly operating. So that, you know, this is strongly consistent. >>Very few no SQL data platforms are strongly consistent or if they are strongly consistent, they will actually suffer performance degradation. And what strongly consistent means is, you know, all your data is always available, it's guaranteed to be available, there is no data lost anytime. So in this configuration that I talked about, if the node in London goes down, your application still continues to operate, right? Your users see no kind of downtime and you know, when London comes up, it rejoins the cluster and everything is back to kind of the way it was before, you know, London left the cluster so to speak. So the op, the ability to do this globally resilient, highly available kind of model is really, really powerful. A lot of our customers actually use that kind of a scenario and we offer other deployment scenarios from a higher availability perspective. So everything starts with HMA or hybrid memory architecture and then we start building out a lot of these other capabilities around the platform. >>And then over the years, what our customers have guided us to do is as they're putting together a modern kind of data infrastructure, we don't live in a silo. So aerospace gets deployed with other technologies like streaming technologies or analytics technologies. So we built connectors into Kafka, pulsar, so that as you're ingesting data from a variety of data sources, you can ingest them at very high ingest speeds and store them persistently into Aerospike. Once the data is in Aerospike, you can actually run spark jobs across that data in a, in a multithreaded parallel fashion to get really insight from that data at really high, high throughput and high speed, >>High throughput, high speed, incredibly important, especially as today's landscape is increasingly distributed. Data centers, multiple public clouds, edge IOT devices, the workforce embracing more and more hybrid these days. How are you ex helping customers to extract more value from data while also lowering costs? Go into some customer examples cause I know you have some great ones. >>Yeah, you know, I think we have, we have built an amazing set of customers and customers actually use us for some really mission critical applications. So, you know, before I get into specific customer examples, let me talk to you about some of kind of the use cases which we see out there. We see a lot of aerospace being used in fraud detection. We see us being used in recommendations and since we use get used in customer data profiles or customer profiles, customer 360 stores, you know, multiplayer gaming and entertainment, these are kind of the repeated use case digital payments. We power most of the digital payment systems across the globe. Specific example from a, from a specific example perspective, the first one I would love to talk about is PayPal. So if you use PayPal today, then you know when you actually paying somebody your transaction is, you know, being sent through aero spike to really decide whether this is a fraudulent transaction or not. >>And when you do that, you know, you and I as a customer not gonna wait around for 10 seconds for PayPal to say yay or me, we expect, you know, the decision to be made in an instant. So we are powering that fraud detection engine at PayPal for every transaction that goes through PayPal before us, you know, PayPal was missing out on about 2% of their SLAs, which was essentially millions of dollars, which they were losing because, you know, they were letting transactions go through and taking the risk that it, it's not a fraudulent transaction with the aerospace. They can now actually get a much better sla and the data set on which they compute the fraud score has gone up by, you know, several factors. So by 30 x if you will. So not only has the data size that is powering the fraud engine actually grown up 30 x with Aerospike. Yeah. But they're actually making decisions in an instant for, you know, 99.95% of their transactions. So that's, >>And that's what we expect as consumers, right? We want to know that there's fraud detection on the swipe regardless of who we're interacting with. >>Yes. And so that's a, that's a really powerful use case and you know, it's, it's a great customer, great customer success story. The other one I would talk about is really Wayfair, right? From retail and you know, from e-commerce. So everybody knows Wayfair global leader in really, you know, online home furnishings and they use us to power their recommendations engine and you know, it's basically if you're purchasing this, people who bought this but also bought these five other things, so on and so forth, they have actually seen the card size at checkout go by up to 30% as a result of actually powering their recommendations in G by through Aerospike. And they, they were able to do this by reducing the server count by nine x. So on one ninth of the servers that were there before aerospace, they're now powering their recommendation engine and seeing card size checkout go up by 30%. Really, really powerful in terms of the business outcome and what we are able to, you know, drive at Wayfair >>Hugely powerful as a business outcome. And that's also what the consumer wants. The consumer is expecting these days to have a very personalized, relevant experience that's gonna show me if I bought this, show me something else that's related to that. We have this expectation that needs to be really fueled by technology. >>Exactly. And you know, another great example you asked about, you know, customer stories, Adobe, who doesn't know Adobe, you know, they, they're on a, they're on a mission to deliver the best customer experience that they can and they're talking about, you know, great customer 360 experience at scale and they're modernizing their entire edge compute infrastructure to support this. With Aerospike going to Aerospike, basically what they have seen is their throughput go up by 70%, their cost has been reduced by three x. So essentially doing it at one third of the cost while their annual data growth continues at, you know, about north of 30%. So not only is their data growing, they're able to actually reduce their cost to actually deliver this great customer experience by one third to one third and continue to deliver great customer 360 experience at scale. Really, really powerful example of how you deliver Customer 360 in a world which is dynamic and you know, on a dataset which is constantly growing at north, north of 30% in this case. >>Those are three great examples, PayPal, Wayfair, Adobe talking about, especially with Wayfair when you talk about increasing their cart checkout sizes, but also with Adobe increasing throughput by over 70%. I'm looking at my notes here. While data is growing at 32%, that's something that every organization has to contend with data growth is continuing to scale and scale and scale. >>Yep. I, I'll give you a fun one here. So, you know, you may not have heard about this company, it's called Dream 11 and it's a company based out of India, but it's a very, you know, it's a fun story because it's the world's largest fantasy sports platform and you know, India is a nation which is cricket crazy. So you know, when, when they have their premier league going on, you know, there's millions of users logged onto the dream alone platform building their fantasy lead teams and you know, playing on that particular platform, it has a hundred million users, a hundred million plus users on the platform, 5.5 million concurrent users and they have been growing at 30%. So they are considered a, an amazing success story in, in terms of what they have accomplished and the way they have architected their platform to operate at scale. And all of that is really powered by aerospace where think about that they are able to deliver all of this and support a hundred million users, 5.5 million concurrent users all with you know, 99 plus percent of their transactions completing in less than one millisecond. Just incredible success story. Not a brand that is you know, world renowned but at least you know from a what we see out there, it's an amazing success story of operating at scale. >>Amazing success story, huge business outcomes. Last question for you as we're almost out of time is talk a little bit about Aerospike aws, the partnership GRAVITON two better together. What are you guys doing together there? >>Great partnership. AWS has multiple layers in terms of partnerships. So you know, we engage with AWS at the executive level. They plan out, really roll out of new instances in partnership with us, making sure that, you know, those instance types work well for us. And then we just released support for Aerospike on the graviton platform and we just announced a benchmark of Aerospike running on graviton on aws. And what we see out there is with the benchmark, a 1.6 x improvement in price performance and you know, about 18% increase in throughput while maintaining a 27% reduction in cost, you know, on graviton. So this is an amazing story from a price performance perspective, performance per wat for greater energy efficiencies, which basically a lot of our customers are starting to kind of talk to us about leveraging this to further meet their sustainability target. So great story from Aero Aerospike and aws, not just from a partnership perspective on a technology and an executive level, but also in terms of what joint outcomes we are able to deliver for our customers. >>And it sounds like a great sustainability story. I wish we had more time so we would talk about this, but thank you so much for talking about the main capabilities of a modern data platform, what's needed, why, and how you guys are delivering that. We appreciate your insights and appreciate your time. >>Thank you very much. I mean, if, if folks are at reinvent next week or this week, come on and see us at our booth. We are in the data analytics pavilion. You can find us pretty easily. Would love to talk to you. >>Perfect. We'll send them there. So Ira, thank you so much for joining me on the program today. We appreciate your insights. >>Thank you Lisa. >>I'm Lisa Martin. You're watching The Cubes coverage of AWS Reinvent 2022. Thanks for watching.